2024-01-31T04:44:44.244 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-01-31T04:44:44.244 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464 --verbose --owner scheduled_vshankar@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm} -- /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/orig.config.yaml 2024-01-31T04:44:44.286 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-31T04:44:44.360 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464 branch: wip-vshankar-testing-20240125.044502 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm} email: vshankar@redhat.com first_in_suite: false job_id: '7539464' kernel: flavor: default kdb: true sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240125.044502 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 661946e7044723130c03627323f6a879a9848cb5 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 661946e7044723130c03627323f6a879a9848cb5 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240125.044502 sha1: 661946e7044723130c03627323f6a879a9848cb5 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9619 sha1: 661946e7044723130c03627323f6a879a9848cb5 sleep_before_teardown: 0 subset: 293/1024 suite: fs suite_branch: wip-vshankar-testing-20240125.044502 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 661946e7044723130c03627323f6a879a9848cb5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXT2G+E8eNejApYk1lrMi8RXrp+DyrOrZfxPpm2gF18EHfuxOlRhwe49k84Q9yDGWvuDtpmI+rBg7+KZtiC31g= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNd0o8lWnbxEkjgzyJ5bQhMY9uRxoEYyIwGAmnnKRxtfWCkzpMyC8jCWqf6zSkFg8YwYsJ2+rBvJ3uOL+Q143XA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-01-31_04:15:12 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-31T04:44:44.360 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa; will attempt to use it 2024-01-31T04:44:44.361 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks 2024-01-31T04:44:44.361 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-31T04:44:44.363 INFO:teuthology.task.internal:Checking packages... 2024-01-31T04:44:44.386 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '661946e7044723130c03627323f6a879a9848cb5' 2024-01-31T04:44:44.386 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-31T04:44:44.386 INFO:teuthology.packaging:ref: None 2024-01-31T04:44:44.387 INFO:teuthology.packaging:tag: None 2024-01-31T04:44:44.387 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240125.044502 2024-01-31T04:44:44.387 INFO:teuthology.packaging:sha1: 661946e7044723130c03627323f6a879a9848cb5 2024-01-31T04:44:44.387 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240125.044502 2024-01-31T04:44:44.608 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-885.g661946e7 2024-01-31T04:44:44.609 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-31T04:44:44.616 INFO:teuthology.task.internal:no buildpackages task found 2024-01-31T04:44:44.616 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-31T04:44:44.634 INFO:teuthology.task.internal:Saving configuration 2024-01-31T04:44:44.651 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-31T04:44:44.657 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T04:44:44.679 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:39:57.227063', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXT2G+E8eNejApYk1lrMi8RXrp+DyrOrZfxPpm2gF18EHfuxOlRhwe49k84Q9yDGWvuDtpmI+rBg7+KZtiC31g='} 2024-01-31T04:44:44.705 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:39:57.225617', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNd0o8lWnbxEkjgzyJ5bQhMY9uRxoEYyIwGAmnnKRxtfWCkzpMyC8jCWqf6zSkFg8YwYsJ2+rBvJ3uOL+Q143XA='} 2024-01-31T04:44:44.706 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-31T04:44:44.712 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-31T04:44:44.712 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-31T04:44:44.712 INFO:teuthology.run_tasks:Running task console_log... 2024-01-31T04:44:44.782 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fadc9b90160>, signals=[15]) 2024-01-31T04:44:44.782 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-31T04:44:44.789 INFO:teuthology.task.internal:Opening connections... 2024-01-31T04:44:44.789 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T04:44:44.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:44:44.881 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T04:44:44.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:44:44.971 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-31T04:44:44.978 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-01-31T04:44:44.996 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-01-31T04:44:44.996 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-01-31T04:44:45.053 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-01-31T04:44:45.053 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-01-31T04:44:45.053 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-01-31T04:44:45.053 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-01-31T04:44:45.053 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-01-31T04:44:45.053 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-31T04:44:45.054 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-31T04:44:45.055 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-01-31T04:44:45.077 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-01-31T04:44:45.095 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-01-31T04:44:45.095 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-01-31T04:44:45.152 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-31T04:44:45.153 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-31T04:44:45.153 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-01-31T04:44:45.177 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-31T04:44:45.186 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-31T04:44:45.192 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-31T04:44:45.192 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-01-31T04:44:45.195 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-01-31T04:44:45.210 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-31T04:44:45.217 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-31T04:44:45.217 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-01-31T04:44:45.251 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-01-31T04:44:45.546 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-31T04:44:45.601 INFO:teuthology.run_tasks:Running task kernel... 2024-01-31T04:44:45.616 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'} 2024-01-31T04:44:45.616 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'}}, timeout 300 2024-01-31T04:44:45.617 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-31T04:44:45.617 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-31T04:44:45.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T04:44:45.634 DEBUG:teuthology.task.kernel:sha1 for mon.a is cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:44:45.634 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "cd42140e628558a450ba1d39a9d0835a2fa62cda"... 2024-01-31T04:44:45.634 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-01-31T04:44:45.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T04:44:45.635 DEBUG:teuthology.task.kernel:sha1 for mon.b is cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:44:45.635 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "cd42140e628558a450ba1d39a9d0835a2fa62cda"... 2024-01-31T04:44:45.635 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-01-31T04:44:45.691 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-361.el9.x86_64 2024-01-31T04:44:45.691 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:44:45.691 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-31T04:44:45.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:44:45.695 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-361.el9.x86_64 2024-01-31T04:44:45.695 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:44:45.695 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-31T04:44:45.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:44:45.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum upgrade -y linux-firmware 2024-01-31T04:44:45.906 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum upgrade -y linux-firmware 2024-01-31T04:44:48.006 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 15 MB/s | 7.8 MB 00:00 2024-01-31T04:44:48.184 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 13 MB/s | 7.8 MB 00:00 2024-01-31T04:44:49.564 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 8.2 MB/s | 2.5 MB 00:00 2024-01-31T04:44:49.868 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.5 MB 00:00 2024-01-31T04:44:51.911 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-01-31T04:44:52.980 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.0 MB 00:02 2024-01-31T04:44:55.367 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 7.8 MB/s | 19 MB 00:02 2024-01-31T04:44:55.866 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-01-31T04:45:00.543 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 5.9 MB 00:01 2024-01-31T04:45:01.223 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 5.9 MB 00:01 2024-01-31T04:45:01.960 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 15 kB 00:00 2024-01-31T04:45:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2024-01-31T04:45:03.713 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-01-31T04:45:04.092 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2024-01-31T04:45:08.727 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-01-31T04:45:09.067 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-01-31T04:45:10.918 INFO:teuthology.orchestra.run.smithi188.stderr:Modular dependency problems: 2024-01-31T04:45:10.918 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-01-31T04:45:10.918 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 1: conflicting requests 2024-01-31T04:45:10.918 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.918 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 2: conflicting requests 2024-01-31T04:45:10.918 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 3: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 4: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 5: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 6: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 7: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 8: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 9: conflicting requests 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.919 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 10: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 11: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 12: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 13: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 14: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 15: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 16: conflicting requests 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.920 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 17: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 18: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 19: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 20: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 21: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 22: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 23: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 24: conflicting requests 2024-01-31T04:45:10.921 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 25: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 26: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 27: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 28: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 29: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 30: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 31: conflicting requests 2024-01-31T04:45:10.922 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 32: conflicting requests 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 33: conflicting requests 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 34: conflicting requests 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 35: conflicting requests 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 36: conflicting requests 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 37: conflicting requests 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.923 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 39: conflicting requests 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.924 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.925 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.926 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T04:45:10.927 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr:Modular dependency problems: 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 1: conflicting requests 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 2: conflicting requests 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 3: conflicting requests 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.305 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 4: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 5: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 6: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 7: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 8: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 9: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 10: conflicting requests 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.306 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 11: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 12: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 13: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 14: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 15: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 16: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 17: conflicting requests 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.307 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 18: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 19: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 20: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 21: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 22: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 23: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 24: conflicting requests 2024-01-31T04:45:11.308 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 25: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 26: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 27: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 28: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 29: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 30: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 31: conflicting requests 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.309 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 32: conflicting requests 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 33: conflicting requests 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 34: conflicting requests 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 35: conflicting requests 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 36: conflicting requests 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 37: conflicting requests 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.310 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 39: conflicting requests 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.311 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.312 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.313 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T04:45:11.436 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-01-31T04:45:11.448 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-01-31T04:45:11.448 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-01-31T04:45:11.448 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-01-31T04:45:11.448 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-01-31T04:45:11.448 INFO:teuthology.orchestra.run.smithi188.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 1 Package 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 354 M 2024-01-31T04:45:11.449 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-01-31T04:45:11.851 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-01-31T04:45:11.861 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-01-31T04:45:11.861 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-01-31T04:45:11.861 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-01-31T04:45:11.861 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-01-31T04:45:11.861 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware noarch 20240111-121.gitb3132c18.el8 CentOS-BaseOS 354 M 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 1 Package 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 354 M 2024-01-31T04:45:11.862 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-01-31T04:45:24.317 INFO:teuthology.orchestra.run.smithi097.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 28 MB/s | 354 MB 00:12 2024-01-31T04:45:24.318 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-01-31T04:45:24.318 INFO:teuthology.orchestra.run.smithi097.stdout:Total 28 MB/s | 354 MB 00:12 2024-01-31T04:45:24.319 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-01-31T04:45:24.404 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-01-31T04:45:24.404 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-01-31T04:45:24.443 INFO:teuthology.orchestra.run.smithi188.stdout:linux-firmware-20240111-121.gitb3132c18.el8.noa 27 MB/s | 354 MB 00:12 2024-01-31T04:45:24.444 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-01-31T04:45:24.444 INFO:teuthology.orchestra.run.smithi188.stdout:Total 27 MB/s | 354 MB 00:12 2024-01-31T04:45:24.444 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-01-31T04:45:24.571 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-01-31T04:45:24.572 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-01-31T04:45:25.903 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-01-31T04:45:25.904 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-01-31T04:45:26.221 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-01-31T04:45:26.222 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-01-31T04:45:47.152 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-01-31T04:45:47.408 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-01-31T04:45:47.512 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-31T04:45:47.536 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-01-31T04:45:47.795 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-01-31T04:45:47.896 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-31T04:45:55.044 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-31T04:45:55.044 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-01-31T04:45:55.459 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-31T04:45:55.459 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : linux-firmware-20240111-121.gitb3132c18.el8.noarch 1/2 2024-01-31T04:45:55.478 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-31T04:45:55.478 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:45:55.479 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-01-31T04:45:55.479 INFO:teuthology.orchestra.run.smithi097.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-01-31T04:45:55.479 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:45:55.479 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-01-31T04:45:55.787 INFO:teuthology.task.kernel:Downloading kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on mon.a... 2024-01-31T04:45:55.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:45:55.902 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-31T04:45:55.902 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:45:55.902 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-01-31T04:45:55.902 INFO:teuthology.orchestra.run.smithi188.stdout: linux-firmware-20240111-121.gitb3132c18.el8.noarch 2024-01-31T04:45:55.902 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:45:55.902 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-01-31T04:45:55.937 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ 2024-01-31T04:45:55.937 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_gcd42140e6285-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-31T04:45:55.939 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2024-01-31T04:45:56.194 INFO:teuthology.task.kernel:Downloading kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on mon.b... 2024-01-31T04:45:56.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-01-31T04:45:56.420 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ 2024-01-31T04:45:56.495 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_gcd42140e6285-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-31T04:45:56.497 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi188.front.sepia.ceph.com to complete... 2024-01-31T04:46:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31 04:46:14 URL:https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/kernel-6.7.0_gcd42140e6285-1.x86_64.rpm [330265785/330265785] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-31T04:46:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2024-01-31 04:46:14-- 2024-01-31T04:46:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 18s 2024-01-31T04:46:14.011 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 315M in 18s (17.7 MB/s) 2024-01-31T04:46:14.014 INFO:teuthology.task.kernel:Installing kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on mon.a... 2024-01-31T04:46:14.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-31T04:46:15.910 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying... ######################################## 2024-01-31T04:46:16.047 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing... ######################################## 2024-01-31T04:46:16.127 INFO:teuthology.orchestra.run.smithi097.stdout:Updating / installing... 2024-01-31T04:46:23.609 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-6.7.0_gcd42140e6285-1 ######################################## 2024-01-31T04:46:38.573 INFO:teuthology.orchestra.run.smithi097.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-31T04:47:00.766 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-31T04:47:00.766 DEBUG:teuthology.orchestra.run.smithi097:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-31T04:47:00.856 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/System.map-6.7.0-gcd42140e6285 2024-01-31T04:47:00.856 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/config-6.7.0-gcd42140e6285 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/boot/vmlinuz-6.7.0-gcd42140e6285 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-31T04:47:00.857 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-31T04:47:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ansi_cprng.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/anubis.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/arc4.ko 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx 2024-01-31T04:47:00.859 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_memcpy.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_pq.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_tx.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_xor.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/raid6test.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authenc.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authencesn.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blake2b_generic.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_common.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_generic.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/camellia_generic.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast5_generic.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast6_generic.ko 2024-01-31T04:47:00.860 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast_common.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ccm.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cmac.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crc32_generic.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_engine.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_user.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cts.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/des_generic.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/dh_generic.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/echainiv.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/essiv.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/fcrypt.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/khazad.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4.ko 2024-01-31T04:47:00.861 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4hc.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/md4.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/michael_mic.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcbc.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcrypt.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/rmd160.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/seed.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/serpent_generic.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tcrypt.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tea.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_common.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_generic.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/vmac.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/wp512.ko 2024-01-31T04:47:00.862 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xcbc.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xor.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xxhash_generic.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_pad.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/custom_method.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/ec_sys.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/platform_profile.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbs.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbshc.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/video.ko 2024-01-31T04:47:00.863 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/acard-ahci.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ahci_platform.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ata_generic.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/libahci_platform.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_acpi.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ali.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_amd.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_artop.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atiixp.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atp867x.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd640.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd64x.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cypress.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_efar.ko 2024-01-31T04:47:00.864 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt366.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt37x.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it8213.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it821x.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_jmicron.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_marvell.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_mpiix.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_netcell.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ninja32.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87410.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87415.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_oldpiix.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_opti.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_optidma.ko 2024-01-31T04:47:00.865 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pcmcia.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_piccolo.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_rdc.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sch.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_serverworks.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sil680.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sis.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sl82c105.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_triflex.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_via.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pdc_adma.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_inic162x.ko 2024-01-31T04:47:00.866 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_mv.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_nv.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_promise.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_qstor.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil24.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sis.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_svw.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sx4.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_uli.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_via.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_vsc.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay/ks0108.ko 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base 2024-01-31T04:47:00.867 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma/bcma.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe/aoe.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/brd.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd/drbd.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/floppy.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/loop.ko 2024-01-31T04:47:00.868 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/nbd.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk/null_blk.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/pktcdvd.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/rbd.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/virtio_blk.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkfront.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram/zram.ko 2024-01-31T04:47:00.869 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hangcheck-timer.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/via-rng.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-31T04:47:00.870 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/lp.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave/mwave.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ppdev.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tlclk.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/uv_mmtimer.ko 2024-01-31T04:47:00.871 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/virtio_console.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp.ko 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel 2024-01-31T04:47:00.872 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-aes.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-sha.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca/dca.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma 2024-01-31T04:47:00.873 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/amd64_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/e752x_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/edac_mce_amd.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3000_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3200_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5100_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5400_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7300_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7core_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i82975x_edac.ko 2024-01-31T04:47:00.874 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/ie31200_edac.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/sb_edac.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/x38_edac.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/edd.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-ich.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-appleir.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-aureal.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-axff.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-dr.ko 2024-01-31T04:47:00.875 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elecom.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elo.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-emsff.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gaff.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gt683r.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gyration.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtekff.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-hyperv.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-icade.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-keytouch.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-kye.ko 2024-01-31T04:47:00.876 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lcpower.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-led.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lenovo.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-multitouch.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-ortek.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-penmount.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-petalynx.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-picolcd.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-pl.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-plantronics.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-primax.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-rmi.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-31T04:47:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-common.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-saitek.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-samsung.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-31T04:47:00.878 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sjoy.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sony.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-speedlink.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-steelseries.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sunplus.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tivo.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tmff.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-topseed.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-twinhan.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-uclogic.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-waltop.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-wiimote.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-xinmo.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zpff.ko 2024-01-31T04:47:00.879 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zydacron.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/uhid.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/wacom.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_balloon.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_utils.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_vmbus.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru3.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7414.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7418.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adc128d818.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1025.ko 2024-01-31T04:47:00.880 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1026.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1029.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1031.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm9240.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ads7828.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7410.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7411.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7462.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7470.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7475.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7x10.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/amc6821.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/applesmc.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asb100.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asc7621.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/atxp1.ko 2024-01-31T04:47:00.881 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/coretemp.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dme1737.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds1621.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds620.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc1403.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc6w201.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71805f.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71882fg.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f75375s.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fam15h_power.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fschmd.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g760a.ko 2024-01-31T04:47:00.882 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g762.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl518sm.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl520sm.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5500_temp.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5k_amb.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmaem.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmpex.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina209.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina2xx.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/it87.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k10temp.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k8temp.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lineage-pem.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm63.ko 2024-01-31T04:47:00.883 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm73.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm75.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm77.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm78.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm80.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm83.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm85.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm87.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm90.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm92.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm93.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95234.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95241.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95245.ko 2024-01-31T04:47:00.884 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc2945.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4151.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4215.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4222.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4245.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4260.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4261.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max16065.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1619.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1668.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max197.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6639.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6650.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6697.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/mcp3021.ko 2024-01-31T04:47:00.885 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6683.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775-core.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct7802.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87360.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87427.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pcf8591.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-31T04:47:00.886 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/powr1220.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5627.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5636.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht15.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht21.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/shtc1.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sis5595.ko 2024-01-31T04:47:00.887 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47b397.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m1.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m192.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/thmc50.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp102.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp103.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp401.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp421.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via-cputemp.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via686a.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt1211.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt8231.ko 2024-01-31T04:47:00.888 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627ehf.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627hf.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83781d.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83791d.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83792d.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83793.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83795.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l785ts.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l786ng.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses 2024-01-31T04:47:00.889 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-31T04:47:00.890 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-dev.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-mux.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-smbus.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-stub.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-31T04:47:00.891 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_core.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca 2024-01-31T04:47:00.892 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-31T04:47:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/gameport.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/joydev.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/appletouch.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/bcm5974.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/cyapatp.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/sermouse.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-31T04:47:00.894 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/altera_ps2.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/arc_ps2.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/serio_raw.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/sparse-keymap.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-blinkm.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lm3530.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lp3944.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-ss4200.ko 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger 2024-01-31T04:47:00.895 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache/bcache.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-bio-prison.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache-smq.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-crypt.ko 2024-01-31T04:47:00.896 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-delay.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-flakey.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-log-userspace.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-multipath.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-queue-length.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-raid.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-round-robin.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-service-time.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-switch.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-thin-pool.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-verity.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/faulty.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/linear.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/multipath.ko 2024-01-31T04:47:00.897 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid0.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid1.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid10.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid456.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptbase.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptctl.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptfc.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptlan.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptsas.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptscsih.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptspi.ko 2024-01-31T04:47:00.898 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_ich.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_sch.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/mfd-core.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/sm501.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/viperboard.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/vx855.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/wl1273-core.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds9802als.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds990x.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/bh1770glc.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710/cb710.ko 2024-01-31T04:47:00.899 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/at24.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/max6875.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/enclosure.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/hpilo.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29003.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29020.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-me.ko 2024-01-31T04:47:00.900 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-txe.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_7xx1.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_core.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tsl2550.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_balloon.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net 2024-01-31T04:47:00.901 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding/bonding.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dummy.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/eql.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-31T04:47:00.902 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera 2024-01-31T04:47:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-31T04:47:00.904 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-31T04:47:00.905 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip 2024-01-31T04:47:00.906 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dnet.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ethoc.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/fealnx.ko 2024-01-31T04:47:00.907 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-31T04:47:00.908 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/jme.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-31T04:47:00.909 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-31T04:47:00.910 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-31T04:47:00.911 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-31T04:47:00.912 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-31T04:47:00.913 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/geneve.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-31T04:47:00.914 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ifb.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvlan.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvtap.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mii.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/net_failover.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/netconsole.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/nlmon.ko 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs 2024-01-31T04:47:00.915 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/amd.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm87xx.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/broadcom.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/cicada.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/davicom.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/dp83640.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/et1011c.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/icplus.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/lxt.ko 2024-01-31T04:47:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/marvell.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/micrel.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/national.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/phylink.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/qsemi.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/realtek.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/smsc.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/ste10Xp.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/vitesse.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_async.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-31T04:47:00.917 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppoe.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppox.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pptp.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slhc.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slip.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/sungem_phy.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tap.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_random.ko 2024-01-31T04:47:00.918 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tun.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/veth.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/virtio_net.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan/vxlan.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netfront.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb/ntb.ko 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host 2024-01-31T04:47:00.919 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme-core.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_cs.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_pc.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_serial.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/xen-pcifront.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/i82092.ko 2024-01-31T04:47:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/pd6729.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acerhdf.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-31T04:47:00.921 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/hdaps.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel_ips.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi.ko 2024-01-31T04:47:00.922 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps_parport.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/pps_core.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp_kvm.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-31T04:47:00.923 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-em3027.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-31T04:47:00.924 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-max6900.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-31T04:47:00.925 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-x1205.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-9xxx.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-sas.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-xxxx.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/BusLogic.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/a100u2w.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/advansys.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-31T04:47:00.926 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/am53c974.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/atp870u.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa/bfa.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ch.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi 2024-01-31T04:47:00.927 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dc395x.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dmx3191d.ko 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r 2024-01-31T04:47:00.928 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esp_scsi.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic/fnic.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hpsa.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hptiop.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hv_storvsc.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/initio.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ipr.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ips.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci/isci.ko 2024-01-31T04:47:00.929 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc/libfc.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas/libsas.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-31T04:47:00.930 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvumi.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pmcraid.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla1280.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/raid_class.ko 2024-01-31T04:47:00.931 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_debug.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ses.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/st.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/stex.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/virtio_scsi.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-31T04:47:00.932 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/wd719x.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb/ssb.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_file.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_iblock.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_mod.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_pscsi.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_user.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-31T04:47:00.933 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-31T04:47:00.934 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_gsm.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_hdlc.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/nozomi.ko 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250 2024-01-31T04:47:00.935 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/arc_uart.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_aec.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_cif.ko 2024-01-31T04:47:00.936 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_pci_generic.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_sercos3.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-acm.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usblp.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usbtmc.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/mdc800.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/microtek.ko 2024-01-31T04:47:00.937 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/adutux.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/appledisplay.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi26.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi62.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ezusb.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/idmouse.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/iowarrior.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ldusb.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/legousbtower.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usb3503.ko 2024-01-31T04:47:00.938 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usblcd.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/uss720.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/yurex.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy/phy-generic.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/aircable.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ark3116.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ch341.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cp210x.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cyberjack.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-31T04:47:00.939 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/empeg.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_ti.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipaq.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipw.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ir-usb.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mct_u232.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7720.ko 2024-01-31T04:47:00.940 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7840.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/navman.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/omninet.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/opticon.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/option.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/oti6858.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/pl2303.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcaux.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcserial.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/quatech2.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/safe_serial.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/sierra.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ssu100.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/symbolserial.ko 2024-01-31T04:47:00.941 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_debug.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/visor.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/whiteheat.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/uas.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-31T04:47:00.942 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-karma.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/usb-storage.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-31T04:47:00.943 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_net.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_scsi.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/apple_bl.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/lcd.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio.ko 2024-01-31T04:47:00.944 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_balloon.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_mmio.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_ring.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2482.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2490.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-31T04:47:00.945 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/wire.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/hpwdt.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/i6300esb.ko 2024-01-31T04:47:00.946 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ib700wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ibmasr.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it87_wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/machzwd.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/nv_tco.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/softdog.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-31T04:47:00.947 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/via_wdt.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/wdt_pci.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/xen_wdt.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-evtchn.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntalloc.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntdev.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-privcmd.ko 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs 2024-01-31T04:47:00.948 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p/9p.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs/affs.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs/befs.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/binfmt_misc.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs/btrfs.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles/cachefiles.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph/ceph.ko 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda 2024-01-31T04:47:00.949 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda/coda.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs/cramfs.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm/dlm.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs/f2fs.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/fat.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/msdos.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/vfat.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache/fscache.ko 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse 2024-01-31T04:47:00.950 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/cuse.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/fuse.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2/gfs2.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs/hfs.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus/hfsplus.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs/isofs.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs/jfs.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd/lockd.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix/minix.ko 2024-01-31T04:47:00.951 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs/netfs.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfs.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv3.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv4.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/grace.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/nfs_acl.ko 2024-01-31T04:47:00.952 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd/nfsd.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2/nilfs2.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-celtic.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-centeuro.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-croatian.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-cyrillic.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-gaelic.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-greek.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-iceland.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-inuit.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-roman.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-romanian.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-turkish.ko 2024-01-31T04:47:00.953 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1250.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1251.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1255.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp737.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp775.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp850.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp852.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp855.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp857.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp860.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp861.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp862.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp863.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp864.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp865.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp866.ko 2024-01-31T04:47:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp869.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp874.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp932.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp936.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp949.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp950.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_euc-jp.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-1.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-13.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-14.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-15.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-2.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-3.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-4.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-5.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-6.ko 2024-01-31T04:47:00.955 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-7.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-9.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-r.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-ru.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-u.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_utf8.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs 2024-01-31T04:47:00.956 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs/overlay.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore/ramoops.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs/reiserfs.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs/romfs.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client/cifs.ko 2024-01-31T04:47:00.957 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_arc4.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_md4.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs/squashfs.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv/sysv.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf/udf.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs/ufs.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs/xfs.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu/rcutorture.ko 2024-01-31T04:47:00.958 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/torture.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/asn1_encoder.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc-itu-t.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc8.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libarc4.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libdes.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/libcrc32c.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lru_cache.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4_compress.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4hc_compress.ko 2024-01-31T04:47:00.959 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math/cordic.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6/raid6_pq.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_bm.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_fsm.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_kmp.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd/zstd_compress.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm/hwpoison-inject.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/6lowpan.ko 2024-01-31T04:47:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_dest.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_fragment.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_hop.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_mobility.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_routing.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_udp.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/garp.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/mrp.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/p8022.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/psnap.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/stp.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q/8021q.ko 2024-01-31T04:47:00.961 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_fd.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_rdma.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_virtio.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv/batman-adv.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/br_netfilter.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/bridge.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-31T04:47:00.962 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtables.ko 2024-01-31T04:47:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph/libceph.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/failover.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/pktgen.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_diag.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv4.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv6.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver/dns_resolver.ko 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa 2024-01-31T04:47:00.964 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/dsa_core.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_brcm.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_dsa.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_none.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_trailer.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ah4.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/esp4.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/fou.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/gre.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/inet_diag.ko 2024-01-31T04:47:00.965 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_gre.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_tunnel.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_vti.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipcomp.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipip.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-31T04:47:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_bic.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_dctcp.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_diag.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_highspeed.ko 2024-01-31T04:47:00.967 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_htcp.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_hybla.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_illinois.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_lp.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_scalable.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_vegas.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_veno.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_westwood.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_yeah.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tunnel4.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_diag.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_tunnel.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6 2024-01-31T04:47:00.968 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ah6.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/esp6.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/fou6.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_tunnel.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_vti.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ipcomp6.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-31T04:47:00.969 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-31T04:47:00.970 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/sit.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/tunnel6.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key/af_key.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_core.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_eth.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip6.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_netlink.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ppp.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc/llc.ko 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154 2024-01-31T04:47:00.971 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154/mac802154.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls/mpls_gso.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-31T04:47:00.972 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-31T04:47:00.973 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conncount.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-31T04:47:00.974 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_log_syslog.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_irc.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_sip.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_tables.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-31T04:47:00.975 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_log.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_chain_nat.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_compat.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_ct.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fib.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_hash.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_limit.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_log.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_masq.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_nat.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_numgen.ko 2024-01-31T04:47:00.976 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_queue.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_quota.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_redir.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject_inet.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_socket.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_tunnel.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_AUDIT.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CT.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_DSCP.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HL.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HMARK.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-31T04:47:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LED.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LOG.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NETMAP.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFLOG.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_RATEEST.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_SECMARK.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TEE.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TPROXY.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TRACE.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_addrtype.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_bpf.ko 2024-01-31T04:47:00.978 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cgroup.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cluster.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_comment.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connbytes.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlabel.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlimit.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connmark.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_conntrack.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cpu.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dccp.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_devgroup.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dscp.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ecn.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_esp.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hashlimit.ko 2024-01-31T04:47:00.979 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_helper.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hl.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipcomp.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_iprange.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipvs.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_l2tp.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_length.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_limit.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mac.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mark.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_multiport.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nat.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nfacct.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_osf.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_owner.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_physdev.ko 2024-01-31T04:47:00.980 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_pkttype.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_policy.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_quota.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_rateest.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_realm.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_recent.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_sctp.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_set.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_socket.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_state.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_statistic.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_string.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_tcpmss.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_time.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_u32.ko 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink 2024-01-31T04:47:00.981 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink/netlink_diag.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh/nsh.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/openvswitch.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-geneve.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-gre.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-vxlan.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet/af_packet_diag.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_rdma.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_tcp.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_csum.ko 2024-01-31T04:47:00.982 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_gact.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_ipt.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_mirred.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_nat.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_pedit.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_police.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_simple.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_skbedit.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_vlan.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_basic.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_bpf.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_flow.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_fw.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_route.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_u32.ko 2024-01-31T04:47:00.983 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_cmp.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_ipset.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_meta.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_nbyte.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_text.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_u32.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_choke.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_codel.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_drr.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_fq.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_gred.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hfsc.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hhf.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_htb.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_ingress.ko 2024-01-31T04:47:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio_lib.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_multiq.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_netem.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_pie.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_plug.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_prio.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_qfq.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_red.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfb.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfq.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_tbf.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_teql.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp.ko 2024-01-31T04:47:00.985 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp_diag.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/sunrpc.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/diag.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/tipc.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix/unix_diag.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock.ko 2024-01-31T04:47:00.986 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys/trusted.ko 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib/irqbypass.ko 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias.bin 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin 2024-01-31T04:47:00.987 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.alias.bin 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.bin 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.modinfo 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep.bin 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.devname 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.order 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.softdep 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols 2024-01-31T04:47:00.988 INFO:teuthology.orchestra.run.smithi097.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols.bin 2024-01-31T04:47:00.989 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-gcd42140e6285 2024-01-31T04:47:00.989 DEBUG:teuthology.orchestra.run.smithi097:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:postinstall scriptlet (using /bin/sh): 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-gcd42140e6285 -a -r /boot/System.map-6.7.0-gcd42140e6285 ]; then 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:cp /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:cp /boot/System.map-6.7.0-gcd42140e6285 /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:rm -f /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/System.map-6.7.0-gcd42140e6285 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/installkernel 6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-01-31T04:47:01.011 INFO:teuthology.orchestra.run.smithi097.stdout:rm -f /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:new-kernel-pkg --remove 6.7.0-gcd42140e6285 --rminitrd --initrdfile=/boot/initramfs-6.7.0-gcd42140e6285.img 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-install remove 6.7.0-gcd42140e6285 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/update-bootloader --remove 6.7.0-gcd42140e6285 2024-01-31T04:47:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:fi 2024-01-31T04:47:01.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 68.el9 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8264254 2024-01-31T04:47:01.088 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:47:01.089 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-01-31T04:47:01.090 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-31T04:47:01.090 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 6.7.0-gcd42140e6285 2024-01-31T04:47:01.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-31T04:47:01.661 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-01-31T04:47:03.007 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-31T04:47:03.038 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-01-31T04:47:03.045 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-01-31T04:47:03.061 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.MKb8FDiFZl 2024-01-31T04:47:03.061 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MKb8FDiFZl 2024-01-31T04:47:03.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.MKb8FDiFZl 2024-01-31T04:47:03.270 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.MKb8FDiFZl is 6KB 2024-01-31T04:47:03.318 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.MKb8FDiFZl 2024-01-31T04:47:03.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-01-31T04:47:03.397 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-31T04:47:03.397 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-31T04:47:03.397 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-31T04:47:03.397 INFO:teuthology.orchestra.run.smithi097.stdout:15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285.conf 2024-01-31T04:47:03.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285 2024-01-31T04:47:03.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-01-31T04:47:33.549 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-31T04:47:33.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-01-31T04:47:33.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:47:51.868 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-01-31T04:48:00.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-01-31T04:48:00.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:48:03.932 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-01-31T04:48:15.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-01-31T04:48:15.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:48:18.771 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31 04:48:18 URL:https://3.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/9/flavors/default/x86_64/kernel-6.7.0_gcd42140e6285-1.x86_64.rpm [330265785/330265785] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-31T04:48:18.772 INFO:teuthology.orchestra.run.smithi188.stderr:FINISHED --2024-01-31 04:48:18-- 2024-01-31T04:48:18.772 INFO:teuthology.orchestra.run.smithi188.stderr:Total wall clock time: 2m 22s 2024-01-31T04:48:18.772 INFO:teuthology.orchestra.run.smithi188.stderr:Downloaded: 1 files, 315M in 2m 22s (2.22 MB/s) 2024-01-31T04:48:18.775 INFO:teuthology.task.kernel:Installing kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on mon.b... 2024-01-31T04:48:18.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-31T04:48:20.679 INFO:teuthology.orchestra.run.smithi188.stdout:Verifying... ######################################## 2024-01-31T04:48:20.818 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing... ######################################## 2024-01-31T04:48:20.899 INFO:teuthology.orchestra.run.smithi188.stdout:Updating / installing... 2024-01-31T04:48:28.361 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-6.7.0_gcd42140e6285-1 ######################################## 2024-01-31T04:48:41.904 INFO:teuthology.orchestra.run.smithi188.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-31T04:48:47.872 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-01-31T04:48:48.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-01-31T04:48:48.496 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.7.0-gcd42140e6285"... 2024-01-31T04:48:48.496 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-01-31T04:48:48.535 INFO:teuthology.orchestra.run.smithi097.stdout:6.7.0-gcd42140e6285 2024-01-31T04:48:48.536 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-gcd42140e6285 vs 6.7.0-gcd42140e6285 2024-01-31T04:48:48.536 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-31T04:48:48.536 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-31T04:48:49.537 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-31T04:48:49.537 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-31T04:48:49.666 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-01-31T04:48:49.718 DEBUG:teuthology.parallel:result is None 2024-01-31T04:49:04.049 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-31T04:49:04.049 DEBUG:teuthology.orchestra.run.smithi188:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-31T04:49:04.133 INFO:teuthology.orchestra.run.smithi188.stdout:/boot/System.map-6.7.0-gcd42140e6285 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/boot/config-6.7.0-gcd42140e6285 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/boot/vmlinuz-6.7.0-gcd42140e6285 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-31T04:49:04.134 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-31T04:49:04.135 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm.ko 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto 2024-01-31T04:49:04.136 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ansi_cprng.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/anubis.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/arc4.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_memcpy.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_pq.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_tx.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_xor.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/raid6test.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authenc.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authencesn.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blake2b_generic.ko 2024-01-31T04:49:04.137 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_common.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_generic.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/camellia_generic.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast5_generic.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast6_generic.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast_common.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ccm.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cmac.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crc32_generic.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_engine.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_user.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cts.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/des_generic.ko 2024-01-31T04:49:04.138 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/dh_generic.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/echainiv.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/essiv.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/fcrypt.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/khazad.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4hc.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/md4.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/michael_mic.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcbc.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcrypt.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/rmd160.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/seed.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/serpent_generic.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tcrypt.ko 2024-01-31T04:49:04.139 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tea.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_common.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_generic.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/vmac.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/wp512.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xcbc.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xor.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xxhash_generic.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_pad.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/custom_method.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/ec_sys.ko 2024-01-31T04:49:04.140 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/platform_profile.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbs.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbshc.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/video.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/acard-ahci.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ahci_platform.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ata_generic.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/libahci_platform.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_acpi.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ali.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_amd.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_artop.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atiixp.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atp867x.ko 2024-01-31T04:49:04.141 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd640.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd64x.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cypress.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_efar.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt366.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt37x.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it8213.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it821x.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_jmicron.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_marvell.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_mpiix.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_netcell.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ninja32.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87410.ko 2024-01-31T04:49:04.142 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87415.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_oldpiix.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_opti.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_optidma.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pcmcia.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_piccolo.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_rdc.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sch.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_serverworks.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sil680.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sis.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sl82c105.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_triflex.ko 2024-01-31T04:49:04.143 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_via.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pdc_adma.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_inic162x.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_mv.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_nv.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_promise.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_qstor.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil24.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sis.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_svw.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sx4.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_uli.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_via.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_vsc.ko 2024-01-31T04:49:04.144 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay/ks0108.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma/bcma.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe/aoe.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/brd.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd/drbd.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/floppy.ko 2024-01-31T04:49:04.145 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/loop.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/nbd.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk/null_blk.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/pktcdvd.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/rbd.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/virtio_blk.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkfront.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram/zram.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hangcheck-timer.ko 2024-01-31T04:49:04.146 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/via-rng.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/lp.ko 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave 2024-01-31T04:49:04.147 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave/mwave.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ppdev.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tlclk.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/uv_mmtimer.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/virtio_console.ko 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq 2024-01-31T04:49:04.148 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-31T04:49:04.149 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-aes.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-sha.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca/dca.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-31T04:49:04.150 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/amd64_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/e752x_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/edac_mce_amd.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3000_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3200_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5100_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5400_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7300_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7core_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i82975x_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/ie31200_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/sb_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/x38_edac.ko 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware 2024-01-31T04:49:04.151 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/edd.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-ich.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-appleir.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-aureal.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-axff.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-dr.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elecom.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elo.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-emsff.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gaff.ko 2024-01-31T04:49:04.152 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gt683r.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gyration.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtekff.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-hyperv.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-icade.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-keytouch.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-kye.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lcpower.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-led.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lenovo.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-31T04:49:04.153 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-multitouch.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-ortek.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-penmount.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-petalynx.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-picolcd.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-pl.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-plantronics.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-primax.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-rmi.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-common.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-31T04:49:04.154 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-saitek.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-samsung.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sjoy.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sony.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-speedlink.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-steelseries.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sunplus.ko 2024-01-31T04:49:04.155 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tivo.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tmff.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-topseed.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-twinhan.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-uclogic.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-waltop.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-wiimote.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-xinmo.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zpff.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zydacron.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/uhid.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/wacom.ko 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv 2024-01-31T04:49:04.156 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_balloon.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_utils.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_vmbus.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru3.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7414.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7418.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adc128d818.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1025.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1026.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1029.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1031.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm9240.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ads7828.ko 2024-01-31T04:49:04.157 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7410.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7411.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7462.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7470.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7475.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7x10.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/amc6821.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/applesmc.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asb100.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asc7621.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/atxp1.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/coretemp.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-31T04:49:04.158 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dme1737.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds1621.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds620.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc1403.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc6w201.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71805f.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71882fg.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f75375s.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fam15h_power.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fschmd.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g760a.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g762.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl518sm.ko 2024-01-31T04:49:04.159 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl520sm.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5500_temp.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5k_amb.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmaem.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmpex.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina209.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina2xx.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/it87.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k10temp.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k8temp.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lineage-pem.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm63.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm73.ko 2024-01-31T04:49:04.160 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm75.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm77.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm78.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm80.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm83.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm85.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm87.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm90.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm92.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm93.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95234.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95241.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95245.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc2945.ko 2024-01-31T04:49:04.161 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4151.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4215.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4222.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4245.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4260.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4261.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max16065.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1619.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1668.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max197.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6639.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6650.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6697.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/mcp3021.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6683.ko 2024-01-31T04:49:04.162 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775-core.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct7802.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87360.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87427.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pcf8591.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-31T04:49:04.163 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/powr1220.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5627.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5636.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht15.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht21.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/shtc1.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sis5595.ko 2024-01-31T04:49:04.164 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47b397.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m1.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m192.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/thmc50.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp102.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp103.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp401.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp421.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via-cputemp.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via686a.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt1211.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt8231.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627ehf.ko 2024-01-31T04:49:04.165 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627hf.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83781d.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83791d.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83792d.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83793.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83795.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l785ts.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l786ng.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-31T04:49:04.166 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-31T04:49:04.167 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-dev.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-mux.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-smbus.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-stub.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-31T04:49:04.168 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_core.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca 2024-01-31T04:49:04.169 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport 2024-01-31T04:49:04.170 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/gameport.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/joydev.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/appletouch.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/bcm5974.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/cyapatp.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/sermouse.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-31T04:49:04.171 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/altera_ps2.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/arc_ps2.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/serio_raw.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/sparse-keymap.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-blinkm.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lm3530.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lp3944.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-ss4200.ko 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger 2024-01-31T04:49:04.172 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache/bcache.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-bio-prison.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache-smq.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache.ko 2024-01-31T04:49:04.173 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-crypt.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-delay.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-flakey.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-log-userspace.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-multipath.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-queue-length.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-raid.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-round-robin.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-service-time.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-switch.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-thin-pool.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-verity.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/faulty.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/linear.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/multipath.ko 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data 2024-01-31T04:49:04.174 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid0.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid1.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid10.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid456.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptbase.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptctl.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptfc.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptlan.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptsas.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptscsih.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptspi.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_ich.ko 2024-01-31T04:49:04.175 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_sch.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/mfd-core.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/sm501.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/viperboard.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/vx855.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/wl1273-core.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds9802als.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds990x.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/bh1770glc.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710/cb710.ko 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom 2024-01-31T04:49:04.176 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/at24.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/max6875.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/enclosure.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/hpilo.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29003.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29020.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-me.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-txe.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei.ko 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru 2024-01-31T04:49:04.177 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_7xx1.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_core.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tsl2550.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_balloon.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding/bonding.ko 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa 2024-01-31T04:49:04.178 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dummy.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/eql.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-31T04:49:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera 2024-01-31T04:49:04.180 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-31T04:49:04.181 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-31T04:49:04.182 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-31T04:49:04.183 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dnet.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ethoc.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/fealnx.ko 2024-01-31T04:49:04.184 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-31T04:49:04.185 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/jme.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-31T04:49:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-31T04:49:04.187 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-31T04:49:04.188 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan 2024-01-31T04:49:04.189 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-31T04:49:04.190 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-31T04:49:04.191 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/geneve.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ifb.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvlan.ko 2024-01-31T04:49:04.192 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvtap.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mii.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/net_failover.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/netconsole.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/nlmon.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy 2024-01-31T04:49:04.193 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/amd.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm87xx.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/broadcom.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/cicada.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/davicom.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/dp83640.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/et1011c.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/icplus.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/lxt.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/marvell.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/micrel.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/national.ko 2024-01-31T04:49:04.194 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/phylink.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/qsemi.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/realtek.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/smsc.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/ste10Xp.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/vitesse.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_async.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-31T04:49:04.195 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppoe.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppox.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pptp.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slhc.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slip.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/sungem_phy.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tap.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_random.ko 2024-01-31T04:49:04.196 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tun.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/veth.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/virtio_net.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan/vxlan.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netfront.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb/ntb.ko 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme 2024-01-31T04:49:04.197 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme-core.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_cs.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_pc.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_serial.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/xen-pcifront.ko 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia 2024-01-31T04:49:04.198 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/i82092.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/pd6729.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acerhdf.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-31T04:49:04.199 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/hdaps.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel_ips.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-31T04:49:04.200 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps_parport.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/pps_core.ko 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp 2024-01-31T04:49:04.201 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp_kvm.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-em3027.ko 2024-01-31T04:49:04.202 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-max6900.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-31T04:49:04.203 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-x1205.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-9xxx.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-sas.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-xxxx.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/BusLogic.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/a100u2w.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/advansys.ko 2024-01-31T04:49:04.204 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/am53c974.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/atp870u.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa/bfa.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-31T04:49:04.205 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ch.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dc395x.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dmx3191d.ko 2024-01-31T04:49:04.206 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esp_scsi.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic/fnic.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hpsa.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hptiop.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hv_storvsc.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/initio.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ipr.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ips.ko 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci 2024-01-31T04:49:04.207 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci/isci.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc/libfc.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas/libsas.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-31T04:49:04.208 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvumi.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pmcraid.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla1280.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-31T04:49:04.209 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/raid_class.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_debug.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ses.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/st.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/stex.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-31T04:49:04.210 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/virtio_scsi.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/wd719x.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb/ssb.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_file.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_iblock.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_mod.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_pscsi.ko 2024-01-31T04:49:04.211 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_user.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-31T04:49:04.212 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_gsm.ko 2024-01-31T04:49:04.213 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_hdlc.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/nozomi.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/arc_uart.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio 2024-01-31T04:49:04.214 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_aec.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_cif.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_pci_generic.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_sercos3.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-acm.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usblp.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usbtmc.ko 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host 2024-01-31T04:49:04.215 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/mdc800.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/microtek.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/adutux.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/appledisplay.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi26.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi62.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ezusb.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/idmouse.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/iowarrior.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ldusb.ko 2024-01-31T04:49:04.216 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/legousbtower.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usb3503.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usblcd.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/uss720.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/yurex.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy/phy-generic.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/aircable.ko 2024-01-31T04:49:04.217 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ark3116.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ch341.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cp210x.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cyberjack.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/empeg.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_ti.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipaq.ko 2024-01-31T04:49:04.218 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipw.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ir-usb.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mct_u232.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7720.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7840.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/navman.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/omninet.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/opticon.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/option.ko 2024-01-31T04:49:04.219 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/oti6858.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/pl2303.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcaux.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcserial.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/quatech2.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/safe_serial.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/sierra.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ssu100.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/symbolserial.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-31T04:49:04.220 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_debug.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/visor.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/whiteheat.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/uas.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-31T04:49:04.221 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-karma.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/usb-storage.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio 2024-01-31T04:49:04.222 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_net.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_scsi.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/apple_bl.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/lcd.ko 2024-01-31T04:49:04.223 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_balloon.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_mmio.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_ring.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2482.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2490.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-31T04:49:04.224 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/wire.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-31T04:49:04.225 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/hpwdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/i6300esb.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ib700wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ibmasr.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it87_wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/machzwd.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/nv_tco.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-31T04:49:04.226 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/softdog.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/via_wdt.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/wdt_pci.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/xen_wdt.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-evtchn.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntalloc.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntdev.ko 2024-01-31T04:49:04.227 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-privcmd.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p/9p.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs/affs.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs/befs.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/binfmt_misc.ko 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs 2024-01-31T04:49:04.228 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs/btrfs.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles/cachefiles.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph/ceph.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda/coda.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs/cramfs.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm/dlm.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs/f2fs.ko 2024-01-31T04:49:04.229 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/fat.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/msdos.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/vfat.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache/fscache.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/cuse.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/fuse.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2/gfs2.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs/hfs.ko 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus 2024-01-31T04:49:04.230 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus/hfsplus.ko 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs/isofs.ko 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs/jfs.ko 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd/lockd.ko 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix/minix.ko 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs/netfs.ko 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout 2024-01-31T04:49:04.231 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfs.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv3.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv4.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/grace.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/nfs_acl.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd/nfsd.ko 2024-01-31T04:49:04.232 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2/nilfs2.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-celtic.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-centeuro.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-croatian.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-cyrillic.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-gaelic.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-greek.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-iceland.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-inuit.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-roman.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-romanian.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-turkish.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1250.ko 2024-01-31T04:49:04.233 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1251.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1255.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp737.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp775.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp850.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp852.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp855.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp857.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp860.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp861.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp862.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp863.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp864.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp865.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp866.ko 2024-01-31T04:49:04.234 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp869.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp874.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp932.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp936.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp949.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp950.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_euc-jp.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-1.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-13.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-14.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-15.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-2.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-3.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-4.ko 2024-01-31T04:49:04.235 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-5.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-6.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-7.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-9.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-r.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-ru.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-u.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_utf8.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm 2024-01-31T04:49:04.236 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs/overlay.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore/ramoops.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs/reiserfs.ko 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs 2024-01-31T04:49:04.237 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs/romfs.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client/cifs.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_arc4.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_md4.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs/squashfs.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv/sysv.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf/udf.ko 2024-01-31T04:49:04.238 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs/ufs.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs/xfs.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu/rcutorture.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/torture.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/asn1_encoder.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc-itu-t.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc8.ko 2024-01-31T04:49:04.239 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libarc4.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libdes.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/libcrc32c.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lru_cache.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4_compress.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4hc_compress.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math/cordic.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6/raid6_pq.ko 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon 2024-01-31T04:49:04.240 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_bm.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_fsm.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_kmp.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd/zstd_compress.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm/hwpoison-inject.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/6lowpan.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_dest.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_fragment.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_hop.ko 2024-01-31T04:49:04.241 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_mobility.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_routing.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_udp.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/garp.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/mrp.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/p8022.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/psnap.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/stp.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q/8021q.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_fd.ko 2024-01-31T04:49:04.242 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_rdma.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_virtio.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv/batman-adv.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/br_netfilter.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/bridge.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-31T04:49:04.243 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtables.ko 2024-01-31T04:49:04.244 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph/libceph.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/failover.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/pktgen.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_diag.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv4.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv6.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver/dns_resolver.ko 2024-01-31T04:49:04.245 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/dsa_core.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_brcm.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_dsa.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_none.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_trailer.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ah4.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/esp4.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/fou.ko 2024-01-31T04:49:04.246 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/gre.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/inet_diag.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_gre.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_tunnel.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_vti.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipcomp.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipip.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-31T04:49:04.247 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-31T04:49:04.248 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_bic.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_dctcp.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_diag.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_highspeed.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_htcp.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_hybla.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_illinois.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_lp.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_scalable.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_vegas.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_veno.ko 2024-01-31T04:49:04.249 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_westwood.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_yeah.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tunnel4.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_diag.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_tunnel.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ah6.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/esp6.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/fou6.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_tunnel.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_vti.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ipcomp6.ko 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter 2024-01-31T04:49:04.250 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-31T04:49:04.251 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/sit.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/tunnel6.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key/af_key.ko 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp 2024-01-31T04:49:04.252 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_core.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_eth.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip6.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_netlink.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ppp.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc/llc.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154/mac802154.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls/mpls_gso.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-31T04:49:04.253 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-31T04:49:04.254 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conncount.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack.ko 2024-01-31T04:49:04.255 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-31T04:49:04.256 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_log_syslog.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_irc.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_sip.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_tables.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_log.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_chain_nat.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_compat.ko 2024-01-31T04:49:04.257 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_ct.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fib.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_hash.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_limit.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_log.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_masq.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_nat.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_numgen.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_queue.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_quota.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_redir.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject_inet.ko 2024-01-31T04:49:04.258 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_socket.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_tunnel.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_AUDIT.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CT.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_DSCP.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HL.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HMARK.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LED.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LOG.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-31T04:49:04.259 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NETMAP.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFLOG.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_RATEEST.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_SECMARK.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TEE.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TPROXY.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TRACE.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_addrtype.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_bpf.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cgroup.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cluster.ko 2024-01-31T04:49:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_comment.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connbytes.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlabel.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlimit.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connmark.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_conntrack.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cpu.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dccp.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_devgroup.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dscp.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ecn.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_esp.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hashlimit.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_helper.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hl.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipcomp.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_iprange.ko 2024-01-31T04:49:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipvs.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_l2tp.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_length.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_limit.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mac.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mark.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_multiport.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nat.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nfacct.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_osf.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_owner.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_physdev.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_pkttype.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_policy.ko 2024-01-31T04:49:04.262 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_quota.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_rateest.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_realm.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_recent.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_sctp.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_set.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_socket.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_state.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_statistic.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_string.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_tcpmss.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_time.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_u32.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink/netlink_diag.ko 2024-01-31T04:49:04.263 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh/nsh.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/openvswitch.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-geneve.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-gre.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-vxlan.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet/af_packet_diag.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_rdma.ko 2024-01-31T04:49:04.264 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_tcp.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_csum.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_gact.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_ipt.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_mirred.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_nat.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_pedit.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_police.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_simple.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_skbedit.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_vlan.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_basic.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_bpf.ko 2024-01-31T04:49:04.265 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_flow.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_fw.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_route.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_u32.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_cmp.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_ipset.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_meta.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_nbyte.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_text.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_u32.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_choke.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_codel.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_drr.ko 2024-01-31T04:49:04.266 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_fq.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_gred.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hfsc.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hhf.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_htb.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_ingress.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio_lib.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_multiq.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_netem.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_pie.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_plug.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_prio.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_qfq.ko 2024-01-31T04:49:04.267 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_red.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfb.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfq.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_tbf.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_teql.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp_diag.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-31T04:49:04.268 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/sunrpc.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/diag.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/tipc.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix/unix_diag.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-31T04:49:04.269 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys/trusted.ko 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib/irqbypass.ko 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias.bin 2024-01-31T04:49:04.270 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.alias.bin 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.bin 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.modinfo 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep.bin 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.devname 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.order 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.softdep 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols 2024-01-31T04:49:04.271 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols.bin 2024-01-31T04:49:04.272 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-gcd42140e6285 2024-01-31T04:49:04.272 DEBUG:teuthology.orchestra.run.smithi188:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:postinstall scriptlet (using /bin/sh): 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-gcd42140e6285 -a -r /boot/System.map-6.7.0-gcd42140e6285 ]; then 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:cp /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:cp /boot/System.map-6.7.0-gcd42140e6285 /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:rm -f /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/System.map-6.7.0-gcd42140e6285 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:/sbin/installkernel 6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-01-31T04:49:04.294 INFO:teuthology.orchestra.run.smithi188.stdout:rm -f /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:new-kernel-pkg --remove 6.7.0-gcd42140e6285 --rminitrd --initrdfile=/boot/initramfs-6.7.0-gcd42140e6285.img 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-install remove 6.7.0-gcd42140e6285 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:/sbin/update-bootloader --remove 6.7.0-gcd42140e6285 2024-01-31T04:49:04.295 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2024-01-31T04:49:04.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-01-31T04:49:04.379 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-01-31T04:49:04.379 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-01-31T04:49:04.379 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-01-31T04:49:04.379 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-01-31T04:49:04.380 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:49:04.381 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-01-31T04:49:04.382 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-31T04:49:04.382 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 6.7.0-gcd42140e6285 2024-01-31T04:49:04.382 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-31T04:49:04.940 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-01-31T04:49:06.383 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-31T04:49:06.421 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-01-31T04:49:06.423 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-01-31T04:49:06.438 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.MT4GRBmWH7 2024-01-31T04:49:06.438 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MT4GRBmWH7 2024-01-31T04:49:06.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.MT4GRBmWH7 2024-01-31T04:49:06.645 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.MT4GRBmWH7 is 6KB 2024-01-31T04:49:06.693 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.MT4GRBmWH7 2024-01-31T04:49:06.707 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-01-31T04:49:06.771 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-31T04:49:06.771 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-31T04:49:06.771 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-31T04:49:06.771 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285.conf 2024-01-31T04:49:06.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-gcd42140e6285 2024-01-31T04:49:06.959 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-01-31T04:49:36.962 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-01-31T04:49:36.963 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-31T04:49:36.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:49:55.516 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-01-31T04:50:04.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-31T04:50:04.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:50:07.580 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-01-31T04:50:19.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-31T04:50:19.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:50:51.532 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-01-31T04:50:52.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-31T04:50:52.136 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.7.0-gcd42140e6285"... 2024-01-31T04:50:52.136 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-01-31T04:50:52.176 INFO:teuthology.orchestra.run.smithi188.stdout:6.7.0-gcd42140e6285 2024-01-31T04:50:52.176 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-gcd42140e6285 vs 6.7.0-gcd42140e6285 2024-01-31T04:50:52.177 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-31T04:50:52.177 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-31T04:50:53.178 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-01-31T04:50:53.178 DEBUG:teuthology.orchestra.run.smithi188:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-31T04:50:53.302 INFO:teuthology.orchestra.run.smithi188.stdout:ttyS1 2024-01-31T04:50:53.360 DEBUG:teuthology.parallel:result is None 2024-01-31T04:50:53.360 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-31T04:50:53.372 INFO:teuthology.task.internal:Creating test directory... 2024-01-31T04:50:53.372 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-31T04:50:53.381 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-31T04:50:53.431 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-31T04:50:53.440 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-31T04:50:53.453 INFO:teuthology.task.internal:Creating archive directory... 2024-01-31T04:50:53.453 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-31T04:50:53.457 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-31T04:50:53.528 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-31T04:50:53.538 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-31T04:50:53.538 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-31T04:50:53.544 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-31T04:50:53.624 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T04:50:53.656 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T04:50:53.659 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T04:50:53.702 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-31T04:50:53.707 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-31T04:50:53.714 INFO:teuthology.task.internal:Configuring sudo... 2024-01-31T04:50:53.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-31T04:50:53.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-31T04:50:53.807 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-31T04:50:53.817 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-31T04:50:53.818 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-31T04:50:53.825 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-31T04:50:53.869 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T04:50:53.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T04:50:54.059 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T04:50:54.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T04:50:54.244 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:50:54.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-31T04:50:54.337 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T04:50:54.413 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-31T04:50:54.497 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T04:50:54.566 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-31T04:50:54.660 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:50:54.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-31T04:50:54.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-01-31T04:50:54.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-01-31T04:50:54.865 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T04:50:54.875 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T04:50:55.319 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-31T04:50:55.326 INFO:teuthology.task.internal:Starting timer... 2024-01-31T04:50:55.326 INFO:teuthology.run_tasks:Running task pcp... 2024-01-31T04:50:55.337 INFO:teuthology.run_tasks:Running task selinux... 2024-01-31T04:50:55.345 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-01-31T04:50:55.480 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-01-31T04:50:55.485 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-01-31T04:50:55.598 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-01-31T04:50:55.603 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-31T04:50:55.603 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-01-31T04:50:55.668 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-01-31T04:50:55.669 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-01-31T04:50:55.728 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-01-31T04:50:55.728 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-01-31T04:50:55.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T04:50:55.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T04:50:55.819 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T04:50:55.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T04:50:55.895 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-31T04:50:55.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-01-31T04:50:55.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-01-31T04:50:56.085 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-31T04:50:56.095 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-01-31T04:50:56.102 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-01-31T04:50:56.252 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-31T04:50:56.268 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-31T04:50:56.269 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-31T04:57:05.836 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-01-31T04:57:05.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-01-31T04:57:05.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:57:05.974 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-01-31T04:57:06.115 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-01-31T04:57:06.116 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-31T04:57:06.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-31T04:57:06.262 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-01-31T04:57:06.419 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-31T04:57:06.419 INFO:teuthology.run_tasks:Running task clock... 2024-01-31T04:57:06.431 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-31T04:57:06.431 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-31T04:57:06.432 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T04:57:06.443 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-31T04:57:06.443 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T04:57:06.538 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-31T04:57:06.544 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-31T04:57:06.591 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-31T04:57:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-31T04:57:06.678 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-01-31T04:57:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-01-31T04:57:06.717 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-01-31T04:57:06.729 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-01-31T04:57:06.765 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-31T04:57:06.779 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-31T04:57:06.818 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-31T04:57:06.831 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-31T04:57:06.941 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-01-31T04:57:06.948 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-01-31T04:57:06.949 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T04:57:06.949 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-01-31T04:57:06.949 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.949 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.949 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.949 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.956 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T04:57:06.957 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-01-31T04:57:06.957 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.957 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.957 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.957 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T04:57:06.958 INFO:teuthology.run_tasks:Running task install... 2024-01-31T04:57:06.968 DEBUG:teuthology.task.install:project ceph 2024-01-31T04:57:06.968 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}} 2024-01-31T04:57:06.969 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'} 2024-01-31T04:57:06.969 INFO:teuthology.task.install:Using flavor: default 2024-01-31T04:57:06.975 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-31T04:57:06.975 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-01-31T04:57:06.975 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-31T04:57:06.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T04:57:06.978 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-31T04:57:06.978 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T04:57:07.135 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-31T04:57:07.135 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-31T04:57:07.139 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-31T04:57:07.139 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-31T04:57:07.280 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-31T04:57:07.280 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:57:07.280 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-31T04:57:07.282 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-31T04:57:07.282 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:57:07.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-31T04:57:07.379 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-31T04:57:07.379 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-31T04:57:07.382 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-31T04:57:07.383 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-31T04:57:07.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-31T04:57:07.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-31T04:57:07.649 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-31T04:57:07.650 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-31T04:57:07.734 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-01-31T04:57:07.737 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-01-31T04:57:07.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-01-31T04:57:07.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-01-31T04:57:08.086 INFO:teuthology.orchestra.run.smithi188.stdout:74 files removed 2024-01-31T04:57:08.102 INFO:teuthology.orchestra.run.smithi097.stdout:74 files removed 2024-01-31T04:57:08.128 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-31T04:57:08.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-31T04:57:09.178 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 150 kB/s | 80 kB 00:00 2024-01-31T04:57:09.755 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 71 kB/s | 80 kB 00:01 2024-01-31T04:57:09.879 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 16 kB/s | 9.5 kB 00:00 2024-01-31T04:57:10.671 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-01-31T04:57:10.806 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 9.9 kB/s | 9.5 kB 00:00 2024-01-31T04:57:11.591 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - AppStream 9.3 MB/s | 7.8 MB 00:00 2024-01-31T04:57:12.348 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 1.3 kB/s | 1.9 kB 00:01 2024-01-31T04:57:12.994 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 14 MB/s | 7.8 MB 00:00 2024-01-31T04:57:13.392 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.5 MB 00:00 2024-01-31T04:57:14.802 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.5 MB 00:00 2024-01-31T04:57:15.546 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-01-31T04:57:19.872 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-01-31T04:57:24.964 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 19 MB 00:08 2024-01-31T04:57:29.306 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.9 MB 00:00 2024-01-31T04:57:31.215 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 15 kB 00:00 2024-01-31T04:57:34.067 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 20 MB 00:02 2024-01-31T04:57:38.996 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:17 2024-01-31T04:57:39.841 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-01-31T04:57:42.418 INFO:teuthology.orchestra.run.smithi097.stderr:Modular dependency problems: 2024-01-31T04:57:42.418 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 1: conflicting requests 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 2: conflicting requests 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 3: conflicting requests 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 4: conflicting requests 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 5: conflicting requests 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 6: conflicting requests 2024-01-31T04:57:42.419 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 7: conflicting requests 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 8: conflicting requests 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 9: conflicting requests 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 10: conflicting requests 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 11: conflicting requests 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 12: conflicting requests 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.420 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 13: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 14: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 15: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 16: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 17: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 18: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 19: conflicting requests 2024-01-31T04:57:42.421 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 20: conflicting requests 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 21: conflicting requests 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 22: conflicting requests 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 23: conflicting requests 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 24: conflicting requests 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 25: conflicting requests 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.422 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 26: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 27: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 28: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 29: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 30: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 31: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 32: conflicting requests 2024-01-31T04:57:42.423 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 33: conflicting requests 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 34: conflicting requests 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 35: conflicting requests 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 36: conflicting requests 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 37: conflicting requests 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.424 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 39: conflicting requests 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.425 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.426 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T04:57:42.427 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.428 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.428 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T04:57:42.428 INFO:teuthology.orchestra.run.smithi097.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T04:57:43.026 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T04:57:43.027 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T04:57:43.050 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-31T04:57:43.051 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-31T04:57:43.052 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-31T04:57:43.054 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-31T04:57:43.056 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-31T04:57:43.060 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-31T04:57:43.061 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-31T04:57:43.153 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-01-31T04:57:43.189 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-01-31T04:57:43.189 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-01-31T04:57:43.189 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-31T04:57:43.190 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-01-31T04:57:43.191 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-01-31T04:57:43.192 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager x86_64 1:1.45.10-1.el9 baseos 2.3 M 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 baseos 1.8 M 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 baseos 43 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 baseos 247 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-01-31T04:57:43.193 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-01-31T04:57:43.194 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-01-31T04:57:43.195 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-31T04:57:43.196 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-01-31T04:57:43.196 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-31T04:57:43.196 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-31T04:57:43.196 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-01-31T04:57:43.199 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-31T04:57:43.200 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-31T04:57:43.201 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-4.el9 appstream 249 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-31T04:57:43.202 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-31T04:57:43.203 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-01-31T04:57:43.204 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================== 2024-01-31T04:57:43.205 INFO:teuthology.orchestra.run.smithi097.stdout:Install 139 Packages 2024-01-31T04:57:43.206 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 16 Packages 2024-01-31T04:57:43.206 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:57:43.206 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 212 M 2024-01-31T04:57:43.206 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-01-31T04:57:44.343 INFO:teuthology.orchestra.run.smithi097.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 16 kB/s | 6.5 kB 00:00 2024-01-31T04:57:44.675 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-01-31T04:57:45.127 INFO:teuthology.orchestra.run.smithi097.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 1.1 MB/s | 886 kB 00:00 2024-01-31T04:57:45.227 INFO:teuthology.orchestra.run.smithi097.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 1.4 MB/s | 145 kB 00:00 2024-01-31T04:57:46.011 INFO:teuthology.orchestra.run.smithi097.stdout:(4/155): ceph-base-19.0.0-885.g661946e7.el9.x86 2.5 MB/s | 5.1 MB 00:02 2024-01-31T04:57:46.103 INFO:teuthology.orchestra.run.smithi097.stdout:(5/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 2.4 MB/s | 2.1 MB 00:00 2024-01-31T04:57:46.469 INFO:teuthology.orchestra.run.smithi097.stdout:(6/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-01-31T04:57:46.570 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 15 kB 00:00 2024-01-31T04:57:47.378 INFO:teuthology.orchestra.run.smithi097.stdout:(7/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 3.7 MB/s | 4.7 MB 00:01 2024-01-31T04:57:48.980 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 20 MB 00:02 2024-01-31T04:57:49.220 INFO:teuthology.orchestra.run.smithi097.stdout:(8/155): ceph-radosgw-19.0.0-885.g661946e7.el9. 4.7 MB/s | 8.7 MB 00:01 2024-01-31T04:57:49.287 INFO:teuthology.orchestra.run.smithi097.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 383 kB/s | 25 kB 00:00 2024-01-31T04:57:49.854 INFO:teuthology.orchestra.run.smithi097.stdout:(10/155): ceph-osd-19.0.0-885.g661946e7.el9.x86 4.9 MB/s | 17 MB 00:03 2024-01-31T04:57:49.962 INFO:teuthology.orchestra.run.smithi097.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 2.0 MB/s | 217 kB 00:00 2024-01-31T04:57:50.046 INFO:teuthology.orchestra.run.smithi097.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 329 kB/s | 27 kB 00:00 2024-01-31T04:57:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 542 kB/s | 31 kB 00:00 2024-01-31T04:57:50.305 INFO:teuthology.orchestra.run.smithi097.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 3.6 MB/s | 743 kB 00:00 2024-01-31T04:57:50.380 INFO:teuthology.orchestra.run.smithi097.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 2.1 MB/s | 163 kB 00:00 2024-01-31T04:57:50.455 INFO:teuthology.orchestra.run.smithi097.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.7 MB/s | 127 kB 00:00 2024-01-31T04:57:50.606 INFO:teuthology.orchestra.run.smithi097.stdout:(17/155): libradosstriper1-19.0.0-885.g661946e7 3.3 MB/s | 499 kB 00:00 2024-01-31T04:57:51.540 INFO:teuthology.orchestra.run.smithi097.stdout:(18/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 5.1 MB/s | 4.8 MB 00:00 2024-01-31T04:57:51.598 INFO:teuthology.orchestra.run.smithi097.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 791 kB/s | 45 kB 00:00 2024-01-31T04:57:51.673 INFO:teuthology.orchestra.run.smithi097.stdout:(20/155): python3-ceph-common-19.0.0-885.g66194 1.7 MB/s | 130 kB 00:00 2024-01-31T04:57:51.740 INFO:teuthology.orchestra.run.smithi097.stdout:(21/155): python3-cephfs-19.0.0-885.g661946e7.e 2.4 MB/s | 162 kB 00:00 2024-01-31T04:57:51.857 INFO:teuthology.orchestra.run.smithi097.stdout:(22/155): python3-rados-19.0.0-885.g661946e7.el 2.7 MB/s | 321 kB 00:00 2024-01-31T04:57:51.998 INFO:teuthology.orchestra.run.smithi097.stdout:(23/155): python3-rbd-19.0.0-885.g661946e7.el9. 2.1 MB/s | 298 kB 00:00 2024-01-31T04:57:52.075 INFO:teuthology.orchestra.run.smithi097.stdout:(24/155): python3-rgw-19.0.0-885.g661946e7.el9. 1.3 MB/s | 99 kB 00:00 2024-01-31T04:57:52.150 INFO:teuthology.orchestra.run.smithi097.stdout:(25/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 1.1 MB/s | 85 kB 00:00 2024-01-31T04:57:52.758 INFO:teuthology.orchestra.run.smithi097.stdout:(26/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 5.1 MB/s | 3.1 MB 00:00 2024-01-31T04:57:52.833 INFO:teuthology.orchestra.run.smithi097.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 2.3 MB/s | 170 kB 00:00 2024-01-31T04:57:52.901 INFO:teuthology.orchestra.run.smithi097.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 432 kB/s | 28 kB 00:00 2024-01-31T04:57:52.976 INFO:teuthology.orchestra.run.smithi097.stdout:(29/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 1.7 MB/s | 133 kB 00:00 2024-01-31T04:57:53.351 INFO:teuthology.orchestra.run.smithi097.stdout:(30/155): ceph-mgr-dashboard-19.0.0-885.g661946 4.6 MB/s | 1.7 MB 00:00 2024-01-31T04:57:53.602 INFO:teuthology.orchestra.run.smithi097.stdout:(31/155): ceph-common-19.0.0-885.g661946e7.el9. 2.0 MB/s | 19 MB 00:09 2024-01-31T04:57:53.794 INFO:teuthology.orchestra.run.smithi097.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 1.3 MB/s | 245 kB 00:00 2024-01-31T04:57:53.913 INFO:teuthology.orchestra.run.smithi097.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 428 kB/s | 51 kB 00:00 2024-01-31T04:57:54.002 INFO:teuthology.orchestra.run.smithi097.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 172 kB/s | 15 kB 00:00 2024-01-31T04:57:54.161 INFO:teuthology.orchestra.run.smithi097.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 1.6 MB/s | 262 kB 00:00 2024-01-31T04:57:54.478 INFO:teuthology.orchestra.run.smithi097.stdout:(36/155): cephadm-19.0.0-885.g661946e7.el9.noar 2.3 MB/s | 742 kB 00:00 2024-01-31T04:57:54.553 INFO:teuthology.orchestra.run.smithi097.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 245 kB/s | 18 kB 00:00 2024-01-31T04:57:54.738 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-01-31T04:57:54.745 INFO:teuthology.orchestra.run.smithi097.stdout:(38/155): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-01-31T04:57:54.787 INFO:teuthology.orchestra.run.smithi097.stdout:(39/155): libaio-devel-0.3.112-1.el8.x86_64.rpm 75 kB/s | 18 kB 00:00 2024-01-31T04:57:55.496 INFO:teuthology.orchestra.run.smithi097.stdout:(40/155): fuse-2.9.9-15.el9.x86_64.rpm 108 kB/s | 80 kB 00:00 2024-01-31T04:57:55.638 INFO:teuthology.orchestra.run.smithi097.stdout:(41/155): jq-1.6-16.el9.x86_64.rpm 220 kB/s | 187 kB 00:00 2024-01-31T04:57:56.572 INFO:teuthology.orchestra.run.smithi097.stdout:(42/155): ceph-test-19.0.0-885.g661946e7.el9.x8 6.3 MB/s | 46 MB 00:07 2024-01-31T04:57:56.963 INFO:teuthology.orchestra.run.smithi097.stdout:(43/155): libquadmath-11.4.1-3.el9.x86_64.rpm 147 kB/s | 195 kB 00:01 2024-01-31T04:57:57.323 INFO:teuthology.orchestra.run.smithi188.stderr:Modular dependency problems: 2024-01-31T04:57:57.323 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-01-31T04:57:57.323 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 1: conflicting requests 2024-01-31T04:57:57.323 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240105104119:20125149.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.323 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 2: conflicting requests 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 3: conflicting requests 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 4: conflicting requests 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 5: conflicting requests 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 6: conflicting requests 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 7: conflicting requests 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.324 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 8: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 9: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 10: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240105093033:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 11: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 12: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 13: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 14: conflicting requests 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.325 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 15: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 16: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 17: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 18: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 19: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 20: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 21: conflicting requests 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.326 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 22: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 23: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820231201124232:182f7c73.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 24: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 25: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 26: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 27: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 28: conflicting requests 2024-01-31T04:57:57.327 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.328 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 29: conflicting requests 2024-01-31T04:57:57.328 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.328 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 30: conflicting requests 2024-01-31T04:57:57.328 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 31: conflicting requests 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 32: conflicting requests 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 33: conflicting requests 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 34: conflicting requests 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 35: conflicting requests 2024-01-31T04:57:57.329 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 36: conflicting requests 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240117192000:9edba152.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 37: conflicting requests 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 38: cannot install the best candidate for the job 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 39: conflicting requests 2024-01-31T04:57:57.330 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 40: cannot install the best candidate for the job 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 41: cannot install the best candidate for the job 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.331 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 42: cannot install the best candidate for the job 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 43: cannot install the best candidate for the job 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 44: cannot install the best candidate for the job 2024-01-31T04:57:57.332 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: Problem 45: cannot install the best candidate for the job 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.333 INFO:teuthology.orchestra.run.smithi188.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-01-31T04:57:57.488 INFO:teuthology.orchestra.run.smithi097.stdout:(44/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 238 kB/s | 218 kB 00:00 2024-01-31T04:57:57.580 INFO:teuthology.orchestra.run.smithi097.stdout:(45/155): libgfortran-11.4.1-3.el9.x86_64.rpm 386 kB/s | 803 kB 00:02 2024-01-31T04:57:57.848 INFO:teuthology.orchestra.run.smithi097.stdout:(46/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 288 kB/s | 253 kB 00:00 2024-01-31T04:57:57.940 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T04:57:57.942 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-31T04:57:57.963 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-31T04:57:57.965 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-31T04:57:57.966 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-31T04:57:57.968 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-31T04:57:57.970 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-31T04:57:57.974 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-31T04:57:57.976 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-31T04:57:58.067 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-01-31T04:57:58.083 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================== 2024-01-31T04:57:58.083 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-01-31T04:57:58.083 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================== 2024-01-31T04:57:58.084 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-01-31T04:57:58.084 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-01-31T04:57:58.084 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-31T04:57:58.084 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-31T04:57:58.084 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-31T04:57:58.084 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-31T04:57:58.085 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-31T04:57:58.086 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-31T04:57:58.087 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-31T04:57:58.088 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-31T04:57:58.088 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T04:57:58.088 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-31T04:57:58.088 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-01-31T04:57:58.088 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-01-31T04:57:58.088 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.45.10-1.el9 baseos 2.3 M 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 baseos 1.8 M 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 baseos 43 k 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 baseos 247 k 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-01-31T04:57:58.089 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-01-31T04:57:58.090 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-31T04:57:58.091 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-01-31T04:57:58.092 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-31T04:57:58.093 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-01-31T04:57:58.094 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-01-31T04:57:58.095 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-31T04:57:58.096 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-31T04:57:58.097 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-31T04:57:58.098 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-4.el9 appstream 249 k 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-31T04:57:58.099 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-01-31T04:57:58.100 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-01-31T04:57:58.101 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-31T04:57:58.102 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-01-31T04:57:58.102 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-31T04:57:58.102 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-01-31T04:57:58.102 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-01-31T04:57:58.103 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-01-31T04:57:58.104 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================== 2024-01-31T04:57:58.105 INFO:teuthology.orchestra.run.smithi188.stdout:Install 139 Packages 2024-01-31T04:57:58.106 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 16 Packages 2024-01-31T04:57:58.106 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:57:58.107 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 212 M 2024-01-31T04:57:58.107 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-01-31T04:57:58.506 INFO:teuthology.orchestra.run.smithi097.stdout:(47/155): python3-ply-3.11-14.el9.noarch.rpm 115 kB/s | 106 kB 00:00 2024-01-31T04:57:58.681 INFO:teuthology.orchestra.run.smithi097.stdout:(48/155): python3-cryptography-36.0.1-4.el9.x86 1.0 MB/s | 1.2 MB 00:01 2024-01-31T04:57:58.865 INFO:teuthology.orchestra.run.smithi097.stdout:(49/155): python3-pycparser-2.20-6.el9.noarch.r 133 kB/s | 135 kB 00:01 2024-01-31T04:57:58.998 INFO:teuthology.orchestra.run.smithi188.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 21 kB/s | 6.5 kB 00:00 2024-01-31T04:57:59.107 INFO:teuthology.orchestra.run.smithi097.stdout:(50/155): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-01-31T04:57:59.257 INFO:teuthology.orchestra.run.smithi097.stdout:(51/155): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-01-31T04:57:59.307 INFO:teuthology.orchestra.run.smithi097.stdout:(52/155): boost-program-options-1.75.0-8.el9.x8 2.3 MB/s | 107 kB 00:00 2024-01-31T04:57:59.349 INFO:teuthology.orchestra.run.smithi097.stdout:(53/155): elfutils-libelf-devel-0.190-2.el9.x86 524 kB/s | 21 kB 00:00 2024-01-31T04:57:59.400 INFO:teuthology.orchestra.run.smithi097.stdout:(54/155): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-01-31T04:57:59.450 INFO:teuthology.orchestra.run.smithi097.stdout:(55/155): flexiblas-3.0.4-8.el9.x86_64.rpm 663 kB/s | 33 kB 00:00 2024-01-31T04:57:59.500 INFO:teuthology.orchestra.run.smithi097.stdout:(56/155): python3-requests-2.25.1-8.el9.noarch. 126 kB/s | 125 kB 00:00 2024-01-31T04:57:59.609 INFO:teuthology.orchestra.run.smithi097.stdout:(57/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-01-31T04:57:59.635 INFO:teuthology.orchestra.run.smithi097.stdout:(58/155): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 18 kB 00:00 2024-01-31T04:57:59.693 INFO:teuthology.orchestra.run.smithi097.stdout:(59/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-01-31T04:57:59.710 INFO:teuthology.orchestra.run.smithi097.stdout:(60/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 608 kB/s | 45 kB 00:00 2024-01-31T04:57:59.760 INFO:teuthology.orchestra.run.smithi097.stdout:(61/155): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-01-31T04:57:59.764 INFO:teuthology.orchestra.run.smithi188.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 1.1 MB/s | 886 kB 00:00 2024-01-31T04:57:59.794 INFO:teuthology.orchestra.run.smithi097.stdout:(62/155): python3-urllib3-1.26.5-5.el9.noarch.r 193 kB/s | 215 kB 00:01 2024-01-31T04:57:59.819 INFO:teuthology.orchestra.run.smithi097.stdout:(63/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 442 kB/s | 26 kB 00:00 2024-01-31T04:57:59.869 INFO:teuthology.orchestra.run.smithi097.stdout:(64/155): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-01-31T04:57:59.895 INFO:teuthology.orchestra.run.smithi097.stdout:(65/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 480 kB/s | 47 kB 00:00 2024-01-31T04:57:59.906 INFO:teuthology.orchestra.run.smithi188.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 1.0 MB/s | 145 kB 00:00 2024-01-31T04:57:59.928 INFO:teuthology.orchestra.run.smithi097.stdout:(66/155): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-01-31T04:57:59.962 INFO:teuthology.orchestra.run.smithi097.stdout:(67/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-01-31T04:57:59.987 INFO:teuthology.orchestra.run.smithi097.stdout:(68/155): openblas-0.3.21-2.el9.x86_64.rpm 385 kB/s | 35 kB 00:00 2024-01-31T04:58:00.007 INFO:teuthology.orchestra.run.smithi188.stdout:(4/155): ceph-base-19.0.0-885.g661946e7.el9.x86 3.9 MB/s | 5.1 MB 00:01 2024-01-31T04:58:00.137 INFO:teuthology.orchestra.run.smithi097.stdout:(69/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 24 MB/s | 4.1 MB 00:00 2024-01-31T04:58:00.238 INFO:teuthology.orchestra.run.smithi097.stdout:(70/155): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-01-31T04:58:00.282 INFO:teuthology.orchestra.run.smithi188.stdout:(5/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 5.4 MB/s | 1.5 MB 00:00 2024-01-31T04:58:00.296 INFO:teuthology.orchestra.run.smithi097.stdout:(71/155): postgresql-13.11-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-01-31T04:58:00.330 INFO:teuthology.orchestra.run.smithi097.stdout:(72/155): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 135 kB 00:00 2024-01-31T04:58:00.380 INFO:teuthology.orchestra.run.smithi097.stdout:(73/155): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 905 kB 00:00 2024-01-31T04:58:00.406 INFO:teuthology.orchestra.run.smithi097.stdout:(74/155): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-01-31T04:58:00.464 INFO:teuthology.orchestra.run.smithi097.stdout:(75/155): python3-devel-3.9.17-2.el9.x86_64.rpm 4.2 MB/s | 245 kB 00:00 2024-01-31T04:58:00.523 INFO:teuthology.orchestra.run.smithi097.stdout:(76/155): python3-jinja2-2.11.3-4.el9.noarch.rp 4.2 MB/s | 249 kB 00:00 2024-01-31T04:58:00.573 INFO:teuthology.orchestra.run.smithi097.stdout:(77/155): python3-jsonpatch-1.21-16.el9.noarch. 521 kB/s | 26 kB 00:00 2024-01-31T04:58:00.648 INFO:teuthology.orchestra.run.smithi097.stdout:(78/155): python3-jsonpointer-2.0-4.el9.noarch. 258 kB/s | 19 kB 00:00 2024-01-31T04:58:00.773 INFO:teuthology.orchestra.run.smithi097.stdout:(79/155): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-01-31T04:58:00.807 INFO:teuthology.orchestra.run.smithi097.stdout:(80/155): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-01-31T04:58:00.832 INFO:teuthology.orchestra.run.smithi097.stdout:(81/155): python3-markupsafe-1.1.1-12.el9.x86_6 598 kB/s | 35 kB 00:00 2024-01-31T04:58:00.957 INFO:teuthology.orchestra.run.smithi188.stdout:(6/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 7.0 MB/s | 4.7 MB 00:00 2024-01-31T04:58:00.966 INFO:teuthology.orchestra.run.smithi097.stdout:(82/155): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-01-31T04:58:00.999 INFO:teuthology.orchestra.run.smithi097.stdout:(83/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-01-31T04:58:01.033 INFO:teuthology.orchestra.run.smithi097.stdout:(84/155): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-01-31T04:58:01.058 INFO:teuthology.orchestra.run.smithi097.stdout:(85/155): python3-prettytable-0.7.2-27.el9.noar 731 kB/s | 42 kB 00:00 2024-01-31T04:58:01.066 INFO:teuthology.orchestra.run.smithi188.stdout:(7/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 1.8 MB/s | 2.1 MB 00:01 2024-01-31T04:58:01.092 INFO:teuthology.orchestra.run.smithi097.stdout:(86/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-01-31T04:58:01.134 INFO:teuthology.orchestra.run.smithi097.stdout:(87/155): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-01-31T04:58:01.159 INFO:teuthology.orchestra.run.smithi097.stdout:(88/155): python3-pytz-2021.1-5.el9.noarch.rpm 768 kB/s | 51 kB 00:00 2024-01-31T04:58:01.185 INFO:teuthology.orchestra.run.smithi097.stdout:(89/155): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-01-31T04:58:01.235 INFO:teuthology.orchestra.run.smithi097.stdout:(90/155): python3-toml-0.10.2-6.el9.noarch.rpm 849 kB/s | 42 kB 00:00 2024-01-31T04:58:01.287 INFO:teuthology.orchestra.run.smithi097.stdout:(91/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 898 kB/s | 44 kB 00:00 2024-01-31T04:58:01.344 INFO:teuthology.orchestra.run.smithi097.stdout:(92/155): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-01-31T04:58:01.452 INFO:teuthology.orchestra.run.smithi097.stdout:(93/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 5.1 MB 00:00 2024-01-31T04:58:01.478 INFO:teuthology.orchestra.run.smithi097.stdout:(94/155): uuid-1.6.2-55.el9.x86_64.rpm 435 kB/s | 58 kB 00:00 2024-01-31T04:58:01.504 INFO:teuthology.orchestra.run.smithi097.stdout:(95/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-01-31T04:58:01.528 INFO:teuthology.orchestra.run.smithi097.stdout:(96/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-01-31T04:58:01.553 INFO:teuthology.orchestra.run.smithi097.stdout:(97/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 914 kB/s | 45 kB 00:00 2024-01-31T04:58:01.579 INFO:teuthology.orchestra.run.smithi097.stdout:(98/155): lua-devel-5.4.4-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2024-01-31T04:58:01.624 INFO:teuthology.orchestra.run.smithi188.stdout:(8/155): ceph-common-19.0.0-885.g661946e7.el9.x 6.6 MB/s | 19 MB 00:02 2024-01-31T04:58:01.691 INFO:teuthology.orchestra.run.smithi188.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 383 kB/s | 25 kB 00:00 2024-01-31T04:58:01.921 INFO:teuthology.orchestra.run.smithi097.stdout:(99/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-01-31T04:58:02.438 INFO:teuthology.orchestra.run.smithi097.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 51 kB/s | 26 kB 00:00 2024-01-31T04:58:02.466 INFO:teuthology.orchestra.run.smithi188.stdout:(10/155): ceph-osd-19.0.0-885.g661946e7.el9.x86 11 MB/s | 17 MB 00:01 2024-01-31T04:58:02.558 INFO:teuthology.orchestra.run.smithi188.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 2.3 MB/s | 217 kB 00:00 2024-01-31T04:58:02.621 INFO:teuthology.orchestra.run.smithi097.stdout:(101/155): gperftools-libs-2.9.1-2.el9.x86_64.r 289 kB/s | 309 kB 00:01 2024-01-31T04:58:02.625 INFO:teuthology.orchestra.run.smithi188.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 411 kB/s | 27 kB 00:00 2024-01-31T04:58:02.655 INFO:teuthology.orchestra.run.smithi097.stdout:(102/155): liboath-2.6.7-2.el9.x86_64.rpm 224 kB/s | 49 kB 00:00 2024-01-31T04:58:02.692 INFO:teuthology.orchestra.run.smithi188.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 473 kB/s | 31 kB 00:00 2024-01-31T04:58:02.755 INFO:teuthology.orchestra.run.smithi097.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 508 kB/s | 67 kB 00:00 2024-01-31T04:58:02.842 INFO:teuthology.orchestra.run.smithi188.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 4.9 MB/s | 743 kB 00:00 2024-01-31T04:58:02.889 INFO:teuthology.orchestra.run.smithi097.stdout:(104/155): luarocks-3.9.2-1.el9.noarch.rpm 648 kB/s | 151 kB 00:00 2024-01-31T04:58:02.918 INFO:teuthology.orchestra.run.smithi188.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 2.1 MB/s | 163 kB 00:00 2024-01-31T04:58:02.993 INFO:teuthology.orchestra.run.smithi188.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.7 MB/s | 127 kB 00:00 2024-01-31T04:58:03.093 INFO:teuthology.orchestra.run.smithi188.stdout:(17/155): libradosstriper1-19.0.0-885.g661946e7 4.9 MB/s | 499 kB 00:00 2024-01-31T04:58:03.552 INFO:teuthology.orchestra.run.smithi188.stdout:(18/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 10 MB/s | 4.8 MB 00:00 2024-01-31T04:58:03.555 INFO:teuthology.orchestra.run.smithi097.stdout:(105/155): python3-asyncssh-2.13.2-3.el9.noarch 823 kB/s | 548 kB 00:00 2024-01-31T04:58:03.644 INFO:teuthology.orchestra.run.smithi188.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 498 kB/s | 45 kB 00:00 2024-01-31T04:58:03.664 INFO:teuthology.orchestra.run.smithi097.stdout:(106/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-01-31T04:58:03.772 INFO:teuthology.orchestra.run.smithi097.stdout:(107/155): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-01-31T04:58:03.852 INFO:teuthology.orchestra.run.smithi188.stdout:(20/155): ceph-radosgw-19.0.0-885.g661946e7.el9 3.1 MB/s | 8.7 MB 00:02 2024-01-31T04:58:03.886 INFO:teuthology.orchestra.run.smithi188.stdout:(21/155): python3-ceph-common-19.0.0-885.g66194 539 kB/s | 130 kB 00:00 2024-01-31T04:58:03.898 INFO:teuthology.orchestra.run.smithi097.stdout:(108/155): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-01-31T04:58:03.928 INFO:teuthology.orchestra.run.smithi188.stdout:(22/155): python3-cephfs-19.0.0-885.g661946e7.e 2.1 MB/s | 162 kB 00:00 2024-01-31T04:58:03.970 INFO:teuthology.orchestra.run.smithi188.stdout:(23/155): python3-rados-19.0.0-885.g661946e7.el 3.8 MB/s | 321 kB 00:00 2024-01-31T04:58:03.973 INFO:teuthology.orchestra.run.smithi097.stdout:(109/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 688 kB/s | 838 kB 00:01 2024-01-31T04:58:04.045 INFO:teuthology.orchestra.run.smithi188.stdout:(24/155): python3-rbd-19.0.0-885.g661946e7.el9. 2.5 MB/s | 298 kB 00:00 2024-01-31T04:58:04.079 INFO:teuthology.orchestra.run.smithi188.stdout:(25/155): python3-rgw-19.0.0-885.g661946e7.el9. 922 kB/s | 99 kB 00:00 2024-01-31T04:58:04.179 INFO:teuthology.orchestra.run.smithi188.stdout:(26/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 643 kB/s | 85 kB 00:00 2024-01-31T04:58:04.181 INFO:teuthology.orchestra.run.smithi097.stdout:(110/155): python3-cheroot-8.6.0-4.el9.noarch.r 607 kB/s | 172 kB 00:00 2024-01-31T04:58:04.279 INFO:teuthology.orchestra.run.smithi188.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-01-31T04:58:04.363 INFO:teuthology.orchestra.run.smithi188.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 344 kB/s | 28 kB 00:00 2024-01-31T04:58:04.438 INFO:teuthology.orchestra.run.smithi188.stdout:(29/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 8.6 MB/s | 3.1 MB 00:00 2024-01-31T04:58:04.464 INFO:teuthology.orchestra.run.smithi188.stdout:(30/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 1.3 MB/s | 133 kB 00:00 2024-01-31T04:58:04.474 INFO:teuthology.orchestra.run.smithi097.stdout:(111/155): python3-cherrypy-18.6.1-2.el9.noarch 717 kB/s | 358 kB 00:00 2024-01-31T04:58:04.602 INFO:teuthology.orchestra.run.smithi097.stdout:(112/155): python3-colorama-0.4.4-12.el9.noarch 277 kB/s | 35 kB 00:00 2024-01-31T04:58:04.633 INFO:teuthology.orchestra.run.smithi097.stdout:(113/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 673 kB/s | 303 kB 00:00 2024-01-31T04:58:04.639 INFO:teuthology.orchestra.run.smithi188.stdout:(31/155): ceph-mgr-dashboard-19.0.0-885.g661946 8.7 MB/s | 1.7 MB 00:00 2024-01-31T04:58:04.731 INFO:teuthology.orchestra.run.smithi188.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 2.6 MB/s | 245 kB 00:00 2024-01-31T04:58:04.758 INFO:teuthology.orchestra.run.smithi097.stdout:(114/155): python3-jaraco-8.2.1-3.el9.noarch.rp 86 kB/s | 11 kB 00:00 2024-01-31T04:58:04.806 INFO:teuthology.orchestra.run.smithi188.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 685 kB/s | 51 kB 00:00 2024-01-31T04:58:04.866 INFO:teuthology.orchestra.run.smithi097.stdout:(115/155): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-01-31T04:58:04.873 INFO:teuthology.orchestra.run.smithi188.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 227 kB/s | 15 kB 00:00 2024-01-31T04:58:04.892 INFO:teuthology.orchestra.run.smithi097.stdout:(116/155): python3-google-auth-2.26.2-1.el9.noa 700 kB/s | 202 kB 00:00 2024-01-31T04:58:04.975 INFO:teuthology.orchestra.run.smithi188.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 2.6 MB/s | 262 kB 00:00 2024-01-31T04:58:04.984 INFO:teuthology.orchestra.run.smithi097.stdout:(117/155): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-01-31T04:58:05.009 INFO:teuthology.orchestra.run.smithi097.stdout:(118/155): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-01-31T04:58:05.090 INFO:teuthology.orchestra.run.smithi188.stdout:(36/155): cephadm-19.0.0-885.g661946e7.el9.noar 6.3 MB/s | 742 kB 00:00 2024-01-31T04:58:05.102 INFO:teuthology.orchestra.run.smithi097.stdout:(119/155): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-01-31T04:58:05.174 INFO:teuthology.orchestra.run.smithi188.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 221 kB/s | 18 kB 00:00 2024-01-31T04:58:05.218 INFO:teuthology.orchestra.run.smithi097.stdout:(120/155): python3-logutils-0.3.5-21.el9.noarch 399 kB/s | 46 kB 00:00 2024-01-31T04:58:05.299 INFO:teuthology.orchestra.run.smithi188.stdout:(38/155): libaio-devel-0.3.112-1.el8.x86_64.rpm 142 kB/s | 18 kB 00:00 2024-01-31T04:58:05.326 INFO:teuthology.orchestra.run.smithi097.stdout:(121/155): python3-more-itertools-8.12.0-2.el9. 731 kB/s | 79 kB 00:00 2024-01-31T04:58:05.433 INFO:teuthology.orchestra.run.smithi188.stdout:(39/155): fuse-2.9.9-15.el9.x86_64.rpm 606 kB/s | 80 kB 00:00 2024-01-31T04:58:05.502 INFO:teuthology.orchestra.run.smithi097.stdout:(122/155): python3-natsort-7.1.1-5.el9.noarch.r 330 kB/s | 58 kB 00:00 2024-01-31T04:58:05.627 INFO:teuthology.orchestra.run.smithi097.stdout:(123/155): python3-kubernetes-26.1.0-2.el9.noar 1.7 MB/s | 1.0 MB 00:00 2024-01-31T04:58:05.736 INFO:teuthology.orchestra.run.smithi097.stdout:(124/155): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-01-31T04:58:05.841 INFO:teuthology.orchestra.run.smithi188.stdout:(40/155): jq-1.6-16.el9.x86_64.rpm 459 kB/s | 187 kB 00:00 2024-01-31T04:58:05.844 INFO:teuthology.orchestra.run.smithi097.stdout:(125/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 837 kB/s | 90 kB 00:00 2024-01-31T04:58:05.895 INFO:teuthology.orchestra.run.smithi097.stdout:(126/155): python3-pecan-1.4.2-3.el9.noarch.rpm 695 kB/s | 272 kB 00:00 2024-01-31T04:58:05.953 INFO:teuthology.orchestra.run.smithi097.stdout:(127/155): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-01-31T04:58:06.004 INFO:teuthology.orchestra.run.smithi097.stdout:(128/155): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-01-31T04:58:06.079 INFO:teuthology.orchestra.run.smithi097.stdout:(129/155): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-01-31T04:58:06.121 INFO:teuthology.orchestra.run.smithi097.stdout:(130/155): python3-rsa-4.9-2.el9.noarch.rpm 509 kB/s | 59 kB 00:00 2024-01-31T04:58:06.196 INFO:teuthology.orchestra.run.smithi097.stdout:(131/155): python3-tempora-5.0.0-2.el9.noarch.r 307 kB/s | 36 kB 00:00 2024-01-31T04:58:06.238 INFO:teuthology.orchestra.run.smithi097.stdout:(132/155): python3-typing-extensions-4.4.0-2.el 439 kB/s | 51 kB 00:00 2024-01-31T04:58:06.313 INFO:teuthology.orchestra.run.smithi097.stdout:(133/155): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-01-31T04:58:06.439 INFO:teuthology.orchestra.run.smithi097.stdout:(134/155): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2024-01-31T04:58:06.464 INFO:teuthology.orchestra.run.smithi097.stdout:(135/155): python3-websocket-client-1.2.3-2.el9 399 kB/s | 90 kB 00:00 2024-01-31T04:58:06.556 INFO:teuthology.orchestra.run.smithi097.stdout:(136/155): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-01-31T04:58:06.642 INFO:teuthology.orchestra.run.smithi188.stdout:(41/155): ceph-test-19.0.0-885.g661946e7.el9.x8 9.3 MB/s | 46 MB 00:04 2024-01-31T04:58:06.684 INFO:teuthology.orchestra.run.smithi188.stdout:(42/155): libgfortran-11.4.1-3.el9.x86_64.rpm 955 kB/s | 803 kB 00:00 2024-01-31T04:58:06.762 INFO:teuthology.orchestra.run.smithi097.stdout:(137/155): re2-20211101-3.el9.x86_64.rpm 646 kB/s | 192 kB 00:00 2024-01-31T04:58:06.826 INFO:teuthology.orchestra.run.smithi188.stdout:(43/155): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-01-31T04:58:06.898 INFO:teuthology.orchestra.run.smithi097.stdout:(138/155): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-01-31T04:58:06.959 INFO:teuthology.orchestra.run.smithi188.stdout:(44/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 794 kB/s | 218 kB 00:00 2024-01-31T04:58:07.068 INFO:teuthology.orchestra.run.smithi188.stdout:(45/155): libquadmath-11.4.1-3.el9.x86_64.rpm 460 kB/s | 195 kB 00:00 2024-01-31T04:58:07.160 INFO:teuthology.orchestra.run.smithi188.stdout:(46/155): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-01-31T04:58:07.193 INFO:teuthology.orchestra.run.smithi188.stdout:(47/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 691 kB/s | 253 kB 00:00 2024-01-31T04:58:07.249 INFO:teuthology.orchestra.run.smithi097.stdout:(139/155): librados2-19.0.0-885.g661946e7.el9.x 6.9 MB/s | 3.4 MB 00:00 2024-01-31T04:58:07.302 INFO:teuthology.orchestra.run.smithi188.stdout:(48/155): python3-pycparser-2.20-6.el9.noarch.r 958 kB/s | 135 kB 00:00 2024-01-31T04:58:07.327 INFO:teuthology.orchestra.run.smithi188.stdout:(49/155): python3-requests-2.25.1-8.el9.noarch. 940 kB/s | 125 kB 00:00 2024-01-31T04:58:07.391 INFO:teuthology.orchestra.run.smithi097.stdout:(140/155): librbd1-19.0.0-885.g661946e7.el9.x86 6.4 MB/s | 3.1 MB 00:00 2024-01-31T04:58:07.444 INFO:teuthology.orchestra.run.smithi188.stdout:(50/155): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-01-31T04:58:07.699 INFO:teuthology.orchestra.run.smithi097.stdout:(141/155): libarrow-9.0.0-10.el9.x86_64.rpm 739 kB/s | 4.4 MB 00:06 2024-01-31T04:58:07.711 INFO:teuthology.orchestra.run.smithi188.stdout:(51/155): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-01-31T04:58:07.819 INFO:teuthology.orchestra.run.smithi188.stdout:(52/155): boost-program-options-1.75.0-8.el9.x8 993 kB/s | 107 kB 00:00 2024-01-31T04:58:07.870 INFO:teuthology.orchestra.run.smithi188.stdout:(53/155): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-01-31T04:58:07.911 INFO:teuthology.orchestra.run.smithi188.stdout:(54/155): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-01-31T04:58:07.928 INFO:teuthology.orchestra.run.smithi188.stdout:(55/155): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-01-31T04:58:07.954 INFO:teuthology.orchestra.run.smithi188.stdout:(56/155): flexiblas-3.0.4-8.el9.x86_64.rpm 803 kB/s | 33 kB 00:00 2024-01-31T04:58:07.996 INFO:teuthology.orchestra.run.smithi188.stdout:(57/155): flexiblas-openblas-openmp-3.0.4-8.el9 443 kB/s | 18 kB 00:00 2024-01-31T04:58:08.054 INFO:teuthology.orchestra.run.smithi188.stdout:(58/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-01-31T04:58:08.105 INFO:teuthology.orchestra.run.smithi188.stdout:(59/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 916 kB/s | 45 kB 00:00 2024-01-31T04:58:08.172 INFO:teuthology.orchestra.run.smithi188.stdout:(60/155): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-01-31T04:58:08.314 INFO:teuthology.orchestra.run.smithi188.stdout:(61/155): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-01-31T04:58:08.364 INFO:teuthology.orchestra.run.smithi188.stdout:(62/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 520 kB/s | 26 kB 00:00 2024-01-31T04:58:08.431 INFO:teuthology.orchestra.run.smithi188.stdout:(63/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 718 kB/s | 47 kB 00:00 2024-01-31T04:58:08.483 INFO:teuthology.orchestra.run.smithi097.stdout:(142/155): NetworkManager-1.45.10-1.el9.x86_64. 1.8 MB/s | 2.3 MB 00:01 2024-01-31T04:58:08.508 INFO:teuthology.orchestra.run.smithi097.stdout:(143/155): NetworkManager-team-1.45.10-1.el9.x8 58 kB/s | 43 kB 00:00 2024-01-31T04:58:08.581 INFO:teuthology.orchestra.run.smithi188.stdout:(64/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-01-31T04:58:08.648 INFO:teuthology.orchestra.run.smithi188.stdout:(65/155): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-01-31T04:58:08.675 INFO:teuthology.orchestra.run.smithi097.stdout:(144/155): NetworkManager-libnm-1.45.10-1.el9.x 1.4 MB/s | 1.8 MB 00:01 2024-01-31T04:58:08.732 INFO:teuthology.orchestra.run.smithi188.stdout:(66/155): libtool-2.4.6-45.el9.x86_64.rpm 1.0 MB/s | 582 kB 00:00 2024-01-31T04:58:08.757 INFO:teuthology.orchestra.run.smithi188.stdout:(67/155): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 35 kB 00:00 2024-01-31T04:58:08.824 INFO:teuthology.orchestra.run.smithi188.stdout:(68/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 746 kB/s | 292 kB 00:00 2024-01-31T04:58:09.342 INFO:teuthology.orchestra.run.smithi097.stdout:(145/155): acl-2.3.1-4.el9.x86_64.rpm 86 kB/s | 71 kB 00:00 2024-01-31T04:58:09.417 INFO:teuthology.orchestra.run.smithi097.stdout:(146/155): elfutils-debuginfod-client-0.190-2.e 49 kB/s | 36 kB 00:00 2024-01-31T04:58:09.492 INFO:teuthology.orchestra.run.smithi097.stdout:(147/155): NetworkManager-tui-1.45.10-1.el9.x86 246 kB/s | 247 kB 00:01 2024-01-31T04:58:09.591 INFO:teuthology.orchestra.run.smithi188.stdout:(69/155): postgresql-13.11-1.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-01-31T04:58:09.983 INFO:teuthology.orchestra.run.smithi188.stdout:(70/155): postgresql-contrib-13.11-1.el9.x86_64 2.3 MB/s | 905 kB 00:00 2024-01-31T04:58:10.126 INFO:teuthology.orchestra.run.smithi097.stdout:(148/155): libacl-2.3.1-4.el9.x86_64.rpm 36 kB/s | 23 kB 00:00 2024-01-31T04:58:10.310 INFO:teuthology.orchestra.run.smithi097.stdout:(149/155): elfutils-libelf-0.190-2.el9.x86_64.r 199 kB/s | 192 kB 00:00 2024-01-31T04:58:10.341 INFO:teuthology.orchestra.run.smithi188.stdout:(71/155): postgresql-private-libs-13.11-1.el9.x 377 kB/s | 135 kB 00:00 2024-01-31T04:58:10.368 INFO:teuthology.orchestra.run.smithi097.stdout:(150/155): elfutils-libs-0.190-2.el9.x86_64.rpm 269 kB/s | 256 kB 00:00 2024-01-31T04:58:11.269 INFO:teuthology.orchestra.run.smithi097.stdout:(151/155): openssl-3.0.7-25.el9.x86_64.rpm 1.1 MB/s | 1.2 MB 00:01 2024-01-31T04:58:11.436 INFO:teuthology.orchestra.run.smithi097.stdout:(152/155): xfsdump-3.1.12-4.el9.x86_64.rpm 324 kB/s | 323 kB 00:00 2024-01-31T04:58:11.536 INFO:teuthology.orchestra.run.smithi097.stdout:(153/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 752 kB/s | 74 kB 00:00 2024-01-31T04:58:11.678 INFO:teuthology.orchestra.run.smithi097.stdout:(154/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 1.6 MB/s | 2.2 MB 00:01 2024-01-31T04:58:12.370 INFO:teuthology.orchestra.run.smithi097.stdout:(155/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 1.1 MB 00:01 2024-01-31T04:58:12.386 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-01-31T04:58:12.386 INFO:teuthology.orchestra.run.smithi097.stdout:Total 7.3 MB/s | 212 MB 00:29 2024-01-31T04:58:15.491 INFO:teuthology.orchestra.run.smithi188.stdout:(72/155): openblas-openmp-0.3.21-2.el9.x86_64.r 718 kB/s | 4.7 MB 00:06 2024-01-31T04:58:15.524 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-01-31T04:58:15.566 INFO:teuthology.orchestra.run.smithi188.stdout:(73/155): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-01-31T04:58:15.885 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-01-31T04:58:15.885 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-01-31T04:58:16.416 INFO:teuthology.orchestra.run.smithi188.stdout:(74/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 551 kB/s | 4.1 MB 00:07 2024-01-31T04:58:17.849 INFO:teuthology.orchestra.run.smithi188.stdout:(75/155): python3-devel-3.9.17-2.el9.x86_64.rpm 171 kB/s | 245 kB 00:01 2024-01-31T04:58:18.262 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-01-31T04:58:18.262 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-01-31T04:58:18.608 INFO:teuthology.orchestra.run.smithi188.stdout:(76/155): python3-jinja2-2.11.3-4.el9.noarch.rp 329 kB/s | 249 kB 00:00 2024-01-31T04:58:18.858 INFO:teuthology.orchestra.run.smithi188.stdout:(77/155): python3-jsonpatch-1.21-16.el9.noarch. 103 kB/s | 26 kB 00:00 2024-01-31T04:58:18.950 INFO:teuthology.orchestra.run.smithi188.stdout:(78/155): python3-jsonpointer-2.0-4.el9.noarch. 210 kB/s | 19 kB 00:00 2024-01-31T04:58:19.317 INFO:teuthology.orchestra.run.smithi188.stdout:(79/155): python3-mako-1.1.4-6.el9.noarch.rpm 470 kB/s | 172 kB 00:00 2024-01-31T04:58:19.384 INFO:teuthology.orchestra.run.smithi188.stdout:(80/155): python3-markupsafe-1.1.1-12.el9.x86_6 528 kB/s | 35 kB 00:00 2024-01-31T04:58:19.492 INFO:teuthology.orchestra.run.smithi188.stdout:(81/155): postgresql-server-13.11-1.el9.x86_64. 643 kB/s | 5.7 MB 00:09 2024-01-31T04:58:19.701 INFO:teuthology.orchestra.run.smithi188.stdout:(82/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-01-31T04:58:19.928 INFO:teuthology.orchestra.run.smithi188.stdout:(83/155): python3-oauthlib-3.1.1-5.el9.noarch.r 990 kB/s | 222 kB 00:00 2024-01-31T04:58:20.051 INFO:teuthology.orchestra.run.smithi188.stdout:(84/155): python3-prettytable-0.7.2-27.el9.noar 340 kB/s | 42 kB 00:00 2024-01-31T04:58:20.293 INFO:teuthology.orchestra.run.smithi188.stdout:(85/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 659 kB/s | 159 kB 00:00 2024-01-31T04:58:20.468 INFO:teuthology.orchestra.run.smithi188.stdout:(86/155): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-01-31T04:58:20.569 INFO:teuthology.orchestra.run.smithi188.stdout:(87/155): python3-pytz-2021.1-5.el9.noarch.rpm 511 kB/s | 51 kB 00:00 2024-01-31T04:58:20.778 INFO:teuthology.orchestra.run.smithi188.stdout:(88/155): python3-requests-oauthlib-1.3.0-12.el 258 kB/s | 54 kB 00:00 2024-01-31T04:58:23.477 INFO:teuthology.orchestra.run.smithi188.stdout:(89/155): python3-babel-2.9.1-2.el9.noarch.rpm 772 kB/s | 6.0 MB 00:07 2024-01-31T04:58:23.536 INFO:teuthology.orchestra.run.smithi188.stdout:(90/155): python3-toml-0.10.2-6.el9.noarch.rpm 727 kB/s | 42 kB 00:00 2024-01-31T04:58:23.595 INFO:teuthology.orchestra.run.smithi188.stdout:(91/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 767 kB/s | 44 kB 00:00 2024-01-31T04:58:23.745 INFO:teuthology.orchestra.run.smithi188.stdout:(92/155): socat-1.7.4.1-5.el9.x86_64.rpm 2.0 MB/s | 305 kB 00:00 2024-01-31T04:58:23.870 INFO:teuthology.orchestra.run.smithi188.stdout:(93/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.1 MB/s | 5.1 MB 00:04 2024-01-31T04:58:23.887 INFO:teuthology.orchestra.run.smithi188.stdout:(94/155): uuid-1.6.2-55.el9.x86_64.rpm 409 kB/s | 58 kB 00:00 2024-01-31T04:58:23.938 INFO:teuthology.orchestra.run.smithi188.stdout:(95/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-01-31T04:58:23.954 INFO:teuthology.orchestra.run.smithi188.stdout:(96/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 961 kB/s | 64 kB 00:00 2024-01-31T04:58:23.979 INFO:teuthology.orchestra.run.smithi188.stdout:(97/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-01-31T04:58:24.171 INFO:teuthology.orchestra.run.smithi188.stdout:(98/155): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.6 MB/s | 309 kB 00:00 2024-01-31T04:58:24.463 INFO:teuthology.orchestra.run.smithi188.stdout:(99/155): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-01-31T04:58:24.522 INFO:teuthology.orchestra.run.smithi188.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 455 kB/s | 26 kB 00:00 2024-01-31T04:58:24.572 INFO:teuthology.orchestra.run.smithi188.stdout:(101/155): liboath-2.6.7-2.el9.x86_64.rpm 984 kB/s | 49 kB 00:00 2024-01-31T04:58:24.647 INFO:teuthology.orchestra.run.smithi188.stdout:(102/155): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-01-31T04:58:24.673 INFO:teuthology.orchestra.run.smithi188.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 676 kB/s | 67 kB 00:00 2024-01-31T04:58:24.773 INFO:teuthology.orchestra.run.smithi188.stdout:(104/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.3 MB/s | 838 kB 00:00 2024-01-31T04:58:24.780 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-01-31T04:58:24.832 INFO:teuthology.orchestra.run.smithi188.stdout:(105/155): luarocks-3.9.2-1.el9.noarch.rpm 828 kB/s | 151 kB 00:00 2024-01-31T04:58:24.893 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-31T04:58:24.899 INFO:teuthology.orchestra.run.smithi188.stdout:(106/155): python3-asyncssh-2.13.2-3.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-01-31T04:58:24.940 INFO:teuthology.orchestra.run.smithi188.stdout:(107/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-01-31T04:58:24.966 INFO:teuthology.orchestra.run.smithi188.stdout:(108/155): python3-cachetools-4.2.4-1.el9.noarc 486 kB/s | 32 kB 00:00 2024-01-31T04:58:24.971 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-31T04:58:25.016 INFO:teuthology.orchestra.run.smithi188.stdout:(109/155): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-01-31T04:58:25.041 INFO:teuthology.orchestra.run.smithi188.stdout:(110/155): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-01-31T04:58:25.075 INFO:teuthology.orchestra.run.smithi188.stdout:(111/155): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-01-31T04:58:25.078 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-31T04:58:25.117 INFO:teuthology.orchestra.run.smithi188.stdout:(112/155): python3-colorama-0.4.4-12.el9.noarch 850 kB/s | 35 kB 00:00 2024-01-31T04:58:25.128 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-31T04:58:25.160 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-31T04:58:25.175 INFO:teuthology.orchestra.run.smithi188.stdout:(113/155): python3-google-auth-2.26.2-1.el9.noa 3.4 MB/s | 202 kB 00:00 2024-01-31T04:58:25.209 INFO:teuthology.orchestra.run.smithi188.stdout:(114/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-01-31T04:58:25.243 INFO:teuthology.orchestra.run.smithi188.stdout:(115/155): python3-jaraco-8.2.1-3.el9.noarch.rp 161 kB/s | 11 kB 00:00 2024-01-31T04:58:25.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-31T04:58:25.268 INFO:teuthology.orchestra.run.smithi188.stdout:(116/155): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-01-31T04:58:25.294 INFO:teuthology.orchestra.run.smithi188.stdout:(117/155): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-01-31T04:58:25.319 INFO:teuthology.orchestra.run.smithi188.stdout:(118/155): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-01-31T04:58:25.352 INFO:teuthology.orchestra.run.smithi188.stdout:(119/155): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-01-31T04:58:25.403 INFO:teuthology.orchestra.run.smithi188.stdout:(120/155): python3-logutils-0.3.5-21.el9.noarch 942 kB/s | 46 kB 00:00 2024-01-31T04:58:25.453 INFO:teuthology.orchestra.run.smithi188.stdout:(121/155): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-01-31T04:58:25.511 INFO:teuthology.orchestra.run.smithi188.stdout:(122/155): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-01-31T04:58:25.578 INFO:teuthology.orchestra.run.smithi188.stdout:(123/155): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-01-31T04:58:25.620 INFO:teuthology.orchestra.run.smithi188.stdout:(124/155): python3-kubernetes-26.1.0-2.el9.noar 3.4 MB/s | 1.0 MB 00:00 2024-01-31T04:58:25.648 INFO:teuthology.orchestra.run.smithi188.stdout:(125/155): python3-portend-3.1.0-2.el9.noarch.r 248 kB/s | 16 kB 00:00 2024-01-31T04:58:25.659 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-31T04:58:25.679 INFO:teuthology.orchestra.run.smithi188.stdout:(126/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-01-31T04:58:25.704 INFO:teuthology.orchestra.run.smithi188.stdout:(127/155): python3-pyperclip-1.8.2-2.el9.noarch 489 kB/s | 28 kB 00:00 2024-01-31T04:58:25.738 INFO:teuthology.orchestra.run.smithi188.stdout:(128/155): python3-repoze-lru-0.7-16.el9.noarch 531 kB/s | 31 kB 00:00 2024-01-31T04:58:25.746 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T04:58:25.772 INFO:teuthology.orchestra.run.smithi188.stdout:(129/155): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-01-31T04:58:25.797 INFO:teuthology.orchestra.run.smithi188.stdout:(130/155): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-01-31T04:58:25.822 INFO:teuthology.orchestra.run.smithi188.stdout:(131/155): python3-tempora-5.0.0-2.el9.noarch.r 723 kB/s | 36 kB 00:00 2024-01-31T04:58:25.856 INFO:teuthology.orchestra.run.smithi188.stdout:(132/155): python3-typing-extensions-4.4.0-2.el 880 kB/s | 51 kB 00:00 2024-01-31T04:58:25.890 INFO:teuthology.orchestra.run.smithi188.stdout:(133/155): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-01-31T04:58:25.923 INFO:teuthology.orchestra.run.smithi188.stdout:(134/155): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-01-31T04:58:25.947 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T04:58:25.977 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T04:58:25.982 INFO:teuthology.orchestra.run.smithi188.stdout:(135/155): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-01-31T04:58:26.007 INFO:teuthology.orchestra.run.smithi188.stdout:(136/155): python3-zc-lockfile-2.0-10.el9.noarc 241 kB/s | 20 kB 00:00 2024-01-31T04:58:26.049 INFO:teuthology.orchestra.run.smithi188.stdout:(137/155): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-01-31T04:58:26.083 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T04:58:26.123 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T04:58:26.259 INFO:teuthology.orchestra.run.smithi188.stdout:(138/155): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-01-31T04:58:26.267 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-31T04:58:26.380 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-31T04:58:26.664 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-31T04:58:26.666 INFO:teuthology.orchestra.run.smithi188.stdout:(139/155): librbd1-19.0.0-885.g661946e7.el9.x86 7.7 MB/s | 3.1 MB 00:00 2024-01-31T04:58:26.717 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-31T04:58:26.767 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T04:58:26.866 INFO:teuthology.orchestra.run.smithi188.stdout:(140/155): librados2-19.0.0-885.g661946e7.el9.x 4.1 MB/s | 3.4 MB 00:00 2024-01-31T04:58:26.954 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T04:58:26.985 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T04:58:27.110 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T04:58:27.434 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-31T04:58:27.542 INFO:teuthology.orchestra.run.smithi188.stdout:(141/155): NetworkManager-1.45.10-1.el9.x86_64. 2.6 MB/s | 2.3 MB 00:00 2024-01-31T04:58:27.625 INFO:teuthology.orchestra.run.smithi188.stdout:(142/155): NetworkManager-libnm-1.45.10-1.el9.x 2.4 MB/s | 1.8 MB 00:00 2024-01-31T04:58:27.631 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T04:58:27.650 INFO:teuthology.orchestra.run.smithi188.stdout:(143/155): NetworkManager-team-1.45.10-1.el9.x8 402 kB/s | 43 kB 00:00 2024-01-31T04:58:27.751 INFO:teuthology.orchestra.run.smithi188.stdout:(144/155): acl-2.3.1-4.el9.x86_64.rpm 719 kB/s | 71 kB 00:00 2024-01-31T04:58:27.776 INFO:teuthology.orchestra.run.smithi188.stdout:(145/155): NetworkManager-tui-1.45.10-1.el9.x86 1.6 MB/s | 247 kB 00:00 2024-01-31T04:58:27.796 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T04:58:27.810 INFO:teuthology.orchestra.run.smithi188.stdout:(146/155): elfutils-debuginfod-client-0.190-2.e 629 kB/s | 36 kB 00:00 2024-01-31T04:58:27.877 INFO:teuthology.orchestra.run.smithi188.stdout:(147/155): elfutils-libelf-0.190-2.el9.x86_64.r 1.9 MB/s | 192 kB 00:00 2024-01-31T04:58:27.927 INFO:teuthology.orchestra.run.smithi188.stdout:(148/155): libacl-2.3.1-4.el9.x86_64.rpm 468 kB/s | 23 kB 00:00 2024-01-31T04:58:27.961 INFO:teuthology.orchestra.run.smithi188.stdout:(149/155): elfutils-libs-0.190-2.el9.x86_64.rpm 1.7 MB/s | 256 kB 00:00 2024-01-31T04:58:28.012 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T04:58:28.076 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T04:58:28.344 INFO:teuthology.orchestra.run.smithi188.stdout:(150/155): openssl-3.0.7-25.el9.x86_64.rpm 2.9 MB/s | 1.2 MB 00:00 2024-01-31T04:58:28.470 INFO:teuthology.orchestra.run.smithi188.stdout:(151/155): xfsdump-3.1.12-4.el9.x86_64.rpm 2.5 MB/s | 323 kB 00:00 2024-01-31T04:58:28.520 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T04:58:28.557 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-31T04:58:28.611 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-31T04:58:28.620 INFO:teuthology.orchestra.run.smithi188.stdout:(152/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 3.3 MB/s | 2.2 MB 00:00 2024-01-31T04:58:28.704 INFO:teuthology.orchestra.run.smithi188.stdout:(153/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 901 kB/s | 74 kB 00:00 2024-01-31T04:58:28.719 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/171 2024-01-31T04:58:28.769 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/171 2024-01-31T04:58:28.815 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/171 2024-01-31T04:58:28.820 INFO:teuthology.orchestra.run.smithi188.stdout:(154/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 3.1 MB/s | 1.1 MB 00:00 2024-01-31T04:58:28.857 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/171 2024-01-31T04:58:28.956 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/171 2024-01-31T04:58:29.019 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/171 2024-01-31T04:58:29.051 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/171 2024-01-31T04:58:29.105 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/171 2024-01-31T04:58:29.202 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/171 2024-01-31T04:58:29.212 INFO:teuthology.orchestra.run.smithi188.stdout:(155/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-01-31T04:58:29.218 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-01-31T04:58:29.219 INFO:teuthology.orchestra.run.smithi188.stdout:Total 6.8 MB/s | 212 MB 00:31 2024-01-31T04:58:29.238 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-31T04:58:29.290 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-31T04:58:29.319 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-31T04:58:29.432 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-31T04:58:29.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-31T04:58:29.713 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-31T04:58:29.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-31T04:58:30.463 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-31T04:58:30.580 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-31T04:58:30.663 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-31T04:58:30.727 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-31T04:58:30.774 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-31T04:58:30.847 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-31T04:58:31.062 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-31T04:58:31.119 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-31T04:58:31.262 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-31T04:58:31.297 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-31T04:58:31.362 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-31T04:58:31.448 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-31T04:58:31.491 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-31T04:58:31.528 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-31T04:58:31.597 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-31T04:58:31.646 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-31T04:58:31.710 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-31T04:58:31.776 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-31T04:58:31.965 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-31T04:58:32.376 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-01-31T04:58:32.407 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-31T04:58:32.453 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-31T04:58:32.516 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-31T04:58:32.700 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-31T04:58:32.738 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-01-31T04:58:32.738 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-01-31T04:58:32.740 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-31T04:58:32.796 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-31T04:58:32.831 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-31T04:58:32.869 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-31T04:58:32.913 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 65/171 2024-01-31T04:58:32.981 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/171 2024-01-31T04:58:33.053 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 67/171 2024-01-31T04:58:33.105 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/171 2024-01-31T04:58:33.154 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/171 2024-01-31T04:58:33.200 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/171 2024-01-31T04:58:33.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/171 2024-01-31T04:58:33.333 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/171 2024-01-31T04:58:33.380 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/171 2024-01-31T04:58:33.430 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/171 2024-01-31T04:58:33.600 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 75/171 2024-01-31T04:58:33.667 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 76/171 2024-01-31T04:58:34.650 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/171 2024-01-31T04:58:34.705 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 78/171 2024-01-31T04:58:34.738 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 79/171 2024-01-31T04:58:34.872 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 80/171 2024-01-31T04:58:34.941 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/171 2024-01-31T04:58:35.040 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 82/171 2024-01-31T04:58:35.082 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 83/171 2024-01-31T04:58:35.123 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-01-31T04:58:35.124 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-01-31T04:58:35.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/171 2024-01-31T04:58:35.337 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/171 2024-01-31T04:58:35.429 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/171 2024-01-31T04:58:36.222 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/171 2024-01-31T04:58:36.364 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/171 2024-01-31T04:58:37.907 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/171 2024-01-31T04:58:37.993 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/171 2024-01-31T04:58:38.030 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 91/171 2024-01-31T04:58:38.065 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 92/171 2024-01-31T04:58:38.425 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 93/171 2024-01-31T04:58:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T04:58:38.577 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T04:58:38.605 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 95/171 2024-01-31T04:58:38.661 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 96/171 2024-01-31T04:58:39.155 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 97/171 2024-01-31T04:58:39.186 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T04:58:39.259 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T04:58:39.287 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 99/171 2024-01-31T04:58:41.658 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-01-31T04:58:41.750 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-31T04:58:41.832 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-31T04:58:41.928 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-31T04:58:41.979 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-31T04:58:42.009 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-31T04:58:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T04:58:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T04:58:42.091 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-31T04:58:42.545 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-31T04:58:42.583 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T04:58:42.695 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T04:58:42.741 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T04:58:42.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T04:58:42.768 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T04:58:42.888 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T04:58:42.919 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T04:58:42.919 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-31T04:58:42.920 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-31T04:58:42.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:58:42.928 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T04:58:42.956 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T04:58:43.088 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-31T04:58:43.180 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-31T04:58:43.462 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-31T04:58:43.516 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-31T04:58:43.541 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T04:58:43.691 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-31T04:58:43.717 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T04:58:43.800 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-31T04:58:44.105 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-31T04:58:44.264 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T04:58:44.352 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-31T04:58:44.558 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T04:58:44.617 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T04:58:45.041 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-31T04:58:45.081 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-31T04:58:45.134 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-31T04:58:45.241 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/171 2024-01-31T04:58:45.288 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/171 2024-01-31T04:58:45.332 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/171 2024-01-31T04:58:45.365 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/171 2024-01-31T04:58:45.481 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/171 2024-01-31T04:58:45.545 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/171 2024-01-31T04:58:45.609 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/171 2024-01-31T04:58:45.670 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/171 2024-01-31T04:58:45.786 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/171 2024-01-31T04:58:45.820 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-31T04:58:45.871 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-31T04:58:45.901 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-31T04:58:46.016 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-31T04:58:46.118 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-31T04:58:46.298 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-31T04:58:46.358 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-31T04:58:47.056 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-31T04:58:47.154 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-31T04:58:47.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-31T04:58:47.248 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-31T04:58:47.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-31T04:58:47.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-31T04:58:47.529 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-31T04:58:47.609 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-31T04:58:47.755 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-31T04:58:47.795 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-31T04:58:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-31T04:58:47.969 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-31T04:58:48.014 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-31T04:58:48.053 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-31T04:58:48.122 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-31T04:58:48.161 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-31T04:58:48.222 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-31T04:58:48.280 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-31T04:58:48.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-31T04:58:48.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-31T04:58:48.580 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-31T04:58:48.628 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-31T04:58:48.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-31T04:58:48.796 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-31T04:58:48.842 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-31T04:58:49.305 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-31T04:58:49.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-31T04:58:49.397 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 65/171 2024-01-31T04:58:49.454 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 66/171 2024-01-31T04:58:49.534 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 67/171 2024-01-31T04:58:49.588 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/171 2024-01-31T04:58:49.635 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/171 2024-01-31T04:58:49.680 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/171 2024-01-31T04:58:49.730 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/171 2024-01-31T04:58:49.804 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/171 2024-01-31T04:58:49.853 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/171 2024-01-31T04:58:49.902 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/171 2024-01-31T04:58:50.075 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 75/171 2024-01-31T04:58:50.137 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 76/171 2024-01-31T04:58:51.114 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/171 2024-01-31T04:58:51.169 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 78/171 2024-01-31T04:58:51.203 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 79/171 2024-01-31T04:58:51.327 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 80/171 2024-01-31T04:58:51.390 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/171 2024-01-31T04:58:51.519 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 82/171 2024-01-31T04:58:51.554 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 83/171 2024-01-31T04:58:51.777 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 84/171 2024-01-31T04:58:51.810 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 85/171 2024-01-31T04:58:51.902 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 86/171 2024-01-31T04:58:52.712 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 87/171 2024-01-31T04:58:52.849 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 88/171 2024-01-31T04:58:54.412 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 89/171 2024-01-31T04:58:54.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 90/171 2024-01-31T04:58:54.545 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 91/171 2024-01-31T04:58:54.580 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 92/171 2024-01-31T04:58:54.933 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 93/171 2024-01-31T04:58:54.972 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T04:58:55.074 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 94/171 2024-01-31T04:58:55.103 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 95/171 2024-01-31T04:58:55.158 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 96/171 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-01-31T04:58:55.567 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:58:55.632 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 97/171 2024-01-31T04:58:55.666 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T04:58:55.756 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 98/171 2024-01-31T04:58:55.784 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 99/171 2024-01-31T04:58:55.797 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T04:58:55.850 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T04:58:55.850 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:58:55.850 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-31T04:58:55.850 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:58:57.484 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T04:58:57.561 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T04:58:57.561 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:58:57.561 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-31T04:58:57.561 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T04:58:57.561 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T04:58:57.561 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:58:57.801 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T04:58:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T04:58:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:58:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-31T04:58:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T04:58:57.870 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T04:58:57.870 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:58:58.248 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T04:58:58.290 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T04:58:58.352 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T04:58:58.418 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T04:58:58.418 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:58:58.418 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-31T04:58:58.419 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T04:58:58.419 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T04:58:58.419 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:58:58.551 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 107/171 2024-01-31T04:58:58.615 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 108/171 2024-01-31T04:58:58.689 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 109/171 2024-01-31T04:58:58.886 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 100/171 2024-01-31T04:58:58.917 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 110/171 2024-01-31T04:58:58.934 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T04:58:58.981 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 111/171 2024-01-31T04:58:59.069 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 112/171 2024-01-31T04:58:59.094 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 101/171 2024-01-31T04:58:59.095 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-31T04:58:59.095 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-31T04:58:59.095 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:58:59.144 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T04:58:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 113/171 2024-01-31T04:58:59.333 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 114/171 2024-01-31T04:58:59.380 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 115/171 2024-01-31T04:58:59.429 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 116/171 2024-01-31T04:58:59.466 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 117/171 2024-01-31T04:58:59.511 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-16.el9.x86_64 118/171 2024-01-31T04:58:59.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 119/171 2024-01-31T04:59:00.236 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T04:59:00.268 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T04:59:00.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-31T04:59:00.407 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-31T04:59:00.444 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T04:59:00.769 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T04:59:00.804 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T04:59:01.930 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T04:59:01.970 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T04:59:02.113 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T04:59:02.291 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-31T04:59:02.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T04:59:02.390 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T04:59:02.390 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:02.390 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-31T04:59:02.390 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T04:59:02.390 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T04:59:02.390 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:02.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T04:59:02.476 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T04:59:02.588 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-31T04:59:07.163 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 130/171 2024-01-31T04:59:07.260 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-31T04:59:08.101 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/171 2024-01-31T04:59:08.138 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T04:59:08.217 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T04:59:08.218 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:08.218 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-31T04:59:08.218 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T04:59:08.218 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T04:59:08.218 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:08.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T04:59:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T04:59:08.345 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:08.345 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-31T04:59:08.345 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:08.415 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T04:59:08.489 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T04:59:08.489 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:08.489 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-31T04:59:08.490 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T04:59:08.490 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T04:59:08.490 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:08.849 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T04:59:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T04:59:08.925 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:08.925 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-31T04:59:08.925 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T04:59:08.925 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T04:59:08.925 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:08.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-31T04:59:09.054 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 138/171 2024-01-31T04:59:09.093 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/171 2024-01-31T04:59:09.132 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/171 2024-01-31T04:59:09.436 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 141/171 2024-01-31T04:59:09.497 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/171 2024-01-31T04:59:09.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-31T04:59:09.570 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-31T04:59:09.637 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-31T04:59:09.715 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T04:59:09.815 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 147/171 2024-01-31T04:59:09.848 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/171 2024-01-31T04:59:10.674 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T04:59:10.720 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T04:59:10.795 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T04:59:10.871 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-31T04:59:11.817 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 102/171 2024-01-31T04:59:11.817 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-01-31T04:59:11.818 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:12.026 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T04:59:12.073 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 103/171 2024-01-31T04:59:12.073 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:12.073 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-31T04:59:12.073 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:12.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-31T04:59:12.327 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-31T04:59:12.450 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/171 2024-01-31T04:59:12.505 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/171 2024-01-31T04:59:12.540 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 155/171 2024-01-31T04:59:12.575 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-31T04:59:12.610 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-31T04:59:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-31T04:59:12.653 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T04:59:12.796 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T04:59:12.844 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/171 2024-01-31T04:59:12.891 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 161/171 2024-01-31T04:59:12.923 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 162/171 2024-01-31T04:59:12.955 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 163/171 2024-01-31T04:59:12.955 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T04:59:13.134 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T04:59:13.165 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-31T04:59:13.201 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T04:59:13.202 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T04:59:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T04:59:13.514 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T04:59:13.663 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T04:59:13.706 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T04:59:13.741 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 104/171 2024-01-31T04:59:13.742 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:13.742 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-31T04:59:13.742 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T04:59:13.742 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-31T04:59:13.742 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:13.751 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-31T04:59:13.790 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-31T04:59:13.836 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-31T04:59:13.964 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T04:59:14.004 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T04:59:14.075 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 105/171 2024-01-31T04:59:14.075 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:14.075 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-31T04:59:14.075 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T04:59:14.075 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-31T04:59:14.075 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T04:59:14.207 INFO:teuthology.orchestra.run.smithi097.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-31T04:59:14.207 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:14.564 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T04:59:14.638 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-31T04:59:14.638 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:14.638 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-31T04:59:14.638 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T04:59:14.638 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-31T04:59:14.638 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:14.763 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 107/171 2024-01-31T04:59:14.826 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 108/171 2024-01-31T04:59:14.916 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 109/171 2024-01-31T04:59:15.144 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 110/171 2024-01-31T04:59:15.192 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 111/171 2024-01-31T04:59:15.277 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 112/171 2024-01-31T04:59:15.386 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 113/171 2024-01-31T04:59:15.545 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 114/171 2024-01-31T04:59:15.592 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 115/171 2024-01-31T04:59:15.633 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 116/171 2024-01-31T04:59:15.671 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 117/171 2024-01-31T04:59:15.713 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 118/171 2024-01-31T04:59:15.751 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 119/171 2024-01-31T04:59:16.391 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T04:59:16.429 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-31T04:59:16.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-31T04:59:16.606 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-31T04:59:16.640 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T04:59:16.962 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-31T04:59:16.999 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T04:59:18.133 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-31T04:59:18.166 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T04:59:18.308 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-31T04:59:18.479 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-31T04:59:18.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T04:59:18.579 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-31T04:59:18.579 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:18.579 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-31T04:59:18.579 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T04:59:18.579 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-31T04:59:18.579 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:18.635 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T04:59:18.673 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-31T04:59:18.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-31T04:59:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-31T04:59:19.270 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-31T04:59:19.271 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-31T04:59:19.272 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-16.el9.x86_64 42/171 2024-01-31T04:59:19.273 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/171 2024-01-31T04:59:19.274 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/171 2024-01-31T04:59:19.275 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 77/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 78/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/171 2024-01-31T04:59:19.276 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/171 2024-01-31T04:59:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/171 2024-01-31T04:59:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/171 2024-01-31T04:59:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/171 2024-01-31T04:59:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/171 2024-01-31T04:59:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/171 2024-01-31T04:59:19.277 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 87/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 88/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 89/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 91/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 92/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 93/171 2024-01-31T04:59:19.278 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 94/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 95/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 96/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 97/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 98/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-31T04:59:19.279 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-31T04:59:19.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-31T04:59:19.281 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-31T04:59:19.282 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 144/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 145/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 147/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 148/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 149/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 150/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-4.el9.x86_64 152/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : acl-2.3.1-3.el9.x86_64 153/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 154/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 155/171 2024-01-31T04:59:19.283 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 156/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 157/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 158/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 159/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/171 2024-01-31T04:59:19.284 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 170/171 2024-01-31T04:59:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 171/171 2024-01-31T04:59:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:21.585 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-01-31T04:59:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:21.585 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: acl-2.3.1-4.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-31T04:59:21.586 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.587 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.588 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-16.el9.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-31T04:59:21.589 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-31T04:59:21.590 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-31T04:59:21.591 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.11-1.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-31T04:59:21.592 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-31T04:59:21.593 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-31T04:59:21.594 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-31T04:59:21.595 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.596 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.597 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:21.598 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-01-31T04:59:22.276 DEBUG:teuthology.parallel:result is None 2024-01-31T04:59:23.452 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 130/171 2024-01-31T04:59:23.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-31T04:59:24.371 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/171 2024-01-31T04:59:24.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T04:59:24.476 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-31T04:59:24.476 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:24.476 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-31T04:59:24.476 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T04:59:24.476 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-31T04:59:24.476 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:24.525 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T04:59:24.593 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 134/171 2024-01-31T04:59:24.593 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:24.593 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-31T04:59:24.593 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:24.668 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T04:59:24.742 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-31T04:59:24.743 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:24.743 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-31T04:59:24.743 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T04:59:24.743 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-31T04:59:24.743 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:25.101 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T04:59:25.169 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-31T04:59:25.169 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-31T04:59:25.169 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-31T04:59:25.169 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T04:59:25.169 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-31T04:59:25.169 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:25.263 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-31T04:59:25.332 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 138/171 2024-01-31T04:59:25.370 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/171 2024-01-31T04:59:25.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/171 2024-01-31T04:59:25.726 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 141/171 2024-01-31T04:59:25.783 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/171 2024-01-31T04:59:25.815 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-31T04:59:25.864 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-31T04:59:25.909 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-31T04:59:25.990 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T04:59:26.087 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 147/171 2024-01-31T04:59:26.117 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/171 2024-01-31T04:59:27.023 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T04:59:27.073 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T04:59:27.164 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-31T04:59:27.251 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-31T04:59:28.635 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-31T04:59:28.763 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-31T04:59:28.879 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/171 2024-01-31T04:59:28.918 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/171 2024-01-31T04:59:28.943 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 155/171 2024-01-31T04:59:28.976 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-31T04:59:29.011 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-31T04:59:29.053 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-31T04:59:29.053 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T04:59:29.181 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-31T04:59:29.234 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/171 2024-01-31T04:59:29.274 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 161/171 2024-01-31T04:59:29.308 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 162/171 2024-01-31T04:59:29.340 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 163/171 2024-01-31T04:59:29.341 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T04:59:29.453 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-31T04:59:29.484 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-31T04:59:29.519 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T04:59:29.519 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T04:59:29.852 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-31T04:59:29.852 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T04:59:29.991 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-31T04:59:30.028 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-31T04:59:30.067 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-31T04:59:30.113 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-31T04:59:30.230 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T04:59:30.487 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-31T04:59:30.487 INFO:teuthology.orchestra.run.smithi188.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-31T04:59:30.487 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:35.554 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-31T04:59:35.555 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-31T04:59:35.556 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-31T04:59:35.557 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-31T04:59:35.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-31T04:59:35.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-31T04:59:35.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-31T04:59:35.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-31T04:59:35.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-31T04:59:35.571 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 42/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/171 2024-01-31T04:59:35.572 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/171 2024-01-31T04:59:35.573 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 64/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 70/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/171 2024-01-31T04:59:35.574 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 77/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 78/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/171 2024-01-31T04:59:35.575 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 87/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 88/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 89/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 91/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 92/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 93/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 94/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 95/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 96/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 97/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 98/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-31T04:59:35.576 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-31T04:59:35.577 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-31T04:59:35.578 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-31T04:59:35.578 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-31T04:59:35.578 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-31T04:59:35.579 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 144/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 145/171 2024-01-31T04:59:35.580 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 146/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 147/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 148/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 149/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 150/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 151/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 152/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 153/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 154/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 155/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 156/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 157/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 158/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 159/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 160/171 2024-01-31T04:59:35.581 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 161/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 162/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 163/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 164/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 165/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 166/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 167/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 168/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 169/171 2024-01-31T04:59:35.582 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 170/171 2024-01-31T04:59:37.813 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 171/171 2024-01-31T04:59:37.813 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:37.813 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.814 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.815 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.816 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-31T04:59:37.817 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.818 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-31T04:59:37.819 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.11-1.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.820 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-31T04:59:37.821 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-31T04:59:37.822 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.823 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-31T04:59:37.824 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-31T04:59:37.825 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-31T04:59:37.826 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-31T04:59:37.826 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-31T04:59:37.826 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:37.826 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-01-31T04:59:38.606 DEBUG:teuthology.parallel:result is None 2024-01-31T04:59:38.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T04:59:38.841 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-31T04:59:38.912 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-885.g661946e7.el9 2024-01-31T04:59:38.912 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-31T04:59:38.912 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-31T04:59:38.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-31T04:59:39.054 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-31T04:59:39.109 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-885.g661946e7.el9 2024-01-31T04:59:39.109 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-31T04:59:39.109 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-31T04:59:39.111 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-31T04:59:39.111 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:39.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-31T04:59:39.210 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:39.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-31T04:59:39.288 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-31T04:59:39.288 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:39.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-01-31T04:59:39.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-31T04:59:39.461 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:39.461 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-01-31T04:59:39.533 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-31T04:59:39.660 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-31T04:59:39.661 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:39.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-31T04:59:39.756 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-31T04:59:39.877 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:39.877 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-31T04:59:39.957 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-31T04:59:40.075 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-31T04:59:40.076 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:40.076 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-01-31T04:59:40.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-31T04:59:40.283 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:40.283 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-01-31T04:59:40.363 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-31T04:59:40.477 INFO:teuthology.run_tasks:Running task ceph... 2024-01-31T04:59:40.564 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-31T04:59:40.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2024-01-31T04:59:40.575 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-01-31T04:59:40.653 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-31T04:59:40.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-31T04:59:40.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-31T04:59:40.733 INFO:tasks.ceph:Creating extra log directories... 2024-01-31T04:59:40.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-31T04:59:40.738 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-31T04:59:40.818 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-31T04:59:40.818 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-31T04:59:40.818 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-31T04:59:40.819 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:40.819 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-31T04:59:40.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-31T04:59:41.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-31T04:59:41.202 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:41.202 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-31T04:59:41.241 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-31T04:59:41.448 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-31T04:59:41.520 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-31T04:59:41.520 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-31T04:59:41.520 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464', 'branch': 'wip-vshankar-testing-20240125.044502', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7539464', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240125.044502'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240125.044502', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9619, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'sleep_before_teardown': 0, 'subset': '293/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240125.044502', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '661946e7044723130c03627323f6a879a9848cb5', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXT2G+E8eNejApYk1lrMi8RXrp+DyrOrZfxPpm2gF18EHfuxOlRhwe49k84Q9yDGWvuDtpmI+rBg7+KZtiC31g=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNd0o8lWnbxEkjgzyJ5bQhMY9uRxoEYyIwGAmnnKRxtfWCkzpMyC8jCWqf6zSkFg8YwYsJ2+rBvJ3uOL+Q143XA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'd9fdb2209e15b39d9f061fd85399f352ce0f0894', 'timestamp': '2024-01-31_04:15:12', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-31T04:59:41.520 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-31T04:59:41.528 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-31T04:59:41.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2024-01-31T04:59:41.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-01-31T04:59:41.675 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:41.675 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-01-31T04:59:41.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-31T04:59:41.726 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 639 Links: 1 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-01-31 04:59:14.690353581 +0000 2024-01-31T04:59:41.822 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-01-31 04:56:19.833374952 +0000 2024-01-31T04:59:41.823 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-01-31 04:56:19.833374952 +0000 2024-01-31T04:59:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-01-31 04:56:19.833374952 +0000 2024-01-31T04:59:41.823 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-31T04:59:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-01-31T04:59:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-01-31T04:59:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000332954 s, 1.5 MB/s 2024-01-31T04:59:41.910 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-31T04:59:41.967 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 656 Links: 1 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-01-31 04:59:14.691353564 +0000 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-01-31 04:56:20.414365042 +0000 2024-01-31T04:59:42.016 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-01-31 04:56:20.414365042 +0000 2024-01-31T04:59:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-01-31 04:56:20.414365042 +0000 2024-01-31T04:59:42.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-31T04:59:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-01-31T04:59:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-01-31T04:59:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000349409 s, 1.5 MB/s 2024-01-31T04:59:42.108 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-31T04:59:42.163 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-01-31T04:59:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-31T04:59:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 677 Links: 1 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-01-31 04:59:14.691353564 +0000 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-01-31 04:56:20.952355865 +0000 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-01-31 04:56:20.952355865 +0000 2024-01-31T04:59:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-01-31 04:56:20.952355865 +0000 2024-01-31T04:59:42.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-31T04:59:42.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-01-31T04:59:42.294 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-01-31T04:59:42.294 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000334054 s, 1.5 MB/s 2024-01-31T04:59:42.298 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-31T04:59:42.352 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-01-31T04:59:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-31T04:59:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:42.399 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 695 Links: 1 2024-01-31T04:59:42.399 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-01-31 04:59:14.692353547 +0000 2024-01-31T04:59:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-01-31 04:56:21.535345920 +0000 2024-01-31T04:59:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-01-31 04:56:21.535345920 +0000 2024-01-31T04:59:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-01-31 04:56:21.535345920 +0000 2024-01-31T04:59:42.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-31T04:59:42.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-01-31T04:59:42.485 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-01-31T04:59:42.485 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00029313 s, 1.7 MB/s 2024-01-31T04:59:42.489 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-31T04:59:42.545 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-31T04:59:42.545 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:42.546 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-01-31T04:59:42.590 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-31T04:59:42.590 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-31 04:59:30.920124541 +0000 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-31 04:56:30.308157867 +0000 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-31 04:56:30.308157867 +0000 2024-01-31T04:59:42.670 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-01-31 04:56:30.308157867 +0000 2024-01-31T04:59:42.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-31T04:59:42.785 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-31T04:59:42.785 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-31T04:59:42.785 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000332005 s, 1.5 MB/s 2024-01-31T04:59:42.788 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-31T04:59:42.830 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-31 04:59:30.921124521 +0000 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-31 04:56:30.857147188 +0000 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-31 04:56:30.857147188 +0000 2024-01-31T04:59:42.924 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-01-31 04:56:30.857147188 +0000 2024-01-31T04:59:42.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-31T04:59:43.011 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-31T04:59:43.011 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-31T04:59:43.011 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000368169 s, 1.4 MB/s 2024-01-31T04:59:43.015 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-31T04:59:43.068 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-31 04:59:30.922124502 +0000 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-31 04:56:31.431136023 +0000 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-31 04:56:31.431136023 +0000 2024-01-31T04:59:43.117 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-01-31 04:56:31.431136023 +0000 2024-01-31T04:59:43.118 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-31T04:59:43.245 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-31T04:59:43.246 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-31T04:59:43.246 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000377979 s, 1.4 MB/s 2024-01-31T04:59:43.250 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-31T04:59:43.304 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-01-31T04:59:43.351 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-31T04:59:43.351 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-31T04:59:43.351 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-01-31T04:59:43.352 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-31T04:59:43.352 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-01-31T04:59:43.352 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-01-31 04:59:30.923124483 +0000 2024-01-31T04:59:43.352 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-01-31 04:56:32.019124585 +0000 2024-01-31T04:59:43.352 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-01-31 04:56:32.019124585 +0000 2024-01-31T04:59:43.352 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-01-31 04:56:32.019124585 +0000 2024-01-31T04:59:43.353 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-31T04:59:43.485 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-01-31T04:59:43.485 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-01-31T04:59:43.486 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000316865 s, 1.6 MB/s 2024-01-31T04:59:43.490 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-31T04:59:43.546 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-31T04:59:43.547 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-31T04:59:43.547 INFO:tasks.ceph:Generating config... 2024-01-31T04:59:43.548 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-31T04:59:43.548 INFO:tasks.ceph:[client] debug client = 20 2024-01-31T04:59:43.548 INFO:tasks.ceph:[client] debug ms = 1 2024-01-31T04:59:43.548 INFO:tasks.ceph:[client] pid file = 2024-01-31T04:59:43.548 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-31T04:59:43.548 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-31T04:59:43.548 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-31T04:59:43.548 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-31T04:59:43.548 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-31T04:59:43.549 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-31T04:59:43.549 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-31T04:59:43.549 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-31T04:59:43.549 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-31T04:59:43.550 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-31T04:59:43.550 INFO:tasks.ceph:Setting up mon.a... 2024-01-31T04:59:43.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-31T04:59:43.693 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2024-01-31T04:59:43.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-31T04:59:43.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-31T04:59:43.919 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-01-31T04:59:43.919 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-31T04:59:43.920 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:43.920 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-31T04:59:43.974 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:44.126 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid dc34a409-d027-49db-a591-dbd89a074581 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:fsid dc34a409-d027-49db-a591-dbd89a074581 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2024-01-31T04:59:44.128280+0000 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-01-31T04:59:44.128280+0000 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 17 (quincy) 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-01-31T04:59:44.127 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-31T04:59:44.130 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-31T04:59:44.178 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc34a409-d027-49db-a591-dbd89a074581... 2024-01-31T04:59:44.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-31T04:59:44.184 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-31T04:59:44.345 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2024-01-31T04:59:44.346 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2024-01-31T04:59:44.347 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2024-01-31T04:59:44.348 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = dc34a409-d027-49db-a591-dbd89a074581 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2024-01-31T04:59:44.349 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2024-01-31T04:59:44.350 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 20 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 20 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 10 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2024-01-31T04:59:44.351 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2024-01-31T04:59:44.352 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-31T04:59:44.353 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2024-01-31T04:59:44.354 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2024-01-31T04:59:44.355 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2024-01-31T04:59:44.356 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2024-01-31T04:59:44.362 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-01-31T04:59:44.362 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-01-31T04:59:44.362 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-31T04:59:44.362 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-01-31T04:59:44.362 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.362 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.363 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-31T04:59:44.364 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.97,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = dc34a409-d027-49db-a591-dbd89a074581 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.365 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-01-31T04:59:44.366 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-01-31T04:59:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-01-31T04:59:44.368 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-01-31T04:59:44.369 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-01-31T04:59:44.370 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-01-31T04:59:44.371 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-01-31T04:59:44.372 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-01-31T04:59:44.373 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-01-31T04:59:44.374 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-01-31T04:59:44.374 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-01-31T04:59:44.374 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-01-31T04:59:44.407 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-31T04:59:44.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-31T04:59:44.551 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-31T04:59:44.551 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:44.552 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-31T04:59:44.602 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:44.602 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-31T04:59:44.697 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T04:59:44.698 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:44.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-31T04:59:44.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-31T04:59:44.845 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T04:59:44.845 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:44.940 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T04:59:44.940 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:44.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-31T04:59:44.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-31T04:59:45.094 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T04:59:45.094 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:45.189 INFO:tasks.ceph:Setting up mon nodes... 2024-01-31T04:59:45.189 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-31T04:59:45.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-31T04:59:45.373 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-31T04:59:45.383 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-31T04:59:45.566 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-31T04:59:45.575 INFO:tasks.ceph:Setting up mds nodes... 2024-01-31T04:59:45.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-31T04:59:45.756 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-31T04:59:45.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-31T04:59:45.860 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-31T04:59:46.026 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-31T04:59:46.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-31T04:59:46.125 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-31T04:59:46.286 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-31T04:59:46.293 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-31T04:59:46.365 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-31T04:59:46.511 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-31T04:59:46.520 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-31T04:59:46.604 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-31T04:59:46.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-31T04:59:46.732 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-31T04:59:46.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-31T04:59:46.905 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-31T04:59:46.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-31T04:59:47.072 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-31T04:59:47.129 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-31T04:59:47.251 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-31T04:59:47.296 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-31T04:59:47.296 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-31T04:59:47.296 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-31T04:59:47.395 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-31T04:59:47.395 INFO:tasks.ceph:role: osd.0 2024-01-31T04:59:47.396 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T04:59:47.396 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-31T04:59:47.499 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T04:59:47.499 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T04:59:47.499 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T04:59:47.500 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T04:59:48.018 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-01-31T04:59:48.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-01-31T04:59:48.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-31T04:59:48.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-31T04:59:48.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-31T04:59:48.403 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-31T04:59:48.403 INFO:tasks.ceph:role: osd.1 2024-01-31T04:59:48.403 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T04:59:48.403 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-31T04:59:48.501 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T04:59:48.502 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T04:59:48.979 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-01-31T04:59:48.984 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-01-31T04:59:48.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-31T04:59:49.116 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-31T04:59:49.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-31T04:59:49.288 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-31T04:59:49.288 INFO:tasks.ceph:role: osd.2 2024-01-31T04:59:49.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T04:59:49.288 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-31T04:59:49.382 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T04:59:49.382 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T04:59:49.382 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T04:59:49.382 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T04:59:49.382 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T04:59:49.383 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-01-31T04:59:49.383 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T04:59:49.383 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T04:59:49.383 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T04:59:49.383 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T04:59:49.901 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-01-31T04:59:49.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-01-31T04:59:49.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-31T04:59:50.076 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-31T04:59:50.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-31T04:59:50.271 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-31T04:59:50.271 INFO:tasks.ceph:role: osd.3 2024-01-31T04:59:50.271 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T04:59:50.271 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-31T04:59:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T04:59:50.364 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T04:59:50.364 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T04:59:50.364 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T04:59:50.364 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T04:59:50.365 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2024-01-31T04:59:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T04:59:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T04:59:50.365 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T04:59:50.365 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T04:59:50.819 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2024-01-31T04:59:50.824 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2024-01-31T04:59:50.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-31T04:59:50.956 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-31T04:59:51.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:51.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:51.201+0000 7f0a59211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-31T04:59:51.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:51.206+0000 7f0a59211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-31T04:59:51.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:51.206+0000 7f0a59211640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-31T04:59:51.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:51.206+0000 7f0a59211640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-31T04:59:51.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:51.206+0000 7f0a59211640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-31T04:59:51.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:51.209+0000 7f0a59211640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-31T04:59:54.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-31T04:59:54.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:54.626+0000 7f4e1c502640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-31T04:59:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:54.627+0000 7f4e1c502640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-31T04:59:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:54.627+0000 7f4e1c502640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-31T04:59:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:54.627+0000 7f4e1c502640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-31T04:59:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:54.627+0000 7f4e1c502640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-31T04:59:54.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:54.629+0000 7f4e1c502640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-31T04:59:57.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-31T04:59:57.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T04:59:57.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:57.789+0000 7f5a5059a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-31T04:59:57.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:57.790+0000 7f5a5059a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-31T04:59:57.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:57.790+0000 7f5a5059a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-31T04:59:57.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:57.790+0000 7f5a5059a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-31T04:59:57.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:57.790+0000 7f5a5059a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-31T04:59:57.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T04:59:57.792+0000 7f5a5059a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-31T05:00:00.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-31T05:00:00.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:00.954+0000 7fa3ce615640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-31T05:00:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:00.955+0000 7fa3ce615640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-31T05:00:00.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:00.955+0000 7fa3ce615640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-31T05:00:00.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:00.955+0000 7fa3ce615640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-31T05:00:00.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:00.955+0000 7fa3ce615640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-31T05:00:00.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:00.957+0000 7fa3ce615640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-31T05:00:04.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-31T05:00:04.213 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-31T05:00:04.320 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-31T05:00:04.320 INFO:tasks.ceph:role: osd.4 2024-01-31T05:00:04.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:00:04.320 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-31T05:00:04.416 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T05:00:04.416 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T05:00:04.416 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T05:00:04.416 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T05:00:04.416 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T05:00:04.417 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-31T05:00:04.417 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T05:00:04.417 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T05:00:04.417 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T05:00:04.417 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T05:00:04.941 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-31T05:00:04.948 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-31T05:00:04.948 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-31T05:00:05.136 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-31T05:00:05.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-31T05:00:05.332 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-31T05:00:05.332 INFO:tasks.ceph:role: osd.5 2024-01-31T05:00:05.332 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:00:05.333 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T05:00:05.430 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T05:00:05.918 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-31T05:00:05.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-31T05:00:05.925 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-31T05:00:06.072 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-31T05:00:06.162 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-31T05:00:06.245 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-31T05:00:06.245 INFO:tasks.ceph:role: osd.6 2024-01-31T05:00:06.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:00:06.245 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T05:00:06.339 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T05:00:06.340 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T05:00:06.856 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-31T05:00:06.862 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-31T05:00:06.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-31T05:00:07.062 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-31T05:00:07.166 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-31T05:00:07.256 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-31T05:00:07.256 INFO:tasks.ceph:role: osd.7 2024-01-31T05:00:07.256 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:00:07.256 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-31T05:00:07.357 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-31T05:00:07.835 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-01-31T05:00:07.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-01-31T05:00:07.840 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-31T05:00:07.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-31T05:00:08.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:08.220 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:08.221+0000 7f45c6fbd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-31T05:00:08.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:08.224+0000 7f45c6fbd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-31T05:00:08.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:08.224+0000 7f45c6fbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-31T05:00:08.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:08.224+0000 7f45c6fbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-31T05:00:08.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:08.224+0000 7f45c6fbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-31T05:00:08.227 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:08.229+0000 7f45c6fbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-31T05:00:11.151 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-31T05:00:11.246 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:11.379 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:11.380+0000 7faba4a22640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-31T05:00:11.380 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:11.381+0000 7faba4a22640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-31T05:00:11.380 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:11.381+0000 7faba4a22640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-31T05:00:11.380 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:11.381+0000 7faba4a22640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-31T05:00:11.380 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:11.381+0000 7faba4a22640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-31T05:00:11.382 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:11.383+0000 7faba4a22640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-31T05:00:11.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:00:11.531 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:00:14.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-31T05:00:14.402 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:14.539 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:14.540+0000 7fb1da811640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-31T05:00:14.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:14.541+0000 7fb1da811640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-31T05:00:14.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:14.541+0000 7fb1da811640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-31T05:00:14.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:14.541+0000 7fb1da811640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-31T05:00:14.540 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:14.541+0000 7fb1da811640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-31T05:00:14.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:14.543+0000 7fb1da811640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-31T05:00:17.726 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-31T05:00:17.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:17.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:17.959+0000 7f49bfe0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-31T05:00:17.958 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:17.959+0000 7f49bfe0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-31T05:00:17.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:17.959+0000 7f49bfe0e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-31T05:00:17.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:17.959+0000 7f49bfe0e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-31T05:00:17.959 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:17.959+0000 7f49bfe0e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-31T05:00:17.960 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:17.962+0000 7f49bfe0e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-31T05:00:20.885 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-31T05:00:20.982 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-31T05:00:21.058 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-31T05:00:21.161 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-31T05:00:21.254 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-31T05:00:21.354 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-31T05:00:21.491 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-31T05:00:21.584 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-31T05:00:21.675 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:21.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-31T05:00:21.809 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:21.809 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-31T05:00:21.905 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:21.905 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-31T05:00:22.000 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.000 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-31T05:00:22.082 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-31T05:00:22.168 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-31T05:00:22.303 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-31T05:00:22.392 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.392 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-31T05:00:22.528 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:22.528 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-31T05:00:22.583 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:22.583 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-31T05:00:22.633 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.633 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-31T05:00:22.691 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:22.691 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-31T05:00:22.744 INFO:tasks.ceph:Adding keys to all mons... 2024-01-31T05:00:22.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-31T05:00:22.752 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-31T05:00:22.838 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2024-01-31T05:00:22.838 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDB07llWApVFhAAsTCld+Pxo/BYtCHNVlnvPQ== 2024-01-31T05:00:22.838 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2024-01-31T05:00:22.838 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDB07llYd8zLRAAXY4glNCfVX1dTNGZUbfFmQ== 2024-01-31T05:00:22.838 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2024-01-31T05:00:22.839 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llaOqoARAAR5IabzNA8GwYKZPChWBslQ== 2024-01-31T05:00:22.839 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2024-01-31T05:00:22.839 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDH07llltQODBAAlc7RryuyTlHz0D9phNkn2w== 2024-01-31T05:00:22.839 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2024-01-31T05:00:22.839 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDK07llEPFhJRAABOCmKBieoS3MS1W+SZKB3A== 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDN07llKeEkLxAAJx3yznrb9PPxr2ok3pcXAw== 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ07ll1vrzOBAAO9IwnSQp37YmgtkSvJ9pTA== 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDB07llSQTaIRAAGEtHw/4lHnjbsB87JXmqYQ== 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2024-01-31T05:00:22.840 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llbUoqERAAS5j0bqhOLOEBYeJJAcw1gg== 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llaDyVHhAAxCxPKz8CdUNYF2635sVSPg== 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDY07llYzg7DRAAcbb39NsWEVWRdIornxlExQ== 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDb07llNPq2FhAASqQSEA/5vGxqRv3LOVIW8Q== 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2024-01-31T05:00:22.841 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDe07ll9GdBIBAAW+tnotDCbokaSSouTInb/w== 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDh07llDRo1ORAASF67CwBasg4V3RLF0lvQjw== 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07ll2MQRNhAAOVu6KYtnRga+FC4ViTTGzA== 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDD07llR2txBBAA7So8sHU4Wrev8Jt381SvFA== 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2024-01-31T05:00:22.842 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDD07llpqYODxAAKYaORRDDq/CCur8PKSPKNw== 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDB07llWApVFhAAsTCld+Pxo/BYtCHNVlnvPQ== 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDB07llYd8zLRAAXY4glNCfVX1dTNGZUbfFmQ== 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDC07llaOqoARAAR5IabzNA8GwYKZPChWBslQ== 2024-01-31T05:00:22.843 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDH07llltQODBAAlc7RryuyTlHz0D9phNkn2w== 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDK07llEPFhJRAABOCmKBieoS3MS1W+SZKB3A== 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDN07llKeEkLxAAJx3yznrb9PPxr2ok3pcXAw== 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDQ07ll1vrzOBAAO9IwnSQp37YmgtkSvJ9pTA== 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDB07llSQTaIRAAGEtHw/4lHnjbsB87JXmqYQ== 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDC07llbUoqERAAS5j0bqhOLOEBYeJJAcw1gg== 2024-01-31T05:00:22.844 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDC07llaDyVHhAAxCxPKz8CdUNYF2635sVSPg== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDY07llYzg7DRAAcbb39NsWEVWRdIornxlExQ== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDb07llNPq2FhAASqQSEA/5vGxqRv3LOVIW8Q== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDe07ll9GdBIBAAW+tnotDCbokaSSouTInb/w== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDh07llDRo1ORAASF67CwBasg4V3RLF0lvQjw== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDC07ll2MQRNhAAOVu6KYtnRga+FC4ViTTGzA== 2024-01-31T05:00:22.845 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-01-31T05:00:22.846 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDD07llR2txBBAA7So8sHU4Wrev8Jt381SvFA== 2024-01-31T05:00:22.846 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2024-01-31T05:00:22.846 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDD07llpqYODxAAKYaORRDDq/CCur8PKSPKNw== 2024-01-31T05:00:22.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-31T05:00:22.896 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-31T05:00:23.038 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:23.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:23.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:23.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:23.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.778 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.785 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:23.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-31T05:00:23.934 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-31T05:00:24.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:24.089 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:24.232 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:24.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-31T05:00:24.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.540 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.684 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-31T05:00:24.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:24.973 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.123 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-31T05:00:25.568 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-31T05:00:25.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-31T05:00:25.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-31T05:00:25.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-31T05:00:25.892 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-31T05:00:25.992 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-31T05:00:26.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-31T05:00:26.234 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-31T05:00:26.322 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-31T05:00:26.471 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-31T05:00:26.556 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:26.564 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-31T05:00:26.615 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-31T05:00:26.615 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-31T05:00:26.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-31T05:00:26.623 INFO:tasks.ceph.mon.a:Started 2024-01-31T05:00:26.623 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-31T05:00:26.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-31T05:00:26.628 INFO:tasks.ceph.mon.b:Started 2024-01-31T05:00:26.628 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-31T05:00:26.628 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-31T05:00:26.633 INFO:tasks.ceph.mon.c:Started 2024-01-31T05:00:26.633 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-31T05:00:26.633 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-31T05:00:26.634 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-31T05:00:26.638 INFO:tasks.ceph.mgr.y:Started 2024-01-31T05:00:26.638 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-31T05:00:26.638 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-31T05:00:26.644 INFO:tasks.ceph.mgr.x:Started 2024-01-31T05:00:26.644 INFO:tasks.ceph:Setting crush tunables to default 2024-01-31T05:00:26.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2024-01-31T05:00:26.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.832+0000 7fd9d6200640 1 Processor -- start 2024-01-31T05:00:26.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.833+0000 7fd9d6200640 1 -- start start 2024-01-31T05:00:26.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9d6200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 0x7fd9d015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:26.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9d6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:26.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9d6200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9d0161080 0x7fd9d0161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9d6200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d00d0460 con 0x7fd9d0161080 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9d6200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d00cded0 con 0x7fd9d015d590 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9d6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d015d970 con 0x7fd9d00cfae0 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9cfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9d0161080 0x7fd9d0161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9cfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9d0161080 0x7fd9d0161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47818/0 (socket says 172.21.15.97:47818) 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9cfe00640 1 -- 172.21.15.97:0/1929116481 learned_addr learned my addr 172.21.15.97:0/1929116481 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9cea00640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.834+0000 7fd9cf400640 1 --2- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 0x7fd9d015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cea00640 1 -- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 msgr2=0x7fd9d00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cea00640 1 -- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 msgr2=0x7fd9d00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cea00640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d00cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cf400640 1 -- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 msgr2=0x7fd9d015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cf400640 1 -- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 msgr2=0x7fd9d015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-31T05:00:26.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cea00640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d00cff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cf400640 1 --2- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 0x7fd9d015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cf400640 1 --2- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 0x7fd9d015e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cfe00640 1 -- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 msgr2=0x7fd9d00cff20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cfe00640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d00cff20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cfe00640 1 -- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 msgr2=0x7fd9d015e540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cfe00640 1 --2- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 0x7fd9d015e540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.835+0000 7fd9cfe00640 1 -- 172.21.15.97:0/1929116481 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d00cf210 con 0x7fd9d0161080 2024-01-31T05:00:26.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.836+0000 7fd9cfe00640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d0161080 0x7fd9d0161510 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd9d008b830 tx=0x7fd9c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64089c0ed25a368b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:26.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.836+0000 7fd9ce000640 1 -- 172.21.15.97:0/1929116481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c802cb00 con 0x7fd9d0161080 2024-01-31T05:00:26.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.836+0000 7fd9ce000640 1 -- 172.21.15.97:0/1929116481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c802cc60 con 0x7fd9d0161080 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.836+0000 7fd9ce000640 1 -- 172.21.15.97:0/1929116481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80343f0 con 0x7fd9d0161080 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 -- 172.21.15.97:0/1929116481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d0161080 msgr2=0x7fd9d0161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d0161080 0x7fd9d0161510 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd9d008b830 tx=0x7fd9c802bf60 comp rx=0 tx=0).stop 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 -- 172.21.15.97:0/1929116481 shutdown_connections 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d0161080 0x7fd9d0161510 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 --2- 172.21.15.97:0/1929116481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d00cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 --2- 172.21.15.97:0/1929116481 >> v2:172.21.15.188:3300/0 conn(0x7fd9d015d590 0x7fd9d015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 -- 172.21.15.97:0/1929116481 >> 172.21.15.97:0/1929116481 conn(0x7fd9d009c560 msgr2=0x7fd9d00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.837+0000 7fd9d6200640 1 -- 172.21.15.97:0/1929116481 shutdown_connections 2024-01-31T05:00:26.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.838+0000 7fd9d6200640 1 -- 172.21.15.97:0/1929116481 wait complete. 2024-01-31T05:00:26.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.840+0000 7fd9d6200640 1 Processor -- start 2024-01-31T05:00:26.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.840+0000 7fd9d6200640 1 -- start start 2024-01-31T05:00:26.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9d6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d01f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9d6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 0x7fd9d00c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9d6200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0161080 0x7fd9d009e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9d6200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00c8280 con 0x7fd9d015d590 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9d6200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00c7670 con 0x7fd9d0161080 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9d6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00c7950 con 0x7fd9d00cfae0 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 0x7fd9d00c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9cf400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d01f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9cf400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d01f2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:54926/0 (socket says 172.21.15.97:54926) 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9cfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0161080 0x7fd9d009e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9cf400640 1 -- 172.21.15.97:0/4103707533 learned_addr learned my addr 172.21.15.97:0/4103707533 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:26.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.841+0000 7fd9cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 0x7fd9d00c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47848/0 (socket says 172.21.15.97:47848) 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cf400640 1 -- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0161080 msgr2=0x7fd9d009e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cf400640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0161080 0x7fd9d009e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cf400640 1 -- 172.21.15.97:0/4103707533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 msgr2=0x7fd9d00c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cf400640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 0x7fd9d00c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cf400640 1 -- 172.21.15.97:0/4103707533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8009b30 con 0x7fd9d00cfae0 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cfe00640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0161080 0x7fd9d009e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:26.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cea00640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 0x7fd9d00c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:26.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.842+0000 7fd9cf400640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d01f2700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000d7d0 tx=0x7fd9c000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.844+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000b6c0 con 0x7fd9d00cfae0 2024-01-31T05:00:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.845+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c000b840 con 0x7fd9d00cfae0 2024-01-31T05:00:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.845+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d00c7c30 con 0x7fd9d00cfae0 2024-01-31T05:00:26.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.845+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000bbf0 con 0x7fd9d00cfae0 2024-01-31T05:00:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.845+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d01f2c40 con 0x7fd9d00cfae0 2024-01-31T05:00:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.845+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd9c000bd50 con 0x7fd9d00cfae0 2024-01-31T05:00:26.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.845+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd9c0014600 con 0x7fd9d00cfae0 2024-01-31T05:00:26.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.846+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d0067830 con 0x7fd9d00cfae0 2024-01-31T05:00:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.849+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fd9c00147c0 con 0x7fd9d00cfae0 2024-01-31T05:00:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.921+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd9d00cece0 con 0x7fd9d00cfae0 2024-01-31T05:00:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.933+0000 7fd9c7400640 1 -- 172.21.15.97:0/4103707533 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd9c0014960 con 0x7fd9d00cfae0 2024-01-31T05:00:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 msgr2=0x7fd9d01f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d01f2700 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000d7d0 tx=0x7fd9c000dc90 comp rx=0 tx=0).stop 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 shutdown_connections 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0161080 0x7fd9d009e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9d015d590 0x7fd9d00c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 --2- 172.21.15.97:0/4103707533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9d00cfae0 0x7fd9d01f2700 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 >> 172.21.15.97:0/4103707533 conn(0x7fd9d009c560 msgr2=0x7fd9d006c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 shutdown_connections 2024-01-31T05:00:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:26.934+0000 7fd9d6200640 1 -- 172.21.15.97:0/4103707533 wait complete. 2024-01-31T05:00:26.950 INFO:tasks.ceph:check_enable_crimson: False 2024-01-31T05:00:26.950 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-31T05:00:26.950 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:26.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-31T05:00:27.027 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:27.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-31T05:00:27.153 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:27.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-31T05:00:27.285 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:27.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-31T05:00:27.409 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:27.409 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-31T05:00:27.410 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:27.411+0000 7fbd8563a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:00:27.472 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:27.473+0000 7fbd8563a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:00:27.501 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:27.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-31T05:00:27.579 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:27.579+0000 7f239dcda040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-31T05:00:27.590 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:27.590 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-31T05:00:27.645 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:27.646+0000 7f239dcda040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-31T05:00:27.715 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:00:27.715 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-31T05:00:27.845 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new c82a8d88-7203-47b9-8e70-5023862b4236 0 2024-01-31T05:00:28.060 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.061+0000 7fbd8563a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:00:28.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.078+0000 7f0806200640 1 Processor -- start 2024-01-31T05:00:28.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.078+0000 7f0806200640 1 -- start start 2024-01-31T05:00:28.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f0806200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08000d3d10 0x7f08000cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f0806200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08000d1200 0x7f08000d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f0806200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08000cdc90 con 0x7f08001606f0 2024-01-31T05:00:28.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f0806200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08000d0b00 con 0x7f08000d3d10 2024-01-31T05:00:28.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f0806200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08000d1b60 con 0x7f08000d1200 2024-01-31T05:00:28.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f07f7c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08000d3d10 0x7f08000cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f07f7c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08000d3d10 0x7f08000cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.188:33756/0 (socket says 172.21.15.188:33756) 2024-01-31T05:00:28.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.080+0000 7f07f7c00640 1 -- 172.21.15.188:0/3404595164 learned_addr learned my addr 172.21.15.188:0/3404595164 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:28.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.081+0000 7f0805800640 1 --2- 172.21.15.188:0/3404595164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08000d1200 0x7f08000d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.081+0000 7f0805800640 1 -- 172.21.15.188:0/3404595164 >> v2:172.21.15.97:3300/0 conn(0x7f08000d3d10 msgr2=0x7f08000cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.081+0000 7f0805800640 1 --2- 172.21.15.188:0/3404595164 >> v2:172.21.15.97:3300/0 conn(0x7f08000d3d10 0x7f08000cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.081+0000 7f0805800640 1 -- 172.21.15.188:0/3404595164 >> v1:172.21.15.97:6789/0 conn(0x7f08001606f0 legacy=0x7f08000d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.081+0000 7f0805800640 1 -- 172.21.15.188:0/3404595164 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000cdec0 con 0x7f08000d1200 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.082+0000 7f0805800640 1 --2- 172.21.15.188:0/3404595164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08000d1200 0x7f08000d1620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07f80097a0 tx=0x7f07f802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c55dc9ca0b781b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.082+0000 7f07ffe00640 1 -- 172.21.15.188:0/3404595164 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031830 con 0x7f08000d1200 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.082+0000 7f07ffe00640 1 -- 172.21.15.188:0/3404595164 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8031990 con 0x7f08000d1200 2024-01-31T05:00:28.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.082+0000 7f07ffe00640 1 -- 172.21.15.188:0/3404595164 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031d20 con 0x7f08000d1200 2024-01-31T05:00:28.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 -- 172.21.15.188:0/3404595164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08000d1200 msgr2=0x7f08000d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 --2- 172.21.15.188:0/3404595164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08000d1200 0x7f08000d1620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f07f80097a0 tx=0x7f07f802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 -- 172.21.15.188:0/3404595164 shutdown_connections 2024-01-31T05:00:28.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 --2- 172.21.15.188:0/3404595164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08000d1200 0x7f08000d1620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 --2- 172.21.15.188:0/3404595164 >> v2:172.21.15.97:3300/0 conn(0x7f08000d3d10 0x7f08000cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 -- 172.21.15.188:0/3404595164 >> 172.21.15.188:0/3404595164 conn(0x7f08000d3180 msgr2=0x7f08000d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 -- 172.21.15.188:0/3404595164 shutdown_connections 2024-01-31T05:00:28.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.083+0000 7f0806200640 1 -- 172.21.15.188:0/3404595164 wait complete. 2024-01-31T05:00:28.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.085+0000 7f0806200640 1 Processor -- start 2024-01-31T05:00:28.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.085+0000 7f0806200640 1 -- start start 2024-01-31T05:00:28.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0806200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 0x7f08000c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0806200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08000d3d10 0x7f080014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0806200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 0x7f08000c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0806200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08000c8570 con 0x7f08000d3d10 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0806200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080014f640 con 0x7f08000d1200 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0806200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080014f920 con 0x7f08001606f0 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0804e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 0x7f08000c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 0x7f08000c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 0x7f08000c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:58860/0 (socket says 172.21.15.188:58860) 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 -- 172.21.15.188:0/1178184124 learned_addr learned my addr 172.21.15.188:0/1178184124 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0804e00640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 0x7f08000c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56176/0 (socket says 172.21.15.188:56176) 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 -- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 msgr2=0x7f08000c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 0x7f08000c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.087 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 -- 172.21.15.188:0/1178184124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08000d3d10 msgr2=0x7f080014ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08000d3d10 0x7f080014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0805800640 1 -- 172.21.15.188:0/1178184124 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8009400 con 0x7f08001606f0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.086+0000 7f0804e00640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 0x7f08000c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f0805800640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 0x7f08000c88a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07f80098c0 tx=0x7f07f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f803a630 con 0x7f08001606f0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f803a790 con 0x7f08001606f0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f803ab40 con 0x7f08001606f0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080014fc00 con 0x7f08001606f0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08000b6120 con 0x7f08001606f0 2024-01-31T05:00:28.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.087+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f07f803aca0 con 0x7f08001606f0 2024-01-31T05:00:28.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.088+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f07f8043540 con 0x7f08001606f0 2024-01-31T05:00:28.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.088+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0800067dc0 con 0x7f08001606f0 2024-01-31T05:00:28.091 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.093+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f07f8048070 con 0x7f08001606f0 2024-01-31T05:00:28.123 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.123+0000 7fbd8563a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:00:28.169 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.170+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c82a8d88-7203-47b9-8e70-5023862b4236", "id": 0} v 0) v1 -- 0x7f0800072b70 con 0x7f08001606f0 2024-01-31T05:00:28.173 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.174+0000 7fbd8563a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:00:28.176 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.178+0000 7f07fd600640 1 -- 172.21.15.188:0/1178184124 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c82a8d88-7203-47b9-8e70-5023862b4236", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f07f8035020 con 0x7f08001606f0 2024-01-31T05:00:28.177 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-01-31T05:00:28.178 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 msgr2=0x7f08000c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.178 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 0x7f08000c88a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07f80098c0 tx=0x7f07f80046c0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.178 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 shutdown_connections 2024-01-31T05:00:28.178 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08001606f0 0x7f08000c88a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.178 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08000d3d10 0x7f080014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 --2- 172.21.15.188:0/1178184124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08000d1200 0x7f08000c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 >> 172.21.15.188:0/1178184124 conn(0x7f08000d3180 msgr2=0x7f08000d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.179+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 shutdown_connections 2024-01-31T05:00:28.179 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.180+0000 7f0806200640 1 -- 172.21.15.188:0/1178184124 wait complete. 2024-01-31T05:00:28.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f6bdaa6b-d5fa-405e-875b-e7cf225c2a17 1 2024-01-31T05:00:28.236 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.237+0000 7fbd8563a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:00:28.257 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.258+0000 7f239dcda040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-31T05:00:28.285 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.285+0000 7fbd8563a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:00:28.324 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.323+0000 7f239dcda040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-31T05:00:28.375 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.376+0000 7f239dcda040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-31T05:00:28.387 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.389+0000 7fc271c00640 1 Processor -- start 2024-01-31T05:00:28.388 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.389+0000 7fc271c00640 1 -- start start 2024-01-31T05:00:28.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.390+0000 7fc271c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c06f150 0x7fc26c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.390+0000 7fc271c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26c0b2ea0 con 0x7fc26c0b48d0 2024-01-31T05:00:28.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.390+0000 7fc271c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26c160b50 con 0x7fc26c160770 2024-01-31T05:00:28.389 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.390+0000 7fc271c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0b3420 con 0x7fc26c06f150 2024-01-31T05:00:28.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.390+0000 7fc26be00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc26c160770 0x7fc26c06d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:45628/0 (socket says 172.21.15.188:45628) 2024-01-31T05:00:28.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.390+0000 7fc26be00640 1 -- 172.21.15.188:0/860039713 learned_addr learned my addr 172.21.15.188:0/860039713 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-31T05:00:28.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.391+0000 7fc263e00640 1 --2- 172.21.15.188:0/860039713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c06f150 0x7fc26c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.390 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.391+0000 7fc26b400640 1 -- 172.21.15.188:0/860039713 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626498589 0 0) 0x7fc26c160b50 con 0x7fc26c160770 2024-01-31T05:00:28.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.391+0000 7fc26b400640 1 -- 172.21.15.188:0/860039713 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003520 con 0x7fc26c160770 2024-01-31T05:00:28.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.391+0000 7fc263e00640 1 -- 172.21.15.188:0/860039713 >> v1:172.21.15.188:6789/0 conn(0x7fc26c160770 legacy=0x7fc26c06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.391+0000 7fc263e00640 1 -- 172.21.15.188:0/860039713 >> v1:172.21.15.97:6789/0 conn(0x7fc26c0b48d0 legacy=0x7fc26c160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.391 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.391+0000 7fc263e00640 1 -- 172.21.15.188:0/860039713 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c072120 con 0x7fc26c06f150 2024-01-31T05:00:28.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.392+0000 7fc263e00640 1 --2- 172.21.15.188:0/860039713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c06f150 0x7fc26c0af410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc2540097d0 tx=0x7fc25402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1ad74e45f0deb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc26b400640 1 -- 172.21.15.188:0/860039713 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254030800 con 0x7fc26c06f150 2024-01-31T05:00:28.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc26b400640 1 -- 172.21.15.188:0/860039713 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254030960 con 0x7fc26c06f150 2024-01-31T05:00:28.392 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc26b400640 1 -- 172.21.15.188:0/860039713 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254030cf0 con 0x7fc26c06f150 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 -- 172.21.15.188:0/860039713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c06f150 msgr2=0x7fc26c0af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 --2- 172.21.15.188:0/860039713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c06f150 0x7fc26c0af410 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc2540097d0 tx=0x7fc25402b940 comp rx=0 tx=0).stop 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 -- 172.21.15.188:0/860039713 shutdown_connections 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 --2- 172.21.15.188:0/860039713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c06f150 0x7fc26c0af410 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 -- 172.21.15.188:0/860039713 >> 172.21.15.188:0/860039713 conn(0x7fc26c0b1b00 msgr2=0x7fc26c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 -- 172.21.15.188:0/860039713 shutdown_connections 2024-01-31T05:00:28.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.393+0000 7fc271c00640 1 -- 172.21.15.188:0/860039713 wait complete. 2024-01-31T05:00:28.394 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.395+0000 7fc271c00640 1 Processor -- start 2024-01-31T05:00:28.394 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.395+0000 7fc271c00640 1 -- start start 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 0x7fc26c146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c0b48d0 0x7fc26c0c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc26c160770 0x7fc26c0c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c146e40 con 0x7fc26c160770 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c157aa0 con 0x7fc26c06f150 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c157d80 con 0x7fc26c0b48d0 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc26be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 0x7fc26c146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc263e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c0b48d0 0x7fc26c0c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc26be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 0x7fc26c146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56186/0 (socket says 172.21.15.188:56186) 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc26be00640 1 -- 172.21.15.188:0/3476818384 learned_addr learned my addr 172.21.15.188:0/3476818384 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.396+0000 7fc271200640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc26c160770 0x7fc26c0c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.397 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc263e00640 1 -- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 msgr2=0x7fc26c146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc263e00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 0x7fc26c146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc263e00640 1 -- 172.21.15.188:0/3476818384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc26c160770 msgr2=0x7fc26c0c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc263e00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc26c160770 0x7fc26c0c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc263e00640 1 -- 172.21.15.188:0/3476818384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009400 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc26be00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 0x7fc26c146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc271200640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc26c160770 0x7fc26c0c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc263e00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c0b48d0 0x7fc26c0c4d70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc254033000 tx=0x7fc254004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25403e040 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254039cd0 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254044630 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.397+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c158060 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.398+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c158620 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.398+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc254044790 con 0x7fc26c0b48d0 2024-01-31T05:00:28.398 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.398+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc25404dcf0 con 0x7fc26c0b48d0 2024-01-31T05:00:28.399 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.399+0000 7fbd8563a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:00:28.399 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.399+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c0cfb70 con 0x7fc26c0b48d0 2024-01-31T05:00:28.399 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.401+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fc254052060 con 0x7fc26c0b48d0 2024-01-31T05:00:28.442 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.443+0000 7f239dcda040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-31T05:00:28.477 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.477+0000 7fbd8563a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:00:28.478 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.479+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f6bdaa6b-d5fa-405e-875b-e7cf225c2a17", "id": 1} v 0) v1 -- 0x7fc26c067dc0 con 0x7fc26c0b48d0 2024-01-31T05:00:28.485 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.486+0000 7fc268c00640 1 -- 172.21.15.188:0/3476818384 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6bdaa6b-d5fa-405e-875b-e7cf225c2a17", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc25405b0b0 con 0x7fc26c0b48d0 2024-01-31T05:00:28.485 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.487+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c0b48d0 msgr2=0x7fc26c0c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.487+0000 7fc271c00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c0b48d0 0x7fc26c0c4d70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc254033000 tx=0x7fc254004730 comp rx=0 tx=0).stop 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 shutdown_connections 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc26c160770 0x7fc26c0c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc26c0b48d0 0x7fc26c0c4d70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 --2- 172.21.15.188:0/3476818384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc26c06f150 0x7fc26c146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 >> 172.21.15.188:0/3476818384 conn(0x7fc26c0b1b00 msgr2=0x7fc26c0af410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 shutdown_connections 2024-01-31T05:00:28.487 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.488+0000 7fc271c00640 1 -- 172.21.15.188:0/3476818384 wait complete. 2024-01-31T05:00:28.495 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.496+0000 7f239dcda040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-31T05:00:28.502 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new b73a2bad-7afc-40e1-846f-1dc55e93edd8 2 2024-01-31T05:00:28.531 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.531+0000 7fbd8563a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:00:28.590 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.590+0000 7fbd8563a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:00:28.615 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.616+0000 7f239dcda040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-31T05:00:28.644 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:28.644+0000 7fbd8563a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:00:28.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.687+0000 7f4c92000640 1 Processor -- start 2024-01-31T05:00:28.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.688+0000 7f4c92000640 1 -- start start 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c92000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c8c1606f0 0x7f4c8c0d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c92000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c8c0cec80 0x7f4c8c0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c92000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c0cdc90 con 0x7f4c8c0d1200 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c92000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c0d0b00 con 0x7f4c8c1606f0 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c92000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c0d0880 con 0x7f4c8c0cec80 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c8be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c8c0cec80 0x7f4c8c0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c8be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c8c0cec80 0x7f4c8c0d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56188/0 (socket says 172.21.15.188:56188) 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c91600640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4c8c0d1200 0x7f4c8c0d1620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:45640/0 (socket says 172.21.15.188:45640) 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.689+0000 7f4c8be00640 1 -- 172.21.15.188:0/2608940084 learned_addr learned my addr 172.21.15.188:0/2608940084 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:28.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.690+0000 7f4c90c00640 1 --2- 172.21.15.188:0/2608940084 >> v2:172.21.15.97:3300/0 conn(0x7f4c8c1606f0 0x7f4c8c0d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.690+0000 7f4c8be00640 1 -- 172.21.15.188:0/2608940084 >> v2:172.21.15.97:3300/0 conn(0x7f4c8c1606f0 msgr2=0x7f4c8c0d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:28.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.690+0000 7f4c8be00640 1 --2- 172.21.15.188:0/2608940084 >> v2:172.21.15.97:3300/0 conn(0x7f4c8c1606f0 0x7f4c8c0d3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.690+0000 7f4c8be00640 1 -- 172.21.15.188:0/2608940084 >> v1:172.21.15.188:6789/0 conn(0x7f4c8c0d1200 legacy=0x7f4c8c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.690+0000 7f4c8b400640 1 -- 172.21.15.188:0/2608940084 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792752219 0 0) 0x7f4c8c0cdc90 con 0x7f4c8c0d1200 2024-01-31T05:00:28.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.690+0000 7f4c8be00640 1 -- 172.21.15.188:0/2608940084 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0cdec0 con 0x7f4c8c0cec80 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.691+0000 7f4c8be00640 1 --2- 172.21.15.188:0/2608940084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c0d0340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0c6c50 tx=0x7f4c7402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef009b6d65b9bd72 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.691+0000 7f4c8b400640 1 -- 172.21.15.188:0/2608940084 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74030860 con 0x7f4c8c0cec80 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.691+0000 7f4c8b400640 1 -- 172.21.15.188:0/2608940084 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c740309c0 con 0x7f4c8c0cec80 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.691+0000 7f4c8b400640 1 -- 172.21.15.188:0/2608940084 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74030d50 con 0x7f4c8c0cec80 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 -- 172.21.15.188:0/2608940084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 msgr2=0x7f4c8c0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 --2- 172.21.15.188:0/2608940084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c0d0340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0c6c50 tx=0x7f4c7402b9a0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 -- 172.21.15.188:0/2608940084 shutdown_connections 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 --2- 172.21.15.188:0/2608940084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c0d0340 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 --2- 172.21.15.188:0/2608940084 >> v2:172.21.15.97:3300/0 conn(0x7f4c8c1606f0 0x7f4c8c0d3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 -- 172.21.15.188:0/2608940084 >> 172.21.15.188:0/2608940084 conn(0x7f4c8c0d3180 msgr2=0x7f4c8c0d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 -- 172.21.15.188:0/2608940084 shutdown_connections 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.692+0000 7f4c92000640 1 -- 172.21.15.188:0/2608940084 wait complete. 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.694+0000 7f4c92000640 1 Processor -- start 2024-01-31T05:00:28.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.694+0000 7f4c92000640 1 -- start start 2024-01-31T05:00:28.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c92000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c92000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 0x7f4c8c06c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c92000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c1606f0 0x7f4c8c06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c92000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0cc2b0 con 0x7f4c8c1606f0 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c92000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0cb2a0 con 0x7f4c8c0cec80 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c92000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0cb4f0 con 0x7f4c8c0d1200 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 0x7f4c8c06c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c90c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 0x7f4c8c06c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:58870/0 (socket says 172.21.15.188:58870) 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 -- 172.21.15.188:0/911128391 learned_addr learned my addr 172.21.15.188:0/911128391 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c91600640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c1606f0 0x7f4c8c06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 -- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 msgr2=0x7f4c8c152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 -- 172.21.15.188:0/911128391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c1606f0 msgr2=0x7f4c8c06ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c1606f0 0x7f4c8c06ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c8be00640 1 -- 172.21.15.188:0/911128391 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740094d0 con 0x7f4c8c0d1200 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.695+0000 7f4c90c00640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:28.696 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c8be00640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 0x7f4c8c06c500 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c74000c00 tx=0x7f4c74004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74039b60 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c74039ce0 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c0cb7d0 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c74043650 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c0cbd90 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.696+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4c740437b0 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.697+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4c7404d440 con 0x7f4c8c0d1200 2024-01-31T05:00:28.697 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.697+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c067dc0 con 0x7f4c8c0d1200 2024-01-31T05:00:28.698 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.700+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f4c74048070 con 0x7f4c8c0d1200 2024-01-31T05:00:28.700 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.701+0000 7f239dcda040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-31T05:00:28.755 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.756+0000 7f239dcda040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-31T05:00:28.777 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.778+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b73a2bad-7afc-40e1-846f-1dc55e93edd8", "id": 2} v 0) v1 -- 0x7f4c8c072b70 con 0x7f4c8c0d1200 2024-01-31T05:00:28.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.785+0000 7f4c88c00640 1 -- 172.21.15.188:0/911128391 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b73a2bad-7afc-40e1-846f-1dc55e93edd8", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c74034070 con 0x7f4c8c0d1200 2024-01-31T05:00:28.784 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 msgr2=0x7f4c8c06c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 0x7f4c8c06c500 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4c74000c00 tx=0x7f4c74004730 comp rx=0 tx=0).stop 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 shutdown_connections 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c1606f0 0x7f4c8c06ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c8c0d1200 0x7f4c8c06c500 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 --2- 172.21.15.188:0/911128391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c0cec80 0x7f4c8c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 >> 172.21.15.188:0/911128391 conn(0x7f4c8c0d3180 msgr2=0x7f4c8c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.786 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 shutdown_connections 2024-01-31T05:00:28.787 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.787+0000 7f4c92000640 1 -- 172.21.15.188:0/911128391 wait complete. 2024-01-31T05:00:28.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 6dacd91a-90c8-4103-9c4c-3b5d22ad8b60 3 2024-01-31T05:00:28.821 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.822+0000 7f239dcda040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-31T05:00:28.880 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:28.881+0000 7f239dcda040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-31T05:00:28.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.992+0000 7f5bc7200640 1 Processor -- start 2024-01-31T05:00:28.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.993+0000 7f5bc7200640 1 -- start start 2024-01-31T05:00:28.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc7200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bc00d0040 0x7f5bc00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bc00d39f0 0x7f5bc00b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc7200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00d2190 con 0x7f5bc015d450 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc7200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3060 con 0x7f5bc00d0040 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00b2a30 con 0x7f5bc00d39f0 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc5e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5bc015d450 0x7f5bc00cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:45650/0 (socket says 172.21.15.188:45650) 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc5e00640 1 -- 172.21.15.188:0/979632292 learned_addr learned my addr 172.21.15.188:0/979632292 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.994+0000 7f5bc6800640 1 --2- 172.21.15.188:0/979632292 >> v2:172.21.15.188:3300/0 conn(0x7f5bc00d39f0 0x7f5bc00b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc5400640 1 --2- 172.21.15.188:0/979632292 >> v2:172.21.15.97:3300/0 conn(0x7f5bc00d0040 0x7f5bc00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc4a00640 1 -- 172.21.15.188:0/979632292 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2084435019 0 0) 0x7f5bc00d2190 con 0x7f5bc015d450 2024-01-31T05:00:28.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc4a00640 1 -- 172.21.15.188:0/979632292 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba0003520 con 0x7f5bc015d450 2024-01-31T05:00:28.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc6800640 1 -- 172.21.15.188:0/979632292 >> v2:172.21.15.97:3300/0 conn(0x7f5bc00d0040 msgr2=0x7f5bc00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:28.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc6800640 1 --2- 172.21.15.188:0/979632292 >> v2:172.21.15.97:3300/0 conn(0x7f5bc00d0040 0x7f5bc00b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc6800640 1 -- 172.21.15.188:0/979632292 >> v1:172.21.15.188:6789/0 conn(0x7f5bc015d450 legacy=0x7f5bc00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc4a00640 1 -- 172.21.15.188:0/979632292 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1593744915 0 0) 0x7f5ba0003520 con 0x7f5bc015d450 2024-01-31T05:00:28.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.995+0000 7f5bc6800640 1 -- 172.21.15.188:0/979632292 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00d1c30 con 0x7f5bc00d39f0 2024-01-31T05:00:28.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.997+0000 7f5bc6800640 1 --2- 172.21.15.188:0/979632292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b39c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cd6b0 tx=0x7f5bb802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bed527103d3c4a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:28.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.997+0000 7f5bc4a00640 1 -- 172.21.15.188:0/979632292 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0001670 con 0x7f5bc00d39f0 2024-01-31T05:00:28.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.997+0000 7f5bc4a00640 1 -- 172.21.15.188:0/979632292 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb802c8c0 con 0x7f5bc00d39f0 2024-01-31T05:00:28.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.997+0000 7f5bc4a00640 1 -- 172.21.15.188:0/979632292 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb802cc50 con 0x7f5bc00d39f0 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 -- 172.21.15.188:0/979632292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc00d39f0 msgr2=0x7f5bc00b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 --2- 172.21.15.188:0/979632292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b39c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cd6b0 tx=0x7f5bb802bd90 comp rx=0 tx=0).stop 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 -- 172.21.15.188:0/979632292 shutdown_connections 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 --2- 172.21.15.188:0/979632292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b39c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 --2- 172.21.15.188:0/979632292 >> v2:172.21.15.97:3300/0 conn(0x7f5bc00d0040 0x7f5bc00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 -- 172.21.15.188:0/979632292 >> 172.21.15.188:0/979632292 conn(0x7f5bc009c400 msgr2=0x7f5bc00d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 -- 172.21.15.188:0/979632292 shutdown_connections 2024-01-31T05:00:28.997 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:28.998+0000 7f5bc7200640 1 -- 172.21.15.188:0/979632292 wait complete. 2024-01-31T05:00:28.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.000+0000 7f5bc7200640 1 Processor -- start 2024-01-31T05:00:28.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.000+0000 7f5bc7200640 1 -- start start 2024-01-31T05:00:29.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 0x7f5bc0155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 0x7f5bc00b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cd060 con 0x7f5bc00d39f0 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cc1b0 con 0x7f5bc015d450 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0155a90 con 0x7f5bc00d0040 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc5e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 0x7f5bc0155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc6800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 0x7f5bc00b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc5e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 0x7f5bc0155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:58882/0 (socket says 172.21.15.188:58882) 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc5e00640 1 -- 172.21.15.188:0/3633516688 learned_addr learned my addr 172.21.15.188:0/3633516688 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc6800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 0x7f5bc00b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56204/0 (socket says 172.21.15.188:56204) 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.001+0000 7f5bc5400640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5e00640 1 -- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 msgr2=0x7f5bc00b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5e00640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 0x7f5bc00b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5e00640 1 -- 172.21.15.188:0/3633516688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc00d39f0 msgr2=0x7f5bc00b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5e00640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5e00640 1 -- 172.21.15.188:0/3633516688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8009a70 con 0x7f5bc00d0040 2024-01-31T05:00:29.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc6800640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 0x7f5bc00b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5400640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc5e00640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 0x7f5bc0155550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c750 tx=0x7f5bb000cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0005ad0 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb0005c30 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc00cc450 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0007070 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.002+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc00cca10 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.003+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5bb00071d0 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.003+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5bb000ae10 con 0x7f5bc00d0040 2024-01-31T05:00:29.003 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.003+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc00676b0 con 0x7f5bc00d0040 2024-01-31T05:00:29.006 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.007+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f5bb0007480 con 0x7f5bc00d0040 2024-01-31T05:00:29.085 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.086+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6dacd91a-90c8-4103-9c4c-3b5d22ad8b60", "id": 3} v 0) v1 -- 0x7f5bc00d3300 con 0x7f5bc00d0040 2024-01-31T05:00:29.092 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.093+0000 7f5bbe000640 1 -- 172.21.15.188:0/3633516688 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6dacd91a-90c8-4103-9c4c-3b5d22ad8b60", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5bb000a5e0 con 0x7f5bc00d0040 2024-01-31T05:00:29.092 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 msgr2=0x7f5bc0155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 0x7f5bc0155550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c750 tx=0x7f5bb000cc10 comp rx=0 tx=0).stop 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 shutdown_connections 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc015d450 0x7f5bc00b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bc00d39f0 0x7f5bc00b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 --2- 172.21.15.188:0/3633516688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc00d0040 0x7f5bc0155550 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.094 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 >> 172.21.15.188:0/3633516688 conn(0x7f5bc009c400 msgr2=0x7f5bc00b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:29.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 shutdown_connections 2024-01-31T05:00:29.095 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.095+0000 7f5bc7200640 1 -- 172.21.15.188:0/3633516688 wait complete. 2024-01-31T05:00:29.098 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.099+0000 7fbd8563a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:00:29.112 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new dfe326eb-d40b-4a65-9998-1133493036f8 4 2024-01-31T05:00:29.256 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.256+0000 7fbd8563a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:00:29.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.347+0000 7f0c1d600640 1 Processor -- start 2024-01-31T05:00:29.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.347+0000 7f0c1d600640 1 -- start start 2024-01-31T05:00:29.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.348+0000 7f0c1d600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c1815d590 0x7f0c1815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c1d600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c18161ea0 con 0x7f0c180cfae0 2024-01-31T05:00:29.349 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c1d600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c180cded0 con 0x7f0c180cff20 2024-01-31T05:00:29.349 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c1d600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1815d970 con 0x7f0c1815d590 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c17400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0c180cff20 0x7f0c18161790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:45666/0 (socket says 172.21.15.188:45666) 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c17400640 1 -- 172.21.15.188:0/3111405914 learned_addr learned my addr 172.21.15.188:0/3111405914 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c16a00640 1 --2- 172.21.15.188:0/3111405914 >> v2:172.21.15.97:3300/0 conn(0x7f0c1815d590 0x7f0c1815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730527772 0 0) 0x7f0c180cded0 con 0x7f0c180cff20 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf8003520 con 0x7f0c180cff20 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.349+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221503550 0 0) 0x7f0bf8003520 con 0x7f0c180cff20 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 >> v2:172.21.15.97:3300/0 conn(0x7f0c1815d590 msgr2=0x7f0c1815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 --2- 172.21.15.188:0/3111405914 >> v2:172.21.15.97:3300/0 conn(0x7f0c1815d590 0x7f0c1815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.351 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 >> v1:172.21.15.97:6789/0 conn(0x7f0c180cfae0 legacy=0x7f0c18161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180cf210 con 0x7f0c180cff20 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f0c100019f0 con 0x7f0c180cff20 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 >> v1:172.21.15.188:6789/0 conn(0x7f0c180cff20 legacy=0x7f0c18161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bf8003970 0x7f0bf8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf80047c0 0x7f0bf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bf8005180 0x7f0bf800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1815d970 con 0x7f0bf80047c0 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf800c030 con 0x7f0bf8005180 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf800c1f0 con 0x7f0bf8003970 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c17400640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bf8005180 0x7f0bf800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c16a00640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bf8003970 0x7f0bf8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.350+0000 7f0c16000640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf80047c0 0x7f0bf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.351+0000 7f0c17400640 1 -- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bf8003970 msgr2=0x7f0bf8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:29.352 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.351+0000 7f0c17400640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bf8003970 0x7f0bf8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.353 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.351+0000 7f0c17400640 1 -- 172.21.15.188:0/3111405914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf80047c0 msgr2=0x7f0bf8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:29.353 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.351+0000 7f0c17400640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf80047c0 0x7f0bf8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.353 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.351+0000 7f0c17400640 1 -- 172.21.15.188:0/3111405914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf8003520 con 0x7f0bf8005180 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.352+0000 7f0c17400640 1 -- 172.21.15.188:0/3111405914 reap_dead start 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.352+0000 7f0c17400640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bf8005180 0x7f0bf800b980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000a830 tx=0x7f0bf80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86771ca5d2194dcf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.352+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100072d0 con 0x7f0bf8005180 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.352+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1002f070 con 0x7f0bf8005180 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.352+0000 7f0c15600640 1 -- 172.21.15.188:0/3111405914 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100076b0 con 0x7f0bf8005180 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 -- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bf8005180 msgr2=0x7f0bf800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bf8005180 0x7f0bf800b980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000a830 tx=0x7f0bf80057f0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 -- 172.21.15.188:0/3111405914 shutdown_connections 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 --2- 172.21.15.188:0/3111405914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bf8005180 0x7f0bf800b980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 -- 172.21.15.188:0/3111405914 >> 172.21.15.188:0/3111405914 conn(0x7f0c1809c560 msgr2=0x7f0c180d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:29.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 -- 172.21.15.188:0/3111405914 shutdown_connections 2024-01-31T05:00:29.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.353+0000 7f0c1d600640 1 -- 172.21.15.188:0/3111405914 wait complete. 2024-01-31T05:00:29.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.355+0000 7f0c1d600640 1 Processor -- start 2024-01-31T05:00:29.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.355+0000 7f0c1d600640 1 -- start start 2024-01-31T05:00:29.355 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:29.355+0000 7f239dcda040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c1d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf8005180 0x7f0c181f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c1d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c1809d680 0x7f0c1809db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c1d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 0x7f0c180a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c1d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1809e040 con 0x7f0bf8005180 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c1d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181f2b70 con 0x7f0c180a01e0 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c1d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c181f2e30 con 0x7f0c1809d680 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 0x7f0c180a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 0x7f0c180a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56216/0 (socket says 172.21.15.188:56216) 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c16000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c1809d680 0x7f0c1809db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 -- 172.21.15.188:0/3589226136 learned_addr learned my addr 172.21.15.188:0/3589226136 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c16a00640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf8005180 0x7f0c181f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 -- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c1809d680 msgr2=0x7f0c1809db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c1809d680 0x7f0c1809db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.358 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 -- 172.21.15.188:0/3589226136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf8005180 msgr2=0x7f0c181f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf8005180 0x7f0c181f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c17400640 1 -- 172.21.15.188:0/3589226136 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c181f3110 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.356+0000 7f0c16a00640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf8005180 0x7f0c181f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c17400640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 0x7f0c180a0660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000a950 tx=0x7f0c10001d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100019b0 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10004b60 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c181f33f0 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007450 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c181f5dc0 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0c10031030 con 0x7f0c180a01e0 2024-01-31T05:00:29.359 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.357+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0c10034940 con 0x7f0c180a01e0 2024-01-31T05:00:29.360 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.358+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18067830 con 0x7f0c180a01e0 2024-01-31T05:00:29.361 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.361+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f0c10043070 con 0x7f0c180a01e0 2024-01-31T05:00:29.390 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.390+0000 7fbd8563a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:00:29.444 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.445+0000 7fbd8563a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:00:29.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.507+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dfe326eb-d40b-4a65-9998-1133493036f8", "id": 4} v 0) v1 -- 0x7f0c181f3910 con 0x7f0c180a01e0 2024-01-31T05:00:29.516 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.516+0000 7f0c0ea00640 1 -- 172.21.15.188:0/3589226136 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dfe326eb-d40b-4a65-9998-1133493036f8", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c10039040 con 0x7f0c180a01e0 2024-01-31T05:00:29.516 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 msgr2=0x7f0c180a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 0x7f0c180a0660 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000a950 tx=0x7f0c10001d80 comp rx=0 tx=0).stop 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 shutdown_connections 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c180a01e0 0x7f0c180a0660 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c1809d680 0x7f0c1809db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 --2- 172.21.15.188:0/3589226136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bf8005180 0x7f0c181f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.517+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 >> 172.21.15.188:0/3589226136 conn(0x7f0c1809c560 msgr2=0x7f0c18161670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.518+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 shutdown_connections 2024-01-31T05:00:29.518 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.518+0000 7f0c1d600640 1 -- 172.21.15.188:0/3589226136 wait complete. 2024-01-31T05:00:29.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c 5 2024-01-31T05:00:29.554 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.554+0000 7fbd8563a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:00:29.577 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:29.577+0000 7f239dcda040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-31T05:00:29.663 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.664+0000 7fbd8563a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:00:29.722 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:29.722+0000 7f239dcda040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-31T05:00:29.724 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.725+0000 7f2845e00640 1 Processor -- start 2024-01-31T05:00:29.724 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.725+0000 7f2845e00640 1 -- start start 2024-01-31T05:00:29.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.726+0000 7f2845e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f28401606f0 0x7f28400d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.726+0000 7f2845e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28400cdc90 con 0x7f28400d1200 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.726+0000 7f2845e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28400d0b00 con 0x7f28400cec80 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.726+0000 7f2845e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28400cdec0 con 0x7f28401606f0 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.726+0000 7f2837e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f28400cec80 0x7f28400d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:45674/0 (socket says 172.21.15.188:45674) 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.726+0000 7f2837e00640 1 -- 172.21.15.188:0/2645082096 learned_addr learned my addr 172.21.15.188:0/2645082096 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f2844a00640 1 --2- 172.21.15.188:0/2645082096 >> v2:172.21.15.97:3300/0 conn(0x7f28401606f0 0x7f28400d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290242249 0 0) 0x7f28400d0b00 con 0x7f28400cec80 2024-01-31T05:00:29.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2820003520 con 0x7f28400cec80 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037071534 0 0) 0x7f2820003520 con 0x7f28400cec80 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 >> v2:172.21.15.97:3300/0 conn(0x7f28401606f0 msgr2=0x7f28400d3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 --2- 172.21.15.188:0/2645082096 >> v2:172.21.15.97:3300/0 conn(0x7f28401606f0 0x7f28400d3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 >> v1:172.21.15.97:6789/0 conn(0x7f28400d1200 legacy=0x7f28400d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400ce1a0 con 0x7f28400cec80 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f2828001670 con 0x7f28400cec80 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 >> v1:172.21.15.188:6789/0 conn(0x7f28400cec80 legacy=0x7f28400d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28200039e0 0x7f2820003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f2820004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.730 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2820005180 0x7f282000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400cdec0 con 0x7f28200039e0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282000c0b0 con 0x7f28200047c0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282000c270 con 0x7f2820005180 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.727+0000 7f2837e00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f2820004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2845400640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2820005180 0x7f282000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2844a00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28200039e0 0x7f2820003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2837e00640 1 -- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2820005180 msgr2=0x7f282000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2837e00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2820005180 0x7f282000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2837e00640 1 -- 172.21.15.188:0/2645082096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28200039e0 msgr2=0x7f2820003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2837e00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28200039e0 0x7f2820003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2837e00640 1 -- 172.21.15.188:0/2645082096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820003520 con 0x7f28200047c0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.728+0000 7f2845400640 1 -- 172.21.15.188:0/2645082096 reap_dead start 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.729+0000 7f2837e00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f2820004c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f282800c240 tx=0x7f28200057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d73de24ee2bd79e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:29.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.729+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828005ad0 con 0x7f28200047c0 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.729+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2828005c50 con 0x7f28200047c0 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.729+0000 7f283fe00640 1 -- 172.21.15.188:0/2645082096 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282802c870 con 0x7f28200047c0 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 -- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 msgr2=0x7f2820004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f2820004c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f282800c240 tx=0x7f28200057f0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 -- 172.21.15.188:0/2645082096 shutdown_connections 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 --2- 172.21.15.188:0/2645082096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f2820004c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 -- 172.21.15.188:0/2645082096 >> 172.21.15.188:0/2645082096 conn(0x7f28400d3180 msgr2=0x7f28400d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 -- 172.21.15.188:0/2645082096 shutdown_connections 2024-01-31T05:00:29.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.730+0000 7f2845e00640 1 -- 172.21.15.188:0/2645082096 wait complete. 2024-01-31T05:00:29.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.732+0000 7f2845e00640 1 Processor -- start 2024-01-31T05:00:29.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.732+0000 7f2845e00640 1 -- start start 2024-01-31T05:00:29.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f28401f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f284014d890 0x7f28400c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28400cc980 0x7f28400cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400c8020 con 0x7f284014d890 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400c6d20 con 0x7f28200047c0 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28400c7000 con 0x7f28400cc980 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28400cc980 0x7f28400cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f28401f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f28401f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56224/0 (socket says 172.21.15.188:56224) 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2845400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28400cc980 0x7f28400cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:58910/0 (socket says 172.21.15.188:58910) 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 -- 172.21.15.188:0/2295091507 learned_addr learned my addr 172.21.15.188:0/2295091507 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2837e00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f284014d890 0x7f28400c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 -- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28400cc980 msgr2=0x7f28400cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.735 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28400cc980 0x7f28400cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 -- 172.21.15.188:0/2295091507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f284014d890 msgr2=0x7f28400c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f284014d890 0x7f28400c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2844a00640 1 -- 172.21.15.188:0/2295091507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400c72e0 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.733+0000 7f2837e00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f284014d890 0x7f28400c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f2844a00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f28401f6e70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2830011600 tx=0x7f2830011ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000f920 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000fa80 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000fdf0 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28400c75c0 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284014dd10 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2830005990 con 0x7f28200047c0 2024-01-31T05:00:29.736 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.734+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2830005c40 con 0x7f28200047c0 2024-01-31T05:00:29.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.735+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284006d120 con 0x7f28200047c0 2024-01-31T05:00:29.737 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.738+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f2830022020 con 0x7f28200047c0 2024-01-31T05:00:29.779 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:29.779+0000 7f239dcda040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-31T05:00:29.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.816+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c", "id": 5} v 0) v1 -- 0x7f2840067dc0 con 0x7f28200047c0 2024-01-31T05:00:29.823 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.824+0000 7f283d600640 1 -- 172.21.15.188:0/2295091507 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2830005f00 con 0x7f28200047c0 2024-01-31T05:00:29.824 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.823+0000 7fbd8563a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:00:29.824 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 msgr2=0x7f28401f6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f28401f6e70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2830011600 tx=0x7f2830011ac0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 shutdown_connections 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28400cc980 0x7f28400cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f284014d890 0x7f28400c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 --2- 172.21.15.188:0/2295091507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28200047c0 0x7f28401f6e70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 >> 172.21.15.188:0/2295091507 conn(0x7f28400d3180 msgr2=0x7f28400d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:29.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 shutdown_connections 2024-01-31T05:00:29.826 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:29.825+0000 7f2845e00640 1 -- 172.21.15.188:0/2295091507 wait complete. 2024-01-31T05:00:29.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 0f493702-f50c-47f9-abe7-82feeea46aac 6 2024-01-31T05:00:29.873 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:29.874+0000 7fbd8563a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:00:29.896 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:29.896+0000 7f239dcda040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:00:30.014 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:30.014+0000 7f239dcda040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-31T05:00:30.028 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:30.028+0000 7fbd8563a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:00:30.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.074+0000 7f0944200640 1 Processor -- start 2024-01-31T05:00:30.074 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.074+0000 7f0944200640 1 -- start start 2024-01-31T05:00:30.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0944200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f093c0b3e90 0x7f093c0b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0944200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0d1b20 con 0x7f093c0cfca0 2024-01-31T05:00:30.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0944200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0d1e00 con 0x7f093c0d3bb0 2024-01-31T05:00:30.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0944200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c0d20e0 con 0x7f093c0b3e90 2024-01-31T05:00:30.076 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0940e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f093c0d3bb0 0x7f093c0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:45686/0 (socket says 172.21.15.188:45686) 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0940e00640 1 -- 172.21.15.188:0/2948688617 learned_addr learned my addr 172.21.15.188:0/2948688617 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0942200640 1 --2- 172.21.15.188:0/2948688617 >> v2:172.21.15.188:3300/0 conn(0x7f093c0b3e90 0x7f093c0b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0942200640 1 -- 172.21.15.188:0/2948688617 >> v1:172.21.15.188:6789/0 conn(0x7f093c0d3bb0 legacy=0x7f093c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0942200640 1 -- 172.21.15.188:0/2948688617 >> v1:172.21.15.97:6789/0 conn(0x7f093c0cfca0 legacy=0x7f093c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.076+0000 7f0942200640 1 -- 172.21.15.188:0/2948688617 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0d4120 con 0x7f093c0b3e90 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.077+0000 7f0942200640 1 --2- 172.21.15.188:0/2948688617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0b3e90 0x7f093c0b3860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0934009d90 tx=0x7f093402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564f35a37b598ae4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:30.077 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f093be00640 1 -- 172.21.15.188:0/2948688617 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093402cc60 con 0x7f093c0b3e90 2024-01-31T05:00:30.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f093be00640 1 -- 172.21.15.188:0/2948688617 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093402cdc0 con 0x7f093c0b3e90 2024-01-31T05:00:30.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f093be00640 1 -- 172.21.15.188:0/2948688617 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934034660 con 0x7f093c0b3e90 2024-01-31T05:00:30.078 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 -- 172.21.15.188:0/2948688617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0b3e90 msgr2=0x7f093c0b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 --2- 172.21.15.188:0/2948688617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0b3e90 0x7f093c0b3860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0934009d90 tx=0x7f093402beb0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 -- 172.21.15.188:0/2948688617 shutdown_connections 2024-01-31T05:00:30.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 --2- 172.21.15.188:0/2948688617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0b3e90 0x7f093c0b3860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 -- 172.21.15.188:0/2948688617 >> 172.21.15.188:0/2948688617 conn(0x7f093c09c570 msgr2=0x7f093c0d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:30.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 -- 172.21.15.188:0/2948688617 shutdown_connections 2024-01-31T05:00:30.079 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.078+0000 7f0944200640 1 -- 172.21.15.188:0/2948688617 wait complete. 2024-01-31T05:00:30.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.080+0000 7f0944200640 1 Processor -- start 2024-01-31T05:00:30.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.080+0000 7f0944200640 1 -- start start 2024-01-31T05:00:30.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0944200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093c0b3e90 0x7f093c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.082 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0944200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 0x7f093c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0944200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093c0d3bb0 0x7f093c0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0944200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b0870 con 0x7f093c0b3e90 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0944200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b13f0 con 0x7f093c0cfca0 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0944200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0b1880 con 0x7f093c0d3bb0 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0940e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 0x7f093c0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0942200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093c0d3bb0 0x7f093c0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0940e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 0x7f093c0b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56230/0 (socket says 172.21.15.188:56230) 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0940e00640 1 -- 172.21.15.188:0/3170311555 learned_addr learned my addr 172.21.15.188:0/3170311555 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.081+0000 7f0941800640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093c0b3e90 0x7f093c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0942200640 1 -- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 msgr2=0x7f093c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0942200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 0x7f093c0b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.083 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0942200640 1 -- 172.21.15.188:0/3170311555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093c0b3e90 msgr2=0x7f093c0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0942200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093c0b3e90 0x7f093c0c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0942200640 1 -- 172.21.15.188:0/3170311555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934009a70 con 0x7f093c0d3bb0 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0940e00640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 0x7f093c0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0942200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093c0d3bb0 0x7f093c0ca360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0934033000 tx=0x7f0934004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934040650 con 0x7f093c0d3bb0 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09340407b0 con 0x7f093c0d3bb0 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c0b0ac0 con 0x7f093c0d3bb0 2024-01-31T05:00:30.084 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934040b60 con 0x7f093c0d3bb0 2024-01-31T05:00:30.085 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.082+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0b1080 con 0x7f093c0d3bb0 2024-01-31T05:00:30.085 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.083+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0934049790 con 0x7f093c0d3bb0 2024-01-31T05:00:30.085 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.083+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f093403ed10 con 0x7f093c0d3bb0 2024-01-31T05:00:30.085 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.084+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c0d0080 con 0x7f093c0d3bb0 2024-01-31T05:00:30.086 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.086+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f0934067db0 con 0x7f093c0d3bb0 2024-01-31T05:00:30.168 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.164+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0f493702-f50c-47f9-abe7-82feeea46aac", "id": 6} v 0) v1 -- 0x7f093c0b8700 con 0x7f093c0d3bb0 2024-01-31T05:00:30.171 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.172+0000 7f0939600640 1 -- 172.21.15.188:0/3170311555 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f493702-f50c-47f9-abe7-82feeea46aac", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0934056030 con 0x7f093c0d3bb0 2024-01-31T05:00:30.172 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-01-31T05:00:30.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093c0d3bb0 msgr2=0x7f093c0ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.173 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093c0d3bb0 0x7f093c0ca360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0934033000 tx=0x7f0934004a40 comp rx=0 tx=0).stop 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 shutdown_connections 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f093c0d3bb0 0x7f093c0ca360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c0cfca0 0x7f093c0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 --2- 172.21.15.188:0/3170311555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f093c0b3e90 0x7f093c0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.173+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 >> 172.21.15.188:0/3170311555 conn(0x7f093c09c570 msgr2=0x7f093c06bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.174+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 shutdown_connections 2024-01-31T05:00:30.174 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.174+0000 7f0944200640 1 -- 172.21.15.188:0/3170311555 wait complete. 2024-01-31T05:00:30.189 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:30.189+0000 7f239dcda040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-31T05:00:30.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 016a4ce2-df66-41f4-b2fa-9e1db0c7ef34 7 2024-01-31T05:00:30.243 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:30.240+0000 7f239dcda040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-31T05:00:30.405 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:30.404+0000 7f239dcda040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-31T05:00:30.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.432+0000 7f36e8c00640 1 Processor -- start 2024-01-31T05:00:30.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.432+0000 7f36e8c00640 1 -- start start 2024-01-31T05:00:30.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e8c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f36e4160770 0x7f36e40af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e8c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36e406f150 0x7f36e406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e40b48d0 0x7f36e40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e8c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40b3420 con 0x7f36e4160770 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e8c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e4071f10 con 0x7f36e406f150 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e8c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40b2ea0 con 0x7f36e40b48d0 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e2a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36e406f150 0x7f36e406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e40b48d0 0x7f36e40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e2a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36e406f150 0x7f36e406d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56238/0 (socket says 172.21.15.188:56238) 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e2a00640 1 -- 172.21.15.188:0/1412415523 learned_addr learned my addr 172.21.15.188:0/1412415523 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.434+0000 7f36e3400640 1 --2- 172.21.15.188:0/1412415523 >> v2:172.21.15.97:3300/0 conn(0x7f36e4160770 0x7f36e40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.435+0000 7f36e2a00640 1 -- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e40b48d0 msgr2=0x7f36e40b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.435+0000 7f36e2a00640 1 --2- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e40b48d0 0x7f36e40b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.435+0000 7f36e2a00640 1 -- 172.21.15.188:0/1412415523 >> v2:172.21.15.97:3300/0 conn(0x7f36e4160770 msgr2=0x7f36e40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:30.435 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.435+0000 7f36e2a00640 1 --2- 172.21.15.188:0/1412415523 >> v2:172.21.15.97:3300/0 conn(0x7f36e4160770 0x7f36e40af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.435+0000 7f36e2a00640 1 -- 172.21.15.188:0/1412415523 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e406d8e0 con 0x7f36e406f150 2024-01-31T05:00:30.436 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.435+0000 7f36e3e00640 1 --2- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e40b48d0 0x7f36e40b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:30.437 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.436+0000 7f36e2a00640 1 --2- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e406f150 0x7f36e406d3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f36e406c7c0 tx=0x7f36cc02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d224c58f8fbcf206 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:30.437 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.436+0000 7f36e2000640 1 -- 172.21.15.188:0/1412415523 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc031870 con 0x7f36e406f150 2024-01-31T05:00:30.437 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.436+0000 7f36e2000640 1 -- 172.21.15.188:0/1412415523 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc0319d0 con 0x7f36e406f150 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.436+0000 7f36e2000640 1 -- 172.21.15.188:0/1412415523 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc031d60 con 0x7f36e406f150 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 -- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e406f150 msgr2=0x7f36e406d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 --2- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e406f150 0x7f36e406d3a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f36e406c7c0 tx=0x7f36cc02b910 comp rx=0 tx=0).stop 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 -- 172.21.15.188:0/1412415523 shutdown_connections 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 --2- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e40b48d0 0x7f36e40b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 --2- 172.21.15.188:0/1412415523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e406f150 0x7f36e406d3a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 --2- 172.21.15.188:0/1412415523 >> v2:172.21.15.97:3300/0 conn(0x7f36e4160770 0x7f36e40af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 -- 172.21.15.188:0/1412415523 >> 172.21.15.188:0/1412415523 conn(0x7f36e40b1b00 msgr2=0x7f36e40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 -- 172.21.15.188:0/1412415523 shutdown_connections 2024-01-31T05:00:30.438 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.437+0000 7f36e8c00640 1 -- 172.21.15.188:0/1412415523 wait complete. 2024-01-31T05:00:30.439 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.438+0000 7f36e8c00640 1 Processor -- start 2024-01-31T05:00:30.439 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.439+0000 7f36e8c00640 1 -- start start 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.439+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e406f150 0x7f36e41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.439+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 0x7f36e41507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.439+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e4160770 0x7f36e406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e8c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4158820 con 0x7f36e406f150 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e8c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4157240 con 0x7f36e40b48d0 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e8c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4157520 con 0x7f36e4160770 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 0x7f36e41507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e4160770 0x7f36e406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 0x7f36e41507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:56240/0 (socket says 172.21.15.188:56240) 2024-01-31T05:00:30.441 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 -- 172.21.15.188:0/2234336722 learned_addr learned my addr 172.21.15.188:0/2234336722 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e3400640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e406f150 0x7f36e41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 -- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e4160770 msgr2=0x7f36e406a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e4160770 0x7f36e406a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 -- 172.21.15.188:0/2234336722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e406f150 msgr2=0x7f36e41582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e406f150 0x7f36e41582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 -- 172.21.15.188:0/2234336722 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36dc009a70 con 0x7f36e40b48d0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e3e00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e4160770 0x7f36e406a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.440+0000 7f36e2a00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 0x7f36e41507e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f36cc000c00 tx=0x7f36cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc03abe0 con 0x7f36e40b48d0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc03ad40 con 0x7f36e40b48d0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc044630 con 0x7f36e40b48d0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36cc0094d0 con 0x7f36e40b48d0 2024-01-31T05:00:30.442 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e4157ae0 con 0x7f36e40b48d0 2024-01-31T05:00:30.443 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f36cc044790 con 0x7f36e40b48d0 2024-01-31T05:00:30.443 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.441+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f36cc02d040 con 0x7f36e40b48d0 2024-01-31T05:00:30.444 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.442+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4157660 con 0x7f36e40b48d0 2024-01-31T05:00:30.444 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.444+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f36e4157660 con 0x7f36e40b48d0 2024-01-31T05:00:30.522 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.522+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "016a4ce2-df66-41f4-b2fa-9e1db0c7ef34", "id": 7} v 0) v1 -- 0x7f36e4067dc0 con 0x7f36e40b48d0 2024-01-31T05:00:30.532 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.532+0000 7f36db400640 1 -- 172.21.15.188:0/2234336722 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "016a4ce2-df66-41f4-b2fa-9e1db0c7ef34", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f36cc052060 con 0x7f36e40b48d0 2024-01-31T05:00:30.532 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-01-31T05:00:30.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 msgr2=0x7f36e41507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 0x7f36e41507e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f36cc000c00 tx=0x7f36cc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 shutdown_connections 2024-01-31T05:00:30.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36e4160770 0x7f36e406a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e40b48d0 0x7f36e41507e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.534 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 --2- 172.21.15.188:0/2234336722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e406f150 0x7f36e41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.535 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 >> 172.21.15.188:0/2234336722 conn(0x7f36e40b1b00 msgr2=0x7f36e40cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:30.535 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 shutdown_connections 2024-01-31T05:00:30.535 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:00:30.534+0000 7f36e8c00640 1 -- 172.21.15.188:0/2234336722 wait complete. 2024-01-31T05:00:30.552 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-31T05:00:30.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-31T05:00:30.557 INFO:tasks.ceph.osd.0:Started 2024-01-31T05:00:30.557 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-31T05:00:30.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-31T05:00:30.562 INFO:tasks.ceph.osd.1:Started 2024-01-31T05:00:30.562 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-31T05:00:30.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-31T05:00:30.567 INFO:tasks.ceph.osd.2:Started 2024-01-31T05:00:30.567 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-31T05:00:30.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-31T05:00:30.571 INFO:tasks.ceph.osd.3:Started 2024-01-31T05:00:30.571 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-31T05:00:30.571 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-31T05:00:30.576 INFO:tasks.ceph.osd.4:Started 2024-01-31T05:00:30.576 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-31T05:00:30.576 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-31T05:00:30.580 INFO:tasks.ceph.osd.5:Started 2024-01-31T05:00:30.581 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-31T05:00:30.581 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-31T05:00:30.585 INFO:tasks.ceph.osd.6:Started 2024-01-31T05:00:30.585 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-31T05:00:30.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-31T05:00:30.590 INFO:tasks.ceph.osd.7:Started 2024-01-31T05:00:30.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:30.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.788+0000 7fad5c000640 1 Processor -- start 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.788+0000 7fad5c000640 1 -- start start 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.789+0000 7fad5c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540cb7d0 0x7fad540b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad5c000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad5406c350 0x7fad540ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad5c000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5406e0a0 con 0x7fad5406dc40 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad5c000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5406cba0 con 0x7fad5406c350 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad5c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5406ce80 con 0x7fad540cb7d0 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad59600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540cb7d0 0x7fad540b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad58c00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fad5406dc40 0x7fad540cbbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:42172/0 (socket says 172.21.15.97:42172) 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad58c00640 1 -- 172.21.15.97:0/1799121255 learned_addr learned my addr 172.21.15.97:0/1799121255 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad5a000640 1 --2- 172.21.15.97:0/1799121255 >> v2:172.21.15.97:3300/0 conn(0x7fad5406c350 0x7fad540ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad53e00640 1 -- 172.21.15.97:0/1799121255 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454433835 0 0) 0x7fad5406e0a0 con 0x7fad5406dc40 2024-01-31T05:00:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad53e00640 1 -- 172.21.15.97:0/1799121255 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad34003520 con 0x7fad5406dc40 2024-01-31T05:00:30.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad59600640 1 -- 172.21.15.97:0/1799121255 >> v2:172.21.15.97:3300/0 conn(0x7fad5406c350 msgr2=0x7fad540ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:30.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad59600640 1 --2- 172.21.15.97:0/1799121255 >> v2:172.21.15.97:3300/0 conn(0x7fad5406c350 0x7fad540ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad59600640 1 -- 172.21.15.97:0/1799121255 >> v1:172.21.15.188:6789/0 conn(0x7fad5406dc40 legacy=0x7fad540cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad53e00640 1 -- 172.21.15.97:0/1799121255 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632436655 0 0) 0x7fad34003520 con 0x7fad5406dc40 2024-01-31T05:00:30.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.790+0000 7fad59600640 1 -- 172.21.15.97:0/1799121255 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5406d160 con 0x7fad540cb7d0 2024-01-31T05:00:30.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.791+0000 7fad5a000640 1 --2- 172.21.15.97:0/1799121255 >> v2:172.21.15.97:3300/0 conn(0x7fad5406c350 0x7fad540ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.792+0000 7fad59600640 1 --2- 172.21.15.97:0/1799121255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540cb7d0 0x7fad540b2e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad440097a0 tx=0x7fad4402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12883e1013247549 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.792+0000 7fad53e00640 1 -- 172.21.15.97:0/1799121255 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c001670 con 0x7fad540cb7d0 2024-01-31T05:00:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.792+0000 7fad53e00640 1 -- 172.21.15.97:0/1799121255 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4402e050 con 0x7fad540cb7d0 2024-01-31T05:00:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.792+0000 7fad53e00640 1 -- 172.21.15.97:0/1799121255 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44030730 con 0x7fad540cb7d0 2024-01-31T05:00:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.792+0000 7fad5c000640 1 -- 172.21.15.97:0/1799121255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540cb7d0 msgr2=0x7fad540b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.792+0000 7fad5c000640 1 --2- 172.21.15.97:0/1799121255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540cb7d0 0x7fad540b2e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad440097a0 tx=0x7fad4402b790 comp rx=0 tx=0).stop 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.793+0000 7fad5c000640 1 -- 172.21.15.97:0/1799121255 shutdown_connections 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.793+0000 7fad5c000640 1 --2- 172.21.15.97:0/1799121255 >> v2:172.21.15.97:3300/0 conn(0x7fad5406c350 0x7fad540ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.793+0000 7fad5c000640 1 --2- 172.21.15.97:0/1799121255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540cb7d0 0x7fad540b2e10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.793+0000 7fad5c000640 1 -- 172.21.15.97:0/1799121255 >> 172.21.15.97:0/1799121255 conn(0x7fad540b0a10 msgr2=0x7fad540b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.793+0000 7fad5c000640 1 -- 172.21.15.97:0/1799121255 shutdown_connections 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.793+0000 7fad5c000640 1 -- 172.21.15.97:0/1799121255 wait complete. 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.794+0000 7fad5c000640 1 Processor -- start 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.794+0000 7fad5c000640 1 -- start start 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.795+0000 7fad5c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 0x7fad540acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.795+0000 7fad5c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5406dc40 0x7fad540ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.795+0000 7fad5c000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad540cb7d0 0x7fad5415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.795+0000 7fad5c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5409a370 con 0x7fad5406c350 2024-01-31T05:00:30.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.795+0000 7fad5c000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5409a650 con 0x7fad540cb7d0 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.795+0000 7fad5c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5409a930 con 0x7fad5406dc40 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 0x7fad540acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad58c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5406dc40 0x7fad540ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 0x7fad540acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47896/0 (socket says 172.21.15.97:47896) 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 -- 172.21.15.97:0/2760357387 learned_addr learned my addr 172.21.15.97:0/2760357387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad5a000640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad540cb7d0 0x7fad5415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 -- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5406dc40 msgr2=0x7fad540ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5406dc40 0x7fad540ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 -- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad540cb7d0 msgr2=0x7fad5415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad540cb7d0 0x7fad5415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 -- 172.21.15.97:0/2760357387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c009a70 con 0x7fad5406c350 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad58c00640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5406dc40 0x7fad540ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:30.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.796+0000 7fad59600640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 0x7fad540acd40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad44035040 tx=0x7fad440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4403d890 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4403da10 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4403ddc0 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad44009400 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5409aef0 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fad44039040 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.797+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fad44032400 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.798+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad540677c0 con 0x7fad5406c350 2024-01-31T05:00:30.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.801+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fad4404d060 con 0x7fad5406c350 2024-01-31T05:00:30.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.866+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad540ae1d0 con 0x7fad5406c350 2024-01-31T05:00:30.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.867+0000 7fad51600640 1 -- 172.21.15.97:0/2760357387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fad44043070 con 0x7fad5406c350 2024-01-31T05:00:30.866 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:30.866 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:30.525121+0000","last_up_change":"0.000000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 msgr2=0x7fad540acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:30.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 0x7fad540acd40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad44035040 tx=0x7fad440046c0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 shutdown_connections 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad540cb7d0 0x7fad5415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5406dc40 0x7fad540ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 --2- 172.21.15.97:0/2760357387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad5406c350 0x7fad540acd40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 >> 172.21.15.97:0/2760357387 conn(0x7fad540b0a10 msgr2=0x7fad540b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 shutdown_connections 2024-01-31T05:00:30.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:30.868+0000 7fad5c000640 1 -- 172.21.15.97:0/2760357387 wait complete. 2024-01-31T05:00:30.882 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-31T05:00:30.883 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-31T05:00:31.043 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:31.044+0000 7fbd8563a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:00:31.248 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-01-31T05:00:31.248+0000 7f779b40e640 -1 Falling back to public interface 2024-01-31T05:00:31.260 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-31T05:00:31.260+0000 7f294dc68640 -1 Falling back to public interface 2024-01-31T05:00:31.268 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-31T05:00:31.268+0000 7f72fcb49640 -1 Falling back to public interface 2024-01-31T05:00:31.269 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-31T05:00:31.269+0000 7fa626699640 -1 Falling back to public interface 2024-01-31T05:00:31.277 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-01-31T05:00:31.278+0000 7f8688b02640 -1 Falling back to public interface 2024-01-31T05:00:31.278 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-01-31T05:00:31.279+0000 7f93f701f640 -1 Falling back to public interface 2024-01-31T05:00:31.283 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-01-31T05:00:31.284+0000 7f5afa8c6640 -1 Falling back to public interface 2024-01-31T05:00:31.292 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-31T05:00:31.292+0000 7fa616224640 -1 Falling back to public interface 2024-01-31T05:00:31.359 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:31.359+0000 7fbd8563a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:00:31.417 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:31.418+0000 7fbd8563a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:00:31.530 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:31.530+0000 7fbd8563a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:00:31.553 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:31.553+0000 7f239dcda040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-31T05:00:31.689 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:31.840 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.840+0000 7f8f47e00640 1 Processor -- start 2024-01-31T05:00:31.840 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.840+0000 7f8f47e00640 1 -- start start 2024-01-31T05:00:31.841 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.841+0000 7f8f47e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f4815e740 0x7f8f480cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:31.841 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.841+0000 7f8f47e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480d1d30 con 0x7f8f480b2230 2024-01-31T05:00:31.841 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.841+0000 7f8f47e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480ccd40 con 0x7f8f480cc960 2024-01-31T05:00:31.841 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.841+0000 7f8f47e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480ce7a0 con 0x7f8f4815e740 2024-01-31T05:00:31.841 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.842+0000 7f8f46a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f4815e740 0x7f8f480cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:31.841 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.842+0000 7f8f3fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8f480b2230 0x7f8f480b0680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46986/0 (socket says 172.21.15.97:46986) 2024-01-31T05:00:31.842 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.842+0000 7f8f3fe00640 1 -- 172.21.15.97:0/614880200 learned_addr learned my addr 172.21.15.97:0/614880200 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:31.842 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.842+0000 7f8f46a00640 1 -- 172.21.15.97:0/614880200 >> v1:172.21.15.188:6789/0 conn(0x7f8f480cc960 legacy=0x7f8f480b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:31.842 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.842+0000 7f8f46a00640 1 -- 172.21.15.97:0/614880200 >> v1:172.21.15.97:6789/0 conn(0x7f8f480b2230 legacy=0x7f8f480b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:31.843 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.842+0000 7f8f46a00640 1 -- 172.21.15.97:0/614880200 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4815eb20 con 0x7f8f4815e740 2024-01-31T05:00:31.843 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.843+0000 7f8f46a00640 1 --2- 172.21.15.97:0/614880200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4815e740 0x7f8f480cd680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f380097a0 tx=0x7f8f3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c6fdc0f560cc68 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.843+0000 7f8f46000640 1 -- 172.21.15.97:0/614880200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3802e070 con 0x7f8f4815e740 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f46000640 1 -- 172.21.15.97:0/614880200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f380315a0 con 0x7f8f4815e740 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f46000640 1 -- 172.21.15.97:0/614880200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38031930 con 0x7f8f4815e740 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 -- 172.21.15.97:0/614880200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4815e740 msgr2=0x7f8f480cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 --2- 172.21.15.97:0/614880200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4815e740 0x7f8f480cd680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f380097a0 tx=0x7f8f3802b840 comp rx=0 tx=0).stop 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 -- 172.21.15.97:0/614880200 shutdown_connections 2024-01-31T05:00:31.844 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 --2- 172.21.15.97:0/614880200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4815e740 0x7f8f480cd680 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.845 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 -- 172.21.15.97:0/614880200 >> 172.21.15.97:0/614880200 conn(0x7f8f480d0db0 msgr2=0x7f8f480d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:31.845 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 -- 172.21.15.97:0/614880200 shutdown_connections 2024-01-31T05:00:31.845 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.844+0000 7f8f47e00640 1 -- 172.21.15.97:0/614880200 wait complete. 2024-01-31T05:00:31.846 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.847+0000 7f8f47e00640 1 Processor -- start 2024-01-31T05:00:31.846 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.847+0000 7f8f47e00640 1 -- start start 2024-01-31T05:00:31.849 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f480b2230 0x7f8f48154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:31.849 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 0x7f8f4806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4815e740 0x7f8f480c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48154950 con 0x7f8f480cc960 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4806d1e0 con 0x7f8f480b2230 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4806bc00 con 0x7f8f4815e740 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 0x7f8f4806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 0x7f8f4806cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47982/0 (socket says 172.21.15.97:47982) 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f3fe00640 1 -- 172.21.15.97:0/2407620861 learned_addr learned my addr 172.21.15.97:0/2407620861 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f46a00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f480b2230 0x7f8f48154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f47400640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4815e740 0x7f8f480c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f3fe00640 1 -- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4815e740 msgr2=0x7f8f480c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:31.850 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.848+0000 7f8f3fe00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4815e740 0x7f8f480c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3fe00640 1 -- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f480b2230 msgr2=0x7f8f48154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3fe00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f480b2230 0x7f8f48154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3fe00640 1 -- 172.21.15.97:0/2407620861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38009400 con 0x7f8f480cc960 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3fe00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 0x7f8f4806cca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c400 tx=0x7f8f3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30005ce0 con 0x7f8f480cc960 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30005e40 con 0x7f8f480cc960 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4806bee0 con 0x7f8f480cc960 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f300072d0 con 0x7f8f480cc960 2024-01-31T05:00:31.851 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.849+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4806c4a0 con 0x7f8f480cc960 2024-01-31T05:00:31.852 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.851+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f48074710 con 0x7f8f480cc960 2024-01-31T05:00:31.852 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.851+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 2) v1 ==== 44628+0+0 (secure 0 0 0) 0x7f8f300074f0 con 0x7f8f480cc960 2024-01-31T05:00:31.852 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.851+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8f30048e30 con 0x7f8f480cc960 2024-01-31T05:00:31.853 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.853+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f8f3004d070 con 0x7f8f480cc960 2024-01-31T05:00:31.865 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:31.865+0000 7f239dcda040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-31T05:00:31.914 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.915+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f4804eaf0 con 0x7f8f480cc960 2024-01-31T05:00:31.915 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.916+0000 7f8f3f400640 1 -- 172.21.15.97:0/2407620861 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f8f30017070 con 0x7f8f480cc960 2024-01-31T05:00:31.915 INFO:teuthology.misc.health.smithi097.stdout: 2024-01-31T05:00:31.916 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:30.525121+0000","last_up_change":"0.000000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 msgr2=0x7f8f4806cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 0x7f8f4806cca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c400 tx=0x7f8f3000c8c0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 shutdown_connections 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f4815e740 0x7f8f480c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f480cc960 0x7f8f4806cca0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 --2- 172.21.15.97:0/2407620861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f480b2230 0x7f8f48154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 >> 172.21.15.97:0/2407620861 conn(0x7f8f480d0db0 msgr2=0x7f8f48072ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 shutdown_connections 2024-01-31T05:00:31.918 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:31.917+0000 7f8f47e00640 1 -- 172.21.15.97:0/2407620861 wait complete. 2024-01-31T05:00:31.924 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:31.924+0000 7f239dcda040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-31T05:00:31.928 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-31T05:00:32.036 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-01-31T05:00:32.036+0000 7f239dcda040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-31T05:00:32.570 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:32.570+0000 7fbd25c00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-31T05:00:32.572 INFO:tasks.ceph.mgr.y.smithi097.stderr:2024-01-31T05:00:32.573+0000 7fbd25c00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-31T05:00:34.311 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-01-31T05:00:34.312+0000 7f779b40e640 -1 osd.0 0 log_to_monitors true 2024-01-31T05:00:34.774 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-31T05:00:34.774+0000 7f294dc68640 -1 osd.6 0 log_to_monitors true 2024-01-31T05:00:34.823 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-31T05:00:34.823+0000 7fa626699640 -1 osd.5 0 log_to_monitors true 2024-01-31T05:00:34.848 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-31T05:00:34.848+0000 7fa616224640 -1 osd.4 0 log_to_monitors true 2024-01-31T05:00:34.849 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-31T05:00:34.849+0000 7f72fcb49640 -1 osd.7 0 log_to_monitors true 2024-01-31T05:00:34.863 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-01-31T05:00:34.863+0000 7f93f701f640 -1 osd.1 0 log_to_monitors true 2024-01-31T05:00:34.900 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-01-31T05:00:34.900+0000 7f5afa8c6640 -1 osd.3 0 log_to_monitors true 2024-01-31T05:00:34.901 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-01-31T05:00:34.901+0000 7f8688b02640 -1 osd.2 0 log_to_monitors true 2024-01-31T05:00:36.860 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-01-31T05:00:36.859+0000 7f7795800640 -1 osd.0 0 waiting for initial osdmap 2024-01-31T05:00:36.872 INFO:tasks.ceph.osd.0.smithi097.stderr:2024-01-31T05:00:36.871+0000 7f778b800640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:37.569 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-31T05:00:37.570+0000 7fa621200640 -1 osd.5 0 waiting for initial osdmap 2024-01-31T05:00:37.589 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-01-31T05:00:37.589+0000 7fa617200640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:37.806 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-31T05:00:37.619+0000 7f2948800640 -1 osd.6 0 waiting for initial osdmap 2024-01-31T05:00:37.806 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-01-31T05:00:37.662+0000 7f293e800640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:38.332 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-31T05:00:38.332+0000 7f72f7600640 -1 osd.7 0 waiting for initial osdmap 2024-01-31T05:00:38.342 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-01-31T05:00:38.342+0000 7f72ed600640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:38.357 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-31T05:00:38.358+0000 7fa610600640 -1 osd.4 0 waiting for initial osdmap 2024-01-31T05:00:38.370 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-01-31T05:00:38.370+0000 7fa606600640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:38.730 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-01-31T05:00:38.731+0000 7f5af5400640 -1 osd.3 0 waiting for initial osdmap 2024-01-31T05:00:38.735 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:38.740 INFO:tasks.ceph.osd.3.smithi097.stderr:2024-01-31T05:00:38.739+0000 7f5aeb400640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:38.748 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-01-31T05:00:38.748+0000 7f93f1600640 -1 osd.1 0 waiting for initial osdmap 2024-01-31T05:00:38.755 INFO:tasks.ceph.osd.1.smithi097.stderr:2024-01-31T05:00:38.756+0000 7f93e7600640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:38.874 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-01-31T05:00:38.875+0000 7f8683600640 -1 osd.2 0 waiting for initial osdmap 2024-01-31T05:00:38.880 INFO:tasks.ceph.osd.2.smithi097.stderr:2024-01-31T05:00:38.880+0000 7f8679600640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-31T05:00:38.881 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.882+0000 7f1013800640 1 Processor -- start 2024-01-31T05:00:38.881 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.882+0000 7f1013800640 1 -- start start 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.883+0000 7f1013800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f100c15d510 0x7f100c0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.883+0000 7f1013800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c0b6330 0x7f100c0b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.883+0000 7f1013800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c0d1420 con 0x7f100c0cdf40 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.883+0000 7f1013800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c15d8f0 con 0x7f100c15d510 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.883+0000 7f1013800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0d1700 con 0x7f100c0b6330 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f100c15d510 0x7f100c0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f100c15d510 0x7f100c0b8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:56048/0 (socket says 172.21.15.97:56048) 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 -- 172.21.15.97:0/2732688955 learned_addr learned my addr 172.21.15.97:0/2732688955 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:38.883 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f100be00640 1 --2- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c0b6330 0x7f100c0b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:38.884 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 -- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c0b6330 msgr2=0x7f100c0b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:38.884 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 --2- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c0b6330 0x7f100c0b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:38.884 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 -- 172.21.15.97:0/2732688955 >> v1:172.21.15.188:6789/0 conn(0x7f100c0cdf40 legacy=0x7f100c0d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:38.884 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.884+0000 7f1010e00640 1 -- 172.21.15.97:0/2732688955 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0d19e0 con 0x7f100c15d510 2024-01-31T05:00:38.885 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.886+0000 7f1010e00640 1 --2- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c15d510 0x7f100c0b8b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009870 tx=0x7f0ffc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382b91ee0845b5b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.888+0000 7f100b400640 1 -- 172.21.15.97:0/2732688955 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc030840 con 0x7f100c15d510 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.888+0000 7f100b400640 1 -- 172.21.15.97:0/2732688955 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc0309a0 con 0x7f100c15d510 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.888+0000 7f100b400640 1 -- 172.21.15.97:0/2732688955 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc030d30 con 0x7f100c15d510 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.888+0000 7f1013800640 1 -- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c15d510 msgr2=0x7f100c0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.888+0000 7f1013800640 1 --2- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c15d510 0x7f100c0b8b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009870 tx=0x7f0ffc02b980 comp rx=0 tx=0).stop 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.889+0000 7f1013800640 1 -- 172.21.15.97:0/2732688955 shutdown_connections 2024-01-31T05:00:38.888 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.889+0000 7f1013800640 1 --2- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c0b6330 0x7f100c0b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:38.889 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.889+0000 7f1013800640 1 --2- 172.21.15.97:0/2732688955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c15d510 0x7f100c0b8b20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:38.889 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.889+0000 7f1013800640 1 -- 172.21.15.97:0/2732688955 >> 172.21.15.97:0/2732688955 conn(0x7f100c0cfc80 msgr2=0x7f100c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:38.889 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.889+0000 7f1013800640 1 -- 172.21.15.97:0/2732688955 shutdown_connections 2024-01-31T05:00:38.889 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.889+0000 7f1013800640 1 -- 172.21.15.97:0/2732688955 wait complete. 2024-01-31T05:00:38.890 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.890+0000 7f1013800640 1 Processor -- start 2024-01-31T05:00:38.890 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.890+0000 7f1013800640 1 -- start start 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.891+0000 7f1013800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 0x7f100c14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.891+0000 7f1013800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c0cdf40 0x7f100c159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.891+0000 7f1013800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c15d510 0x7f100c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.891+0000 7f1013800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0cb700 con 0x7f100c0b6330 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.891+0000 7f1013800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0cb9e0 con 0x7f100c0cdf40 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.891+0000 7f1013800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0cbcc0 con 0x7f100c15d510 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 0x7f100c14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 0x7f100c14cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46026/0 (socket says 172.21.15.97:46026) 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 -- 172.21.15.97:0/4273331652 learned_addr learned my addr 172.21.15.97:0/4273331652 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:38.892 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f100be00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c0cdf40 0x7f100c159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 -- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c15d510 msgr2=0x7f100c15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c15d510 0x7f100c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 -- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c0cdf40 msgr2=0x7f100c159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c0cdf40 0x7f100c159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 -- 172.21.15.97:0/4273331652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc0094d0 con 0x7f100c0b6330 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1010e00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 0x7f100c14cf10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc000c00 tx=0x7f0ffc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc039a90 con 0x7f100c0b6330 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc039bf0 con 0x7f100c0b6330 2024-01-31T05:00:38.893 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.892+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c0cbfa0 con 0x7f100c0b6330 2024-01-31T05:00:38.894 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.893+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc043410 con 0x7f100c0b6330 2024-01-31T05:00:38.894 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.893+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c14d450 con 0x7f100c0b6330 2024-01-31T05:00:38.894 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.894+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c1572b0 con 0x7f100c0b6330 2024-01-31T05:00:38.894 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.894+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7f0ffc04c3b0 con 0x7f100c0b6330 2024-01-31T05:00:38.895 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.896+0000 7f1008c00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0fe406ae40 0x7f0fe406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:38.895 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.896+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3071+0+0 (secure 0 0 0) 0x7f0ffc0aaf60 con 0x7f100c0b6330 2024-01-31T05:00:38.895 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.896+0000 7f100be00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0fe406ae40 0x7f0fe406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:38.896 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.896+0000 7f100be00640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0fe406ae40 0x7f0fe406d300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f100c0b46f0 tx=0x7f0ff4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:38.904 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:38.901+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f100c1572b0 con 0x7f100c0b6330 2024-01-31T05:00:39.091 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.091+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f100c1572b0 con 0x7f100c0b6330 2024-01-31T05:00:39.092 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.092+0000 7f1008c00640 1 -- 172.21.15.97:0/4273331652 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10978 (secure 0 0 0) 0x7f100c1572b0 con 0x7f100c0b6330 2024-01-31T05:00:39.092 INFO:teuthology.misc.health.smithi097.stdout: 2024-01-31T05:00:39.092 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":15,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:38.948149+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0fe406ae40 msgr2=0x7f0fe406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0fe406ae40 0x7f0fe406d300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f100c0b46f0 tx=0x7f0ff4008040 comp rx=0 tx=0).stop 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 msgr2=0x7f100c14cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 0x7f100c14cf10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc000c00 tx=0x7f0ffc004730 comp rx=0 tx=0).stop 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 shutdown_connections 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f100c15d510 0x7f100c15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f100c0cdf40 0x7f100c159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0fe406ae40 0x7f0fe406d300 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 --2- 172.21.15.97:0/4273331652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f100c0b6330 0x7f100c14cf10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.095 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 >> 172.21.15.97:0/4273331652 conn(0x7f100c0cfc80 msgr2=0x7f100c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:39.096 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 shutdown_connections 2024-01-31T05:00:39.096 INFO:teuthology.misc.health.smithi097.stderr:2024-01-31T05:00:39.095+0000 7f1013800640 1 -- 172.21.15.97:0/4273331652 wait complete. 2024-01-31T05:00:39.107 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-31T05:00:39.107 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-31T05:00:39.108 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-31T05:00:39.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-31T05:00:39.155 INFO:tasks.ceph.mds.a:Started 2024-01-31T05:00:39.155 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-31T05:00:39.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-31T05:00:39.162 INFO:tasks.ceph.mds.c:Started 2024-01-31T05:00:39.162 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-31T05:00:39.162 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-31T05:00:39.171 INFO:tasks.ceph.mds.b:Started 2024-01-31T05:00:39.171 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-31T05:00:39.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-31T05:00:39.178 INFO:tasks.ceph.mds.d:Started 2024-01-31T05:00:39.179 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-31T05:00:39.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:39.300 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-01-31T05:00:39.301 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-01-31T05:00:39.330 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-31T05:00:39.339 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-31T05:00:39.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.352+0000 7f7b18400640 1 Processor -- start 2024-01-31T05:00:39.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.353+0000 7f7b18400640 1 -- start start 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b18400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7b1015d590 0x7f7b1015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b18400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b100cfae0 0x7f7b100cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b18400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b10161080 0x7f7b10161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b18400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b100d0460 con 0x7f7b100cfae0 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b18400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b100cded0 con 0x7f7b1015d590 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b18400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1015d970 con 0x7f7b10161080 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b100cfae0 0x7f7b100cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b100cfae0 0x7f7b100cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46076/0 (socket says 172.21.15.97:46076) 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 -- 172.21.15.97:0/2614706905 learned_addr learned my addr 172.21.15.97:0/2614706905 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15a00640 1 --2- 172.21.15.97:0/2614706905 >> v2:172.21.15.188:3300/0 conn(0x7f7b1015d590 0x7f7b1015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b16400640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b10161080 0x7f7b10161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 -- 172.21.15.97:0/2614706905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b10161080 msgr2=0x7f7b10161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b10161080 0x7f7b10161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 -- 172.21.15.97:0/2614706905 >> v2:172.21.15.188:3300/0 conn(0x7f7b1015d590 msgr2=0x7f7b1015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 --2- 172.21.15.97:0/2614706905 >> v2:172.21.15.188:3300/0 conn(0x7f7b1015d590 0x7f7b1015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15000640 1 -- 172.21.15.97:0/2614706905 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100cf210 con 0x7f7b100cfae0 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b16400640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b10161080 0x7f7b10161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.354+0000 7f7b15a00640 1 --2- 172.21.15.97:0/2614706905 >> v2:172.21.15.188:3300/0 conn(0x7f7b1015d590 0x7f7b1015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.355+0000 7f7b15000640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b100cfae0 0x7f7b100cff20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b1008b830 tx=0x7f7af802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9433df5ae24c84de server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b0fe00640 1 -- 172.21.15.97:0/2614706905 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8030850 con 0x7f7b100cfae0 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b0fe00640 1 -- 172.21.15.97:0/2614706905 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af80309b0 con 0x7f7b100cfae0 2024-01-31T05:00:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b0fe00640 1 -- 172.21.15.97:0/2614706905 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8030d40 con 0x7f7b100cfae0 2024-01-31T05:00:39.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 -- 172.21.15.97:0/2614706905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b100cfae0 msgr2=0x7f7b100cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b100cfae0 0x7f7b100cff20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7b1008b830 tx=0x7f7af802b990 comp rx=0 tx=0).stop 2024-01-31T05:00:39.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 -- 172.21.15.97:0/2614706905 shutdown_connections 2024-01-31T05:00:39.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b10161080 0x7f7b10161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 --2- 172.21.15.97:0/2614706905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b100cfae0 0x7f7b100cff20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 --2- 172.21.15.97:0/2614706905 >> v2:172.21.15.188:3300/0 conn(0x7f7b1015d590 0x7f7b1015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 -- 172.21.15.97:0/2614706905 >> 172.21.15.97:0/2614706905 conn(0x7f7b1009c560 msgr2=0x7f7b100d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 -- 172.21.15.97:0/2614706905 shutdown_connections 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.356+0000 7f7b18400640 1 -- 172.21.15.97:0/2614706905 wait complete. 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.358+0000 7f7b18400640 1 Processor -- start 2024-01-31T05:00:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.358+0000 7f7b18400640 1 -- start start 2024-01-31T05:00:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b18400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b100cfae0 0x7f7b1015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b18400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1015d590 0x7f7b100cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b18400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 0x7f7b100cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b18400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1015b8b0 con 0x7f7b10161080 2024-01-31T05:00:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b18400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100c7090 con 0x7f7b100cfae0 2024-01-31T05:00:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b18400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100c7370 con 0x7f7b1015d590 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b16400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 0x7f7b100cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b16400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 0x7f7b100cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46084/0 (socket says 172.21.15.97:46084) 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b16400640 1 -- 172.21.15.97:0/806770637 learned_addr learned my addr 172.21.15.97:0/806770637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b15a00640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b100cfae0 0x7f7b1015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.359+0000 7f7b15000640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1015d590 0x7f7b100cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b16400640 1 -- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1015d590 msgr2=0x7f7b100cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b16400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1015d590 0x7f7b100cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b16400640 1 -- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b100cfae0 msgr2=0x7f7b1015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b16400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b100cfae0 0x7f7b1015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b16400640 1 -- 172.21.15.97:0/806770637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80094d0 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b16400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 0x7f7b100cc3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009fa0 tx=0x7f7b0800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08003b90 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800f040 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08004030 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b100c7650 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.360+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b100c7c10 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.361+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b100c7e00 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.362+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7f7b08004190 con 0x7f7b10161080 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.363+0000 7f7b0d600640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7ae806ae40 0x7f7ae806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.363+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7b08089480 con 0x7f7b10161080 2024-01-31T05:00:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.363+0000 7f7b15a00640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7ae806ae40 0x7f7ae806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.364+0000 7f7b15a00640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7ae806ae40 0x7f7ae806d300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009950 tx=0x7f7b00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.367+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7b08054880 con 0x7f7b10161080 2024-01-31T05:00:39.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.546+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b1006b540 con 0x7f7b10161080 2024-01-31T05:00:39.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.547+0000 7f7b0d600640 1 -- 172.21.15.97:0/806770637 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10978 (secure 0 0 0) 0x7f7b0801a020 con 0x7f7b10161080 2024-01-31T05:00:39.546 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:39.547 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":15,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:38.948149+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:39.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.549+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7ae806ae40 msgr2=0x7f7ae806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.549+0000 7f7b18400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7ae806ae40 0x7f7ae806d300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009950 tx=0x7f7b00008040 comp rx=0 tx=0).stop 2024-01-31T05:00:39.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.549+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 msgr2=0x7f7b100cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.549+0000 7f7b18400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 0x7f7b100cc3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009fa0 tx=0x7f7b0800d560 comp rx=0 tx=0).stop 2024-01-31T05:00:39.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 shutdown_connections 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7ae806ae40 0x7f7ae806d300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b10161080 0x7f7b100cc3c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b1015d590 0x7f7b100cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 --2- 172.21.15.97:0/806770637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b100cfae0 0x7f7b1015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 >> 172.21.15.97:0/806770637 conn(0x7f7b1009c560 msgr2=0x7f7b1015f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 shutdown_connections 2024-01-31T05:00:39.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.550+0000 7f7b18400640 1 -- 172.21.15.97:0/806770637 wait complete. 2024-01-31T05:00:39.571 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-01-31T05:00:39.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:00:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.747+0000 7f87df200640 1 Processor -- start 2024-01-31T05:00:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.748+0000 7f87df200640 1 -- start start 2024-01-31T05:00:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.749+0000 7f87df200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87d815ec70 0x7f87d8071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.749+0000 7f87df200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.749+0000 7f87df200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d8070400 con 0x7f87d80ae3b0 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.749+0000 7f87df200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d80ad8c0 con 0x7f87d815ec70 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.749+0000 7f87df200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80ca580 con 0x7f87d80723a0 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87d815ec70 0x7f87d8071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87d815ec70 0x7f87d8071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46098/0 (socket says 172.21.15.97:46098) 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dd400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f87d80ae3b0 0x7f87d806cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60224/0 (socket says 172.21.15.97:60224) 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 -- 172.21.15.97:0/3475522889 learned_addr learned my addr 172.21.15.97:0/3475522889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87de800640 1 --2- 172.21.15.97:0/3475522889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 -- 172.21.15.97:0/3475522889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 msgr2=0x7f87d815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 --2- 172.21.15.97:0/3475522889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 -- 172.21.15.97:0/3475522889 >> v1:172.21.15.97:6789/0 conn(0x7f87d80ae3b0 legacy=0x7f87d806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dde00640 1 -- 172.21.15.97:0/3475522889 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8070650 con 0x7f87d815ec70 2024-01-31T05:00:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.750+0000 7f87dca00640 1 -- 172.21.15.97:0/3475522889 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899273118 0 0) 0x7f87d8070400 con 0x7f87d80ae3b0 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.751+0000 7f87dde00640 1 --2- 172.21.15.97:0/3475522889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d8071e60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009870 tx=0x7f87c802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6b7d62383d0b5f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.751+0000 7f87dca00640 1 -- 172.21.15.97:0/3475522889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80317c0 con 0x7f87d815ec70 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.751+0000 7f87dca00640 1 -- 172.21.15.97:0/3475522889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8031920 con 0x7f87d815ec70 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.751+0000 7f87dca00640 1 -- 172.21.15.97:0/3475522889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8031cb0 con 0x7f87d815ec70 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 -- 172.21.15.97:0/3475522889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 msgr2=0x7f87d8071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 --2- 172.21.15.97:0/3475522889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d8071e60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009870 tx=0x7f87c802b860 comp rx=0 tx=0).stop 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 -- 172.21.15.97:0/3475522889 shutdown_connections 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 --2- 172.21.15.97:0/3475522889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 --2- 172.21.15.97:0/3475522889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d8071e60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 -- 172.21.15.97:0/3475522889 >> 172.21.15.97:0/3475522889 conn(0x7f87d80cec40 msgr2=0x7f87d80cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 -- 172.21.15.97:0/3475522889 shutdown_connections 2024-01-31T05:00:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.752+0000 7f87df200640 1 -- 172.21.15.97:0/3475522889 wait complete. 2024-01-31T05:00:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.754+0000 7f87df200640 1 Processor -- start 2024-01-31T05:00:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.754+0000 7f87df200640 1 -- start start 2024-01-31T05:00:39.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87df200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d8158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87df200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d80ae3b0 0x7f87d815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87df200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87df200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8158f70 con 0x7f87d815ec70 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87df200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8099af0 con 0x7f87d80ae3b0 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87df200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8099d90 con 0x7f87d80723a0 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46102/0 (socket says 172.21.15.97:46102) 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 -- 172.21.15.97:0/2158173548 learned_addr learned my addr 172.21.15.97:0/2158173548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87dde00640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d8158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 -- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 msgr2=0x7f87d8158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d8158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 -- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d80ae3b0 msgr2=0x7f87d815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87dd400640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d80ae3b0 0x7f87d815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d80ae3b0 0x7f87d815d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.755+0000 7f87de800640 1 -- 172.21.15.97:0/2158173548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c80094d0 con 0x7f87d815ec70 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.756+0000 7f87de800640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d815d930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007e60 tx=0x7f87d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.756+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0014420 con 0x7f87d815ec70 2024-01-31T05:00:39.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.756+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d00145a0 con 0x7f87d815ec70 2024-01-31T05:00:39.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.756+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0014950 con 0x7f87d815ec70 2024-01-31T05:00:39.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.756+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d809a070 con 0x7f87d815ec70 2024-01-31T05:00:39.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.756+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d814bd00 con 0x7f87d815ec70 2024-01-31T05:00:39.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.757+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d809a590 con 0x7f87d815ec70 2024-01-31T05:00:39.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.758+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7f87d0014ab0 con 0x7f87d815ec70 2024-01-31T05:00:39.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.759+0000 7f87d6000640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f87b006ae40 0x7f87b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:39.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.759+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f87d0089980 con 0x7f87d815ec70 2024-01-31T05:00:39.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.759+0000 7f87dde00640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f87b006ae40 0x7f87b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:39.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.760+0000 7f87dde00640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f87b006ae40 0x7f87b006d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87c8036000 tx=0x7f87c803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:39.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.765+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f87d809a590 con 0x7f87d815ec70 2024-01-31T05:00:39.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.982+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87d809a590 con 0x7f87d815ec70 2024-01-31T05:00:39.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.983+0000 7f87d6000640 1 -- 172.21.15.97:0/2158173548 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f87d809a590 con 0x7f87d815ec70 2024-01-31T05:00:39.984 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:39.984 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-31T05:00:39.984 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-01-31T05:00:39.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f87b006ae40 msgr2=0x7f87b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f87b006ae40 0x7f87b006d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87c8036000 tx=0x7f87c803d040 comp rx=0 tx=0).stop 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 msgr2=0x7f87d815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d815d930 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f87d0007e60 tx=0x7f87d000e4b0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 shutdown_connections 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f87b006ae40 0x7f87b006d300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87d815ec70 0x7f87d815d930 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d80ae3b0 0x7f87d815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 --2- 172.21.15.97:0/2158173548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d80723a0 0x7f87d8158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 >> 172.21.15.97:0/2158173548 conn(0x7f87d80cec40 msgr2=0x7f87d80c7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 shutdown_connections 2024-01-31T05:00:39.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:39.986+0000 7f87df200640 1 -- 172.21.15.97:0/2158173548 wait complete. 2024-01-31T05:00:41.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:00:41.688 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:00:42.003 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:00:42.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.198+0000 7fbce9800640 1 Processor -- start 2024-01-31T05:00:42.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.198+0000 7fbce9800640 1 -- start start 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.199+0000 7fbce9800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbce415b4f0 0x7fbce40d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.199+0000 7fbce9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40ccb60 0x7fbce4164e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.199+0000 7fbce9800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce40cbda0 con 0x7fbce40cf400 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.199+0000 7fbce9800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce40cc080 con 0x7fbce415b4f0 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.199+0000 7fbce9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce40cc360 con 0x7fbce40ccb60 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbce40cf400 0x7fbce40cf7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60238/0 (socket says 172.21.15.97:60238) 2024-01-31T05:00:42.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3e00640 1 -- 172.21.15.97:0/1556190821 learned_addr learned my addr 172.21.15.97:0/1556190821 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:42.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3400640 1 --2- 172.21.15.97:0/1556190821 >> v2:172.21.15.97:3300/0 conn(0x7fbce415b4f0 0x7fbce40d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce8e00640 1 --2- 172.21.15.97:0/1556190821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40ccb60 0x7fbce4164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce2a00640 1 -- 172.21.15.97:0/1556190821 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165169668 0 0) 0x7fbce40cbda0 con 0x7fbce40cf400 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce2a00640 1 -- 172.21.15.97:0/1556190821 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc4003520 con 0x7fbce40cf400 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3400640 1 -- 172.21.15.97:0/1556190821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40ccb60 msgr2=0x7fbce4164e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3400640 1 --2- 172.21.15.97:0/1556190821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40ccb60 0x7fbce4164e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3400640 1 -- 172.21.15.97:0/1556190821 >> v1:172.21.15.97:6789/0 conn(0x7fbce40cf400 legacy=0x7fbce40cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.200+0000 7fbce3400640 1 -- 172.21.15.97:0/1556190821 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40cc640 con 0x7fbce415b4f0 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.201+0000 7fbce3400640 1 --2- 172.21.15.97:0/1556190821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce415b4f0 0x7fbce40d1940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0097a0 tx=0x7fbccc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3e6f559d079cac server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:42.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce2a00640 1 -- 172.21.15.97:0/1556190821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc02d040 con 0x7fbce415b4f0 2024-01-31T05:00:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce2a00640 1 -- 172.21.15.97:0/1556190821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc031810 con 0x7fbce415b4f0 2024-01-31T05:00:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce2a00640 1 -- 172.21.15.97:0/1556190821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc031ba0 con 0x7fbce415b4f0 2024-01-31T05:00:42.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce9800640 1 -- 172.21.15.97:0/1556190821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce415b4f0 msgr2=0x7fbce40d1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce9800640 1 --2- 172.21.15.97:0/1556190821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce415b4f0 0x7fbce40d1940 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0097a0 tx=0x7fbccc02b8b0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce9800640 1 -- 172.21.15.97:0/1556190821 shutdown_connections 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce9800640 1 --2- 172.21.15.97:0/1556190821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40ccb60 0x7fbce4164e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce9800640 1 --2- 172.21.15.97:0/1556190821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce415b4f0 0x7fbce40d1940 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.202+0000 7fbce9800640 1 -- 172.21.15.97:0/1556190821 >> 172.21.15.97:0/1556190821 conn(0x7fbce40d0e20 msgr2=0x7fbce40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.203+0000 7fbce9800640 1 -- 172.21.15.97:0/1556190821 shutdown_connections 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.203+0000 7fbce9800640 1 -- 172.21.15.97:0/1556190821 wait complete. 2024-01-31T05:00:42.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.204+0000 7fbce9800640 1 Processor -- start 2024-01-31T05:00:42.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.204+0000 7fbce9800640 1 -- start start 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.205+0000 7fbce9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 0x7fbce41503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.205+0000 7fbce9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40cf400 0x7fbce41f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.205+0000 7fbce9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbce41f4220 0x7fbce40b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.205+0000 7fbce9800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce40c7ef0 con 0x7fbce40ccb60 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.205+0000 7fbce9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce40c8d10 con 0x7fbce41f4220 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.205+0000 7fbce9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce40c8100 con 0x7fbce40cf400 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 0x7fbce41503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 0x7fbce41503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46136/0 (socket says 172.21.15.97:46136) 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 -- 172.21.15.97:0/879224965 learned_addr learned my addr 172.21.15.97:0/879224965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:42.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce8e00640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbce41f4220 0x7fbce40b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3400640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40cf400 0x7fbce41f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 -- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40cf400 msgr2=0x7fbce41f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40cf400 0x7fbce41f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 -- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbce41f4220 msgr2=0x7fbce40b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbce41f4220 0x7fbce40b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 -- 172.21.15.97:0/879224965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc009400 con 0x7fbce40ccb60 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce8e00640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbce41f4220 0x7fbce40b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:42.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.206+0000 7fbce3e00640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 0x7fbce41503b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c430 tx=0x7fbcd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.207+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4005ce0 con 0x7fbce40ccb60 2024-01-31T05:00:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.207+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd4005e40 con 0x7fbce40ccb60 2024-01-31T05:00:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.207+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce40c83e0 con 0x7fbce40ccb60 2024-01-31T05:00:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.207+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40072f0 con 0x7fbce40ccb60 2024-01-31T05:00:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.207+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce40c89a0 con 0x7fbce40ccb60 2024-01-31T05:00:42.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.208+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce4067dc0 con 0x7fbce40ccb60 2024-01-31T05:00:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.209+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7fbcd4007450 con 0x7fbce40ccb60 2024-01-31T05:00:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.210+0000 7fbcdbe00640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbcbc06ae40 0x7fbcbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.210+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbcd4088920 con 0x7fbce40ccb60 2024-01-31T05:00:42.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.210+0000 7fbce3400640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbcbc06ae40 0x7fbcbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.211+0000 7fbce3400640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbcbc06ae40 0x7fbcbc06d300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbccc036000 tx=0x7fbccc03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:42.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.216+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fbcd4053b80 con 0x7fbce40ccb60 2024-01-31T05:00:42.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.462+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbce40cfa30 con 0x7fbce40ccb60 2024-01-31T05:00:42.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.462+0000 7fbcdbe00640 1 -- 172.21.15.97:0/879224965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7fbcd4017020 con 0x7fbce40ccb60 2024-01-31T05:00:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:42.462 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-31T05:00:42.462 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-01-31T05:00:42.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbcbc06ae40 msgr2=0x7fbcbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbcbc06ae40 0x7fbcbc06d300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbccc036000 tx=0x7fbccc03c040 comp rx=0 tx=0).stop 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 msgr2=0x7fbce41503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 0x7fbce41503b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c430 tx=0x7fbcd400c8f0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 shutdown_connections 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbce41f4220 0x7fbce40b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbce40cf400 0x7fbce41f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbcbc06ae40 0x7fbcbc06d300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 --2- 172.21.15.97:0/879224965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbce40ccb60 0x7fbce41503b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 >> 172.21.15.97:0/879224965 conn(0x7fbce40d0e20 msgr2=0x7fbce415a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 shutdown_connections 2024-01-31T05:00:42.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.465+0000 7fbce9800640 1 -- 172.21.15.97:0/879224965 wait complete. 2024-01-31T05:00:42.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:42.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.667+0000 7ff3b3400640 1 Processor -- start 2024-01-31T05:00:42.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.667+0000 7ff3b3400640 1 -- start start 2024-01-31T05:00:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.668+0000 7ff3b3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 0x7ff3ac0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.668+0000 7ff3b3400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06d1a0 con 0x7ff3ac0ccd00 2024-01-31T05:00:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.668+0000 7ff3b3400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac06c600 con 0x7ff3ac15b400 2024-01-31T05:00:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.668+0000 7ff3b3400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cc960 con 0x7ff3ac0cb810 2024-01-31T05:00:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.669+0000 7ff3b1400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff3ac0ccd00 0x7ff3ac071cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60252/0 (socket says 172.21.15.97:60252) 2024-01-31T05:00:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.669+0000 7ff3b1400640 1 -- 172.21.15.97:0/28626282 learned_addr learned my addr 172.21.15.97:0/28626282 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.669+0000 7ff3b0a00640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 0x7ff3ac0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.669+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168309713 0 0) 0x7ff3ac06d1a0 con 0x7ff3ac0ccd00 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.669+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff38c003520 con 0x7ff3ac0ccd00 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098608011 0 0) 0x7ff38c003520 con 0x7ff3ac0ccd00 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 msgr2=0x7ff3ac0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 0x7ff3ac0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 >> v1:172.21.15.188:6789/0 conn(0x7ff3ac15b400 legacy=0x7ff3ac06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0cc130 con 0x7ff3ac0ccd00 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff3a40019f0 con 0x7ff3ac0ccd00 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 >> v1:172.21.15.97:6789/0 conn(0x7ff3ac0ccd00 legacy=0x7ff3ac071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38c0039a0 0x7ff38c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 0x7ff38c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff38c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0cc960 con 0x7ff38c004d00 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c00ba30 con 0x7ff38c0039a0 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c00bbf0 con 0x7ff3ac0cb810 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b1400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff38c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b0a00640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38c0039a0 0x7ff38c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3abe00640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 0x7ff38c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b1400640 1 -- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 msgr2=0x7ff38c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b1400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac0cb810 0x7ff38c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b1400640 1 -- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38c0039a0 msgr2=0x7ff38c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b1400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff38c0039a0 0x7ff38c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.670+0000 7ff3b1400640 1 -- 172.21.15.97:0/28626282 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c003520 con 0x7ff38c004d00 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3b1400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff38c005180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400a7f0 tx=0x7ff38c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8286bda92e029f48 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40072d0 con 0x7ff38c004d00 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a402f070 con 0x7ff38c004d00 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3ab400640 1 -- 172.21.15.97:0/28626282 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40076b0 con 0x7ff38c004d00 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3b3400640 1 -- 172.21.15.97:0/28626282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 msgr2=0x7ff38c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3b3400640 1 --2- 172.21.15.97:0/28626282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff38c005180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400a7f0 tx=0x7ff38c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.672+0000 7ff3b1400640 1 -- 172.21.15.97:0/28626282 reap_dead start 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.673+0000 7ff3b3400640 1 -- 172.21.15.97:0/28626282 shutdown_connections 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.673+0000 7ff3b3400640 1 -- 172.21.15.97:0/28626282 >> 172.21.15.97:0/28626282 conn(0x7ff3ac0b0a10 msgr2=0x7ff3ac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.673+0000 7ff3b3400640 1 -- 172.21.15.97:0/28626282 shutdown_connections 2024-01-31T05:00:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.673+0000 7ff3b3400640 1 -- 172.21.15.97:0/28626282 wait complete. 2024-01-31T05:00:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.675+0000 7ff3b3400640 1 Processor -- start 2024-01-31T05:00:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.675+0000 7ff3b3400640 1 -- start start 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff3ac14b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac09a010 0x7ff3ac09c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b3400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac15a300 0x7ff3ac15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac09cec0 con 0x7ff38c004d00 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b3400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac15acc0 con 0x7ff3ac15a300 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b3400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac15d030 con 0x7ff3ac09a010 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff3ac14b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff3ac14b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46156/0 (socket says 172.21.15.97:46156) 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 -- 172.21.15.97:0/3269855747 learned_addr learned my addr 172.21.15.97:0/3269855747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 -- 172.21.15.97:0/3269855747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac09a010 msgr2=0x7ff3ac09c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac09a010 0x7ff3ac09c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 -- 172.21.15.97:0/3269855747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac15a300 msgr2=0x7ff3ac15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac15a300 0x7ff3ac15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.676+0000 7ff3b0a00640 1 -- 172.21.15.97:0/3269855747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac09cbe0 con 0x7ff38c004d00 2024-01-31T05:00:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.677+0000 7ff3b0a00640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff3ac14b770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff39c011630 tx=0x7ff39c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.677+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c015870 con 0x7ff38c004d00 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.677+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c0159d0 con 0x7ff38c004d00 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.677+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c015d40 con 0x7ff38c004d00 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.677+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac15c9e0 con 0x7ff38c004d00 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.677+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac09c7a0 con 0x7ff38c004d00 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.678+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac0677c0 con 0x7ff38c004d00 2024-01-31T05:00:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.679+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7ff39c013430 con 0x7ff38c004d00 2024-01-31T05:00:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.680+0000 7ff3a8c00640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38406ae70 0x7ff38406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.680+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff39c089600 con 0x7ff38c004d00 2024-01-31T05:00:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.680+0000 7ff3abe00640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38406ae70 0x7ff38406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:42.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.681+0000 7ff3abe00640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38406ae70 0x7ff38406d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac1481c0 tx=0x7ff394009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:42.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.685+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff39c054860 con 0x7ff38c004d00 2024-01-31T05:00:42.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.876+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3ac073f30 con 0x7ff38c004d00 2024-01-31T05:00:42.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.879+0000 7ff3a8c00640 1 -- 172.21.15.97:0/3269855747 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12851 (secure 0 0 0) 0x7ff39c022090 con 0x7ff38c004d00 2024-01-31T05:00:42.879 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:42.879 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:42.336004+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.881+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38406ae70 msgr2=0x7ff38406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.881+0000 7ff3b3400640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38406ae70 0x7ff38406d330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac1481c0 tx=0x7ff394009040 comp rx=0 tx=0).stop 2024-01-31T05:00:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.881+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 msgr2=0x7ff3ac14b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:42.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.881+0000 7ff3b3400640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff3ac14b770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff39c011630 tx=0x7ff39c011af0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 shutdown_connections 2024-01-31T05:00:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3ac15a300 0x7ff3ac15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3ac09a010 0x7ff3ac09c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38406ae70 0x7ff38406d330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 --2- 172.21.15.97:0/3269855747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff38c004d00 0x7ff3ac14b770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 >> 172.21.15.97:0/3269855747 conn(0x7ff3ac0b0a10 msgr2=0x7ff3ac0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:42.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 shutdown_connections 2024-01-31T05:00:42.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:42.882+0000 7ff3b3400640 1 -- 172.21.15.97:0/3269855747 wait complete. 2024-01-31T05:00:42.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:00:42.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:00:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.106+0000 7fade1c00640 1 Processor -- start 2024-01-31T05:00:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.106+0000 7fade1c00640 1 -- start start 2024-01-31T05:00:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fade1c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faddc15a350 0x7faddc0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fade1c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15a730 0x7faddc06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fade1c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faddc06dfc0 0x7faddc0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fade1c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc0b2c90 con 0x7faddc06dfc0 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fade1c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc15abc0 con 0x7faddc15a350 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fade1c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc15aea0 con 0x7faddc15a730 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7faddb400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faddc06dfc0 0x7faddc0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7faddb400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faddc06dfc0 0x7faddc0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46172/0 (socket says 172.21.15.97:46172) 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7faddb400640 1 -- 172.21.15.97:0/1346540291 learned_addr learned my addr 172.21.15.97:0/1346540291 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7faddaa00640 1 --2- 172.21.15.97:0/1346540291 >> v2:172.21.15.188:3300/0 conn(0x7faddc15a350 0x7faddc0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.108+0000 7fadda000640 1 --2- 172.21.15.97:0/1346540291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15a730 0x7faddc06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.109+0000 7faddb400640 1 -- 172.21.15.97:0/1346540291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15a730 msgr2=0x7faddc06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.109+0000 7faddb400640 1 --2- 172.21.15.97:0/1346540291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15a730 0x7faddc06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.109+0000 7faddb400640 1 -- 172.21.15.97:0/1346540291 >> v2:172.21.15.188:3300/0 conn(0x7faddc15a350 msgr2=0x7faddc0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.109+0000 7faddb400640 1 --2- 172.21.15.97:0/1346540291 >> v2:172.21.15.188:3300/0 conn(0x7faddc15a350 0x7faddc0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.109+0000 7faddb400640 1 -- 172.21.15.97:0/1346540291 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc15b180 con 0x7faddc06dfc0 2024-01-31T05:00:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.109+0000 7faddaa00640 1 --2- 172.21.15.97:0/1346540291 >> v2:172.21.15.188:3300/0 conn(0x7faddc15a350 0x7faddc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.110+0000 7faddb400640 1 --2- 172.21.15.97:0/1346540291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc06dfc0 0x7faddc0cde10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faddc067370 tx=0x7fadd402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118a8535f4357741 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.110+0000 7fadd9600640 1 -- 172.21.15.97:0/1346540291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd402ec80 con 0x7faddc06dfc0 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.110+0000 7fadd9600640 1 -- 172.21.15.97:0/1346540291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd402ede0 con 0x7faddc06dfc0 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.110+0000 7fadd9600640 1 -- 172.21.15.97:0/1346540291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4036610 con 0x7faddc06dfc0 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.110+0000 7fade1c00640 1 -- 172.21.15.97:0/1346540291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc06dfc0 msgr2=0x7faddc0cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.110+0000 7fade1c00640 1 --2- 172.21.15.97:0/1346540291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc06dfc0 0x7faddc0cde10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faddc067370 tx=0x7fadd402e0e0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 -- 172.21.15.97:0/1346540291 shutdown_connections 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 --2- 172.21.15.97:0/1346540291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc06dfc0 0x7faddc0cde10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 --2- 172.21.15.97:0/1346540291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc15a730 0x7faddc06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 --2- 172.21.15.97:0/1346540291 >> v2:172.21.15.188:3300/0 conn(0x7faddc15a350 0x7faddc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 -- 172.21.15.97:0/1346540291 >> 172.21.15.97:0/1346540291 conn(0x7faddc0b0a10 msgr2=0x7faddc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 -- 172.21.15.97:0/1346540291 shutdown_connections 2024-01-31T05:00:43.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.111+0000 7fade1c00640 1 -- 172.21.15.97:0/1346540291 wait complete. 2024-01-31T05:00:43.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.112+0000 7fade1c00640 1 Processor -- start 2024-01-31T05:00:43.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.112+0000 7fade1c00640 1 -- start start 2024-01-31T05:00:43.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.113+0000 7fade1c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc06dfc0 0x7faddc149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.113+0000 7fade1c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15a350 0x7faddc1ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.113+0000 7fade1c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 0x7faddc0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.113+0000 7fade1c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc1ef1c0 con 0x7faddc1eec30 2024-01-31T05:00:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.113+0000 7fade1c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc1ef4a0 con 0x7faddc15a350 2024-01-31T05:00:43.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.113+0000 7fade1c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc099b80 con 0x7faddc06dfc0 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 0x7faddc0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 0x7faddc0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46178/0 (socket says 172.21.15.97:46178) 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 -- 172.21.15.97:0/3975366519 learned_addr learned my addr 172.21.15.97:0/3975366519 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddaa00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc06dfc0 0x7faddc149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7fadda000640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15a350 0x7faddc1ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 -- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc06dfc0 msgr2=0x7faddc149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc06dfc0 0x7faddc149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 -- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15a350 msgr2=0x7faddc1ee6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15a350 0x7faddc1ee6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 -- 172.21.15.97:0/3975366519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4009b30 con 0x7faddc1eec30 2024-01-31T05:00:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7fadda000640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15a350 0x7faddc1ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.114+0000 7faddb400640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 0x7faddc0abcf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadd4038040 tx=0x7fadd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.115+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd402ec80 con 0x7faddc1eec30 2024-01-31T05:00:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.115+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd4009510 con 0x7faddc1eec30 2024-01-31T05:00:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.115+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc099e60 con 0x7faddc1eec30 2024-01-31T05:00:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.115+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd403e630 con 0x7faddc1eec30 2024-01-31T05:00:43.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.115+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc15d020 con 0x7faddc1eec30 2024-01-31T05:00:43.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.116+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc0999c0 con 0x7faddc1eec30 2024-01-31T05:00:43.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.117+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7fadd403e790 con 0x7faddc1eec30 2024-01-31T05:00:43.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.119+0000 7fadd2a00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fadb406ae40 0x7fadb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.119+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fadd40ab8c0 con 0x7faddc1eec30 2024-01-31T05:00:43.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.120+0000 7faddaa00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fadb406ae40 0x7fadb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.121+0000 7faddaa00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fadb406ae40 0x7fadb406d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0098b0 tx=0x7fadcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.124+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fadd4076b20 con 0x7faddc1eec30 2024-01-31T05:00:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.317+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faddc1ef6c0 con 0x7faddc1eec30 2024-01-31T05:00:43.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.318+0000 7fadd2a00640 1 -- 172.21.15.97:0/3975366519 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadd407a140 con 0x7faddc1eec30 2024-01-31T05:00:43.317 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:00:43.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.320+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fadb406ae40 msgr2=0x7fadb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.320+0000 7fade1c00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fadb406ae40 0x7fadb406d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0098b0 tx=0x7fadcc008040 comp rx=0 tx=0).stop 2024-01-31T05:00:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.320+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 msgr2=0x7faddc0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.320+0000 7fade1c00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 0x7faddc0abcf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadd4038040 tx=0x7fadd4004a40 comp rx=0 tx=0).stop 2024-01-31T05:00:43.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 shutdown_connections 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fadb406ae40 0x7fadb406d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faddc1eec30 0x7faddc0abcf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc15a350 0x7faddc1ee6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 --2- 172.21.15.97:0/3975366519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faddc06dfc0 0x7faddc149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 >> 172.21.15.97:0/3975366519 conn(0x7faddc0b0a10 msgr2=0x7faddc158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 shutdown_connections 2024-01-31T05:00:43.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.321+0000 7fade1c00640 1 -- 172.21.15.97:0/3975366519 wait complete. 2024-01-31T05:00:43.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-31T05:00:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.530+0000 7f04bf200640 1 Processor -- start 2024-01-31T05:00:43.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.530+0000 7f04bf200640 1 -- start start 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.531+0000 7f04bf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9f40 0x7f04c00ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.531+0000 7f04bf200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04c00c9420 0x7f04c00abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.531+0000 7f04bf200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c015a5c0 con 0x7f04c00cdd60 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.531+0000 7f04bf200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c00cd8d0 con 0x7f04c00c9420 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.531+0000 7f04bf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00cd2c0 con 0x7f04c00c9f40 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04be800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04c00cdd60 0x7f04c00ce1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60268/0 (socket says 172.21.15.97:60268) 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04be800640 1 -- 172.21.15.97:0/2631083169 learned_addr learned my addr 172.21.15.97:0/2631083169 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04b7e00640 1 --2- 172.21.15.97:0/2631083169 >> v2:172.21.15.97:3300/0 conn(0x7f04c00c9420 0x7f04c00abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04bde00640 1 --2- 172.21.15.97:0/2631083169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9f40 0x7f04c00ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04bd400640 1 -- 172.21.15.97:0/2631083169 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957016660 0 0) 0x7f04c015a5c0 con 0x7f04c00cdd60 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04bd400640 1 -- 172.21.15.97:0/2631083169 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a0003520 con 0x7f04c00cdd60 2024-01-31T05:00:43.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04b7e00640 1 -- 172.21.15.97:0/2631083169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9f40 msgr2=0x7f04c00ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04b7e00640 1 --2- 172.21.15.97:0/2631083169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9f40 0x7f04c00ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04b7e00640 1 -- 172.21.15.97:0/2631083169 >> v1:172.21.15.97:6789/0 conn(0x7f04c00cdd60 legacy=0x7f04c00ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.532+0000 7f04b7e00640 1 -- 172.21.15.97:0/2631083169 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00ca860 con 0x7f04c00c9420 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.533+0000 7f04b7e00640 1 --2- 172.21.15.97:0/2631083169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00c9420 0x7f04c00abcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04a80097a0 tx=0x7f04a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2044dc58927672 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bd400640 1 -- 172.21.15.97:0/2631083169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8031830 con 0x7f04c00c9420 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bd400640 1 -- 172.21.15.97:0/2631083169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8031990 con 0x7f04c00c9420 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bd400640 1 -- 172.21.15.97:0/2631083169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8031d20 con 0x7f04c00c9420 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 -- 172.21.15.97:0/2631083169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00c9420 msgr2=0x7f04c00abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 --2- 172.21.15.97:0/2631083169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00c9420 0x7f04c00abcf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04a80097a0 tx=0x7f04a802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 -- 172.21.15.97:0/2631083169 shutdown_connections 2024-01-31T05:00:43.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 --2- 172.21.15.97:0/2631083169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00c9420 0x7f04c00abcf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 --2- 172.21.15.97:0/2631083169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9f40 0x7f04c00ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 -- 172.21.15.97:0/2631083169 >> 172.21.15.97:0/2631083169 conn(0x7f04c00cec40 msgr2=0x7f04c00cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 -- 172.21.15.97:0/2631083169 shutdown_connections 2024-01-31T05:00:43.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.534+0000 7f04bf200640 1 -- 172.21.15.97:0/2631083169 wait complete. 2024-01-31T05:00:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.536+0000 7f04bf200640 1 Processor -- start 2024-01-31T05:00:43.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 -- start start 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9420 0x7f04c014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c00c9f40 0x7f04c00cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 0x7f04c00cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014e8f0 con 0x7f04c00cdd60 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014dcb0 con 0x7f04c00c9f40 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.537+0000 7f04bf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c014ed10 con 0x7f04c00c9420 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 0x7f04c00cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 0x7f04c00cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46186/0 (socket says 172.21.15.97:46186) 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 -- 172.21.15.97:0/2363396230 learned_addr learned my addr 172.21.15.97:0/2363396230 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04bde00640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9420 0x7f04c014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04b7e00640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c00c9f40 0x7f04c00cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 -- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9420 msgr2=0x7f04c014d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9420 0x7f04c014d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 -- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c00c9f40 msgr2=0x7f04c00cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c00c9f40 0x7f04c00cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 -- 172.21.15.97:0/2363396230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8009400 con 0x7f04c00cdd60 2024-01-31T05:00:43.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.538+0000 7f04be800640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 0x7f04c00cb870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04b800ca70 tx=0x7f04b800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.539+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b800abd0 con 0x7f04c00cdd60 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.539+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b800ad30 con 0x7f04c00cdd60 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.539+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8012600 con 0x7f04c00cdd60 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.539+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c014d9c0 con 0x7f04c00cdd60 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.539+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c014e230 con 0x7f04c00cdd60 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.540+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c00730c0 con 0x7f04c00cdd60 2024-01-31T05:00:43.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.541+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7f04b8007070 con 0x7f04c00cdd60 2024-01-31T05:00:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.542+0000 7f04b6000640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f049806ae40 0x7f049806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.542+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04b8089110 con 0x7f04c00cdd60 2024-01-31T05:00:43.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.542+0000 7f04bde00640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f049806ae40 0x7f049806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.543+0000 7f04bde00640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f049806ae40 0x7f049806d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009b40 tx=0x7f04b0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.546+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f04c00730c0 con 0x7f04c00cdd60 2024-01-31T05:00:43.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.748+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f04c014ea30 con 0x7f04c00cdd60 2024-01-31T05:00:43.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.748+0000 7f04b6000640 1 -- 172.21.15.97:0/2363396230 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f04c014ea30 con 0x7f04c00cdd60 2024-01-31T05:00:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:43.748 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-01-31T05:00:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:00:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-01-31T05:00:43.748 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-01-31T05:00:43.749 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:00:43.749 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f049806ae40 msgr2=0x7f049806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f049806ae40 0x7f049806d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009b40 tx=0x7f04b0009190 comp rx=0 tx=0).stop 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 msgr2=0x7f04c00cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 0x7f04c00cb870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04b800ca70 tx=0x7f04b800cf30 comp rx=0 tx=0).stop 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 shutdown_connections 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f049806ae40 0x7f049806d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04c00cdd60 0x7f04c00cb870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c00c9f40 0x7f04c00cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 --2- 172.21.15.97:0/2363396230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04c00c9420 0x7f04c014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.751+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 >> 172.21.15.97:0/2363396230 conn(0x7f04c00cec40 msgr2=0x7f04c00ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.752+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 shutdown_connections 2024-01-31T05:00:43.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.752+0000 7f04bf200640 1 -- 172.21.15.97:0/2363396230 wait complete. 2024-01-31T05:00:43.768 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-31T05:00:43.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-31T05:00:43.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.974+0000 7fbad7e00640 1 Processor -- start 2024-01-31T05:00:43.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.974+0000 7fbad7e00640 1 -- start start 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.975+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad815d3f0 0x7fbad80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.975+0000 7fbad7e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80ce7e0 con 0x7fbad80d1e60 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.975+0000 7fbad7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad80d3cb0 con 0x7fbad80d3850 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.975+0000 7fbad7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80d09b0 con 0x7fbad815d3f0 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.976+0000 7fbad6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad815d3f0 0x7fbad80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.976+0000 7fbad6000640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fbad80d3850 0x7fbad80d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:37280/0 (socket says 172.21.15.97:37280) 2024-01-31T05:00:43.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.976+0000 7fbad7400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbad80d1e60 0x7fbad80b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60276/0 (socket says 172.21.15.97:60276) 2024-01-31T05:00:43.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.976+0000 7fbad6000640 1 -- 172.21.15.97:0/950232021 learned_addr learned my addr 172.21.15.97:0/950232021 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:43.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.977+0000 7fbad5600640 1 -- 172.21.15.97:0/950232021 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129106702 0 0) 0x7fbad80d3cb0 con 0x7fbad80d3850 2024-01-31T05:00:43.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.977+0000 7fbad5600640 1 -- 172.21.15.97:0/950232021 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab8003520 con 0x7fbad80d3850 2024-01-31T05:00:43.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.977+0000 7fbad6a00640 1 -- 172.21.15.97:0/950232021 >> v1:172.21.15.188:6789/0 conn(0x7fbad80d3850 legacy=0x7fbad80d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.977+0000 7fbad6a00640 1 -- 172.21.15.97:0/950232021 >> v1:172.21.15.97:6789/0 conn(0x7fbad80d1e60 legacy=0x7fbad80b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.977+0000 7fbad6a00640 1 -- 172.21.15.97:0/950232021 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80cdeb0 con 0x7fbad815d3f0 2024-01-31T05:00:43.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.978+0000 7fbad6a00640 1 --2- 172.21.15.97:0/950232021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad815d3f0 0x7fbad80cec70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbac80097d0 tx=0x7fbac802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89874032f6f5005b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.978+0000 7fbad5600640 1 -- 172.21.15.97:0/950232021 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8030800 con 0x7fbad815d3f0 2024-01-31T05:00:43.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.978+0000 7fbad5600640 1 -- 172.21.15.97:0/950232021 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac8030960 con 0x7fbad815d3f0 2024-01-31T05:00:43.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.978+0000 7fbad5600640 1 -- 172.21.15.97:0/950232021 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8030cd0 con 0x7fbad815d3f0 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 -- 172.21.15.97:0/950232021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad815d3f0 msgr2=0x7fbad80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 --2- 172.21.15.97:0/950232021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad815d3f0 0x7fbad80cec70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbac80097d0 tx=0x7fbac802b940 comp rx=0 tx=0).stop 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 -- 172.21.15.97:0/950232021 shutdown_connections 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 --2- 172.21.15.97:0/950232021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad815d3f0 0x7fbad80cec70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 -- 172.21.15.97:0/950232021 >> 172.21.15.97:0/950232021 conn(0x7fbad808cb00 msgr2=0x7fbad809c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 -- 172.21.15.97:0/950232021 shutdown_connections 2024-01-31T05:00:43.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.979+0000 7fbad7e00640 1 -- 172.21.15.97:0/950232021 wait complete. 2024-01-31T05:00:43.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.981+0000 7fbad7e00640 1 Processor -- start 2024-01-31T05:00:43.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.981+0000 7fbad7e00640 1 -- start start 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 0x7fbad8149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad80d3850 0x7fbad814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad815d3f0 0x7fbad814cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80ca9a0 con 0x7fbad80d1e60 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80c9770 con 0x7fbad815d3f0 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80ca2e0 con 0x7fbad80d3850 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 0x7fbad8149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 0x7fbad8149c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46210/0 (socket says 172.21.15.97:46210) 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad6a00640 1 -- 172.21.15.97:0/1261205735 learned_addr learned my addr 172.21.15.97:0/1261205735 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:43.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.982+0000 7fbad7400640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad815d3f0 0x7fbad814cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad6000640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad80d3850 0x7fbad814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad7400640 1 -- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad80d3850 msgr2=0x7fbad814cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad7400640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad80d3850 0x7fbad814cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad7400640 1 -- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 msgr2=0x7fbad8149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad7400640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 0x7fbad8149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad7400640 1 -- 172.21.15.97:0/1261205735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac8009400 con 0x7fbad815d3f0 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.983+0000 7fbad6a00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 0x7fbad8149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.984+0000 7fbad7400640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad815d3f0 0x7fbad814cff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbad80d4270 tx=0x7fbad000cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.984+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0004d60 con 0x7fbad815d3f0 2024-01-31T05:00:43.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.984+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad000a640 con 0x7fbad815d3f0 2024-01-31T05:00:43.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.984+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000aa10 con 0x7fbad815d3f0 2024-01-31T05:00:43.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.984+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad80ca5c0 con 0x7fbad815d3f0 2024-01-31T05:00:43.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.984+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad815aa20 con 0x7fbad815d3f0 2024-01-31T05:00:43.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.985+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad80caae0 con 0x7fbad815d3f0 2024-01-31T05:00:43.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.987+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 4) v1 ==== 89150+0+0 (secure 0 0 0) 0x7fbad000ab70 con 0x7fbad815d3f0 2024-01-31T05:00:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.988+0000 7fbacf400640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbab006ae70 0x7fbab006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.988+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbad0088500 con 0x7fbad815d3f0 2024-01-31T05:00:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.989+0000 7fbad6a00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbab006ae70 0x7fbab006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:43.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:43.989+0000 7fbad6a00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbab006ae70 0x7fbab006d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbac8000c00 tx=0x7fbac803d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:44.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.003+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fbad0053890 con 0x7fbad815d3f0 2024-01-31T05:00:44.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.199+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbad80caae0 con 0x7fbad815d3f0 2024-01-31T05:00:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.554+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fbad0053c60 con 0x7fbad815d3f0 2024-01-31T05:00:44.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.556+0000 7fbacf400640 1 -- 172.21.15.97:0/1261205735 <== mon.1 v2:172.21.15.188:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbad007a590 con 0x7fbad815d3f0 2024-01-31T05:00:44.556 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-01-31T05:00:44.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.563+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbab006ae70 msgr2=0x7fbab006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:44.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.563+0000 7fbad7e00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbab006ae70 0x7fbab006d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbac8000c00 tx=0x7fbac803d040 comp rx=0 tx=0).stop 2024-01-31T05:00:44.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.563+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad815d3f0 msgr2=0x7fbad814cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:44.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.563+0000 7fbad7e00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad815d3f0 0x7fbad814cff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbad80d4270 tx=0x7fbad000cb90 comp rx=0 tx=0).stop 2024-01-31T05:00:44.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 shutdown_connections 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbad815d3f0 0x7fbad814cff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbad80d3850 0x7fbad814cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbab006ae70 0x7fbab006d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 --2- 172.21.15.97:0/1261205735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad80d1e60 0x7fbad8149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 >> 172.21.15.97:0/1261205735 conn(0x7fbad808cb00 msgr2=0x7fbad8160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 shutdown_connections 2024-01-31T05:00:44.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.565+0000 7fbad7e00640 1 -- 172.21.15.97:0/1261205735 wait complete. 2024-01-31T05:00:44.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-01-31T05:00:44.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.778+0000 7f4792400640 1 Processor -- start 2024-01-31T05:00:44.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.778+0000 7f4792400640 1 -- start start 2024-01-31T05:00:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.780+0000 7f4792400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f478c15d340 0x7f478c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.780+0000 7f4792400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f478c070e40 0x7f478c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.780+0000 7f4792400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c0cc070 con 0x7f478c0b0f50 2024-01-31T05:00:44.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.780+0000 7f4792400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c071a00 con 0x7f478c070e40 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.780+0000 7f4792400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c0cc350 con 0x7f478c15d340 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.780+0000 7f478be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f478c070e40 0x7f478c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f478be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f478c070e40 0x7f478c0b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46212/0 (socket says 172.21.15.97:46212) 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f478be00640 1 -- 172.21.15.97:0/1376501294 learned_addr learned my addr 172.21.15.97:0/1376501294 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f4791000640 1 --2- 172.21.15.97:0/1376501294 >> v2:172.21.15.188:3300/0 conn(0x7f478c15d340 0x7f478c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f478be00640 1 -- 172.21.15.97:0/1376501294 >> v2:172.21.15.188:3300/0 conn(0x7f478c15d340 msgr2=0x7f478c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f478be00640 1 --2- 172.21.15.97:0/1376501294 >> v2:172.21.15.188:3300/0 conn(0x7f478c15d340 0x7f478c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f478be00640 1 -- 172.21.15.97:0/1376501294 >> v1:172.21.15.188:6789/0 conn(0x7f478c0b0f50 legacy=0x7f478c160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:44.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.781+0000 7f478be00640 1 -- 172.21.15.97:0/1376501294 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0cc630 con 0x7f478c070e40 2024-01-31T05:00:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.782+0000 7f478be00640 1 --2- 172.21.15.97:0/1376501294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c070e40 0x7f478c0b1bb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f478c067040 tx=0x7f477402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd998874ff41ef9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.782+0000 7f478b400640 1 -- 172.21.15.97:0/1376501294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774031870 con 0x7f478c070e40 2024-01-31T05:00:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.782+0000 7f478b400640 1 -- 172.21.15.97:0/1376501294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47740319d0 con 0x7f478c070e40 2024-01-31T05:00:44.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.782+0000 7f478b400640 1 -- 172.21.15.97:0/1376501294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774031d60 con 0x7f478c070e40 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 -- 172.21.15.97:0/1376501294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c070e40 msgr2=0x7f478c0b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 --2- 172.21.15.97:0/1376501294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c070e40 0x7f478c0b1bb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f478c067040 tx=0x7f477402b910 comp rx=0 tx=0).stop 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 -- 172.21.15.97:0/1376501294 shutdown_connections 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 --2- 172.21.15.97:0/1376501294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c070e40 0x7f478c0b1bb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 --2- 172.21.15.97:0/1376501294 >> v2:172.21.15.188:3300/0 conn(0x7f478c15d340 0x7f478c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 -- 172.21.15.97:0/1376501294 >> 172.21.15.97:0/1376501294 conn(0x7f478c066490 msgr2=0x7f478c0668a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 -- 172.21.15.97:0/1376501294 shutdown_connections 2024-01-31T05:00:44.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.783+0000 7f4792400640 1 -- 172.21.15.97:0/1376501294 wait complete. 2024-01-31T05:00:44.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.785+0000 7f4792400640 1 Processor -- start 2024-01-31T05:00:44.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.785+0000 7f4792400640 1 -- start start 2024-01-31T05:00:44.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f4792400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f478c070e40 0x7f478c15f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:44.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f4792400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 0x7f478c15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:44.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f4792400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f478c15d340 0x7f478c143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:44.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f4792400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15f550 con 0x7f478c0b0f50 2024-01-31T05:00:44.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f4792400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15df70 con 0x7f478c15d340 2024-01-31T05:00:44.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f4792400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15e250 con 0x7f478c070e40 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f478be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 0x7f478c15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f478be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 0x7f478c15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46224/0 (socket says 172.21.15.97:46224) 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.786+0000 7f478be00640 1 -- 172.21.15.97:0/2944165228 learned_addr learned my addr 172.21.15.97:0/2944165228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f4791000640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f478c070e40 0x7f478c15f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f4791a00640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f478c15d340 0x7f478c143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f478be00640 1 -- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f478c070e40 msgr2=0x7f478c15f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f478be00640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f478c070e40 0x7f478c15f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f478be00640 1 -- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f478c15d340 msgr2=0x7f478c143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f478be00640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f478c15d340 0x7f478c143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f478be00640 1 -- 172.21.15.97:0/2944165228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47740094d0 con 0x7f478c0b0f50 2024-01-31T05:00:44.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f4791a00640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f478c15d340 0x7f478c143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f478be00640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 0x7f478c15d720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f47740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.787+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477403abe0 con 0x7f478c0b0f50 2024-01-31T05:00:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.788+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477403ad40 con 0x7f478c0b0f50 2024-01-31T05:00:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.788+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774044610 con 0x7f478c0b0f50 2024-01-31T05:00:44.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.788+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c15e530 con 0x7f478c0b0f50 2024-01-31T05:00:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.788+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c15eaf0 con 0x7f478c0b0f50 2024-01-31T05:00:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.789+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c15ece0 con 0x7f478c0b0f50 2024-01-31T05:00:44.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.790+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f477404d450 con 0x7f478c0b0f50 2024-01-31T05:00:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.790+0000 7f4788c00640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f476406aed0 0x7f476406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.791+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f477402d040 con 0x7f478c0b0f50 2024-01-31T05:00:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.791+0000 7f4791000640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f476406aed0 0x7f476406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:44.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.791+0000 7f4791000640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f476406aed0 0x7f476406d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4784009a60 tx=0x7f4784003990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:44.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.796+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4774076af0 con 0x7f478c0b0f50 2024-01-31T05:00:44.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:44.990+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f478c067dc0 con 0x7f478c0b0f50 2024-01-31T05:00:45.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.538+0000 7f4788c00640 1 -- 172.21.15.97:0/2944165228 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f47740350d0 con 0x7f478c0b0f50 2024-01-31T05:00:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-01-31T05:00:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f476406aed0 msgr2=0x7f476406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f476406aed0 0x7f476406d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4784009a60 tx=0x7f4784003990 comp rx=0 tx=0).stop 2024-01-31T05:00:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 msgr2=0x7f478c15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 0x7f478c15d720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f47740046c0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 shutdown_connections 2024-01-31T05:00:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f478c15d340 0x7f478c143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f476406aed0 0x7f476406d390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f478c0b0f50 0x7f478c15d720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 --2- 172.21.15.97:0/2944165228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f478c070e40 0x7f478c15f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.541+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 >> 172.21.15.97:0/2944165228 conn(0x7f478c066490 msgr2=0x7f478c159790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.542+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 shutdown_connections 2024-01-31T05:00:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.542+0000 7f4792400640 1 -- 172.21.15.97:0/2944165228 wait complete. 2024-01-31T05:00:45.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-31T05:00:45.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.758+0000 7f86d3000640 1 Processor -- start 2024-01-31T05:00:45.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.758+0000 7f86d3000640 1 -- start start 2024-01-31T05:00:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.760+0000 7f86d3000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f86cc160770 0x7f86cc0af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.760+0000 7f86d3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.760+0000 7f86d3000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86cc0b2ea0 con 0x7f86cc06f150 2024-01-31T05:00:45.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.760+0000 7f86d3000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc0b3420 con 0x7f86cc160770 2024-01-31T05:00:45.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.760+0000 7f86d3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc071f10 con 0x7f86cc0b48d0 2024-01-31T05:00:45.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.760+0000 7f86d2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:45.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d1c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f86cc160770 0x7f86cc0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:45.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc0b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:35832/0 (socket says 172.21.15.97:35832) 2024-01-31T05:00:45.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d2600640 1 -- 172.21.15.97:0/2614061695 learned_addr learned my addr 172.21.15.97:0/2614061695 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:45.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d1200640 1 --1- 172.21.15.97:0/2614061695 >> v1:172.21.15.188:6789/0 conn(0x7f86cc06f150 0x7f86cc06d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:58554/0 (socket says 172.21.15.97:58554) 2024-01-31T05:00:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d2600640 1 -- 172.21.15.97:0/2614061695 >> v2:172.21.15.188:3300/0 conn(0x7f86cc160770 msgr2=0x7f86cc0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d2600640 1 --2- 172.21.15.97:0/2614061695 >> v2:172.21.15.188:3300/0 conn(0x7f86cc160770 0x7f86cc0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d2600640 1 -- 172.21.15.97:0/2614061695 >> v1:172.21.15.188:6789/0 conn(0x7f86cc06f150 legacy=0x7f86cc06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d2600640 1 -- 172.21.15.97:0/2614061695 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc160b50 con 0x7f86cc0b48d0 2024-01-31T05:00:45.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.761+0000 7f86d1c00640 1 --2- 172.21.15.97:0/2614061695 >> v2:172.21.15.188:3300/0 conn(0x7f86cc160770 0x7f86cc0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:00:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.763+0000 7f86d2600640 1 --2- 172.21.15.97:0/2614061695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc0b4d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3283ec1e1c22f0a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.763+0000 7f86cbe00640 1 -- 172.21.15.97:0/2614061695 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4030840 con 0x7f86cc0b48d0 2024-01-31T05:00:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.763+0000 7f86cbe00640 1 -- 172.21.15.97:0/2614061695 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c40309a0 con 0x7f86cc0b48d0 2024-01-31T05:00:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.763+0000 7f86cbe00640 1 -- 172.21.15.97:0/2614061695 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4030d30 con 0x7f86cc0b48d0 2024-01-31T05:00:45.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 -- 172.21.15.97:0/2614061695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 msgr2=0x7f86cc0b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 --2- 172.21.15.97:0/2614061695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc0b4d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c402b980 comp rx=0 tx=0).stop 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 -- 172.21.15.97:0/2614061695 shutdown_connections 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 --2- 172.21.15.97:0/2614061695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc0b4d60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 --2- 172.21.15.97:0/2614061695 >> v2:172.21.15.188:3300/0 conn(0x7f86cc160770 0x7f86cc0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 -- 172.21.15.97:0/2614061695 >> 172.21.15.97:0/2614061695 conn(0x7f86cc0b1b00 msgr2=0x7f86cc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 -- 172.21.15.97:0/2614061695 shutdown_connections 2024-01-31T05:00:45.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.764+0000 7f86d3000640 1 -- 172.21.15.97:0/2614061695 wait complete. 2024-01-31T05:00:45.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.766+0000 7f86d3000640 1 Processor -- start 2024-01-31T05:00:45.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.766+0000 7f86d3000640 1 -- start start 2024-01-31T05:00:45.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d3000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86cc06f150 0x7f86cc1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:45.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d3000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 0x7f86cc0ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d3000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc14b0a0 con 0x7f86cc160770 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d3000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1f48f0 con 0x7f86cc06f150 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1f4bd0 con 0x7f86cc0b48d0 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d2600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 0x7f86cc0ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d2600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 0x7f86cc0ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46546/0 (socket says 172.21.15.97:46546) 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d2600640 1 -- 172.21.15.97:0/3164358096 learned_addr learned my addr 172.21.15.97:0/3164358096 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d1c00640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86cc06f150 0x7f86cc1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:45.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.767+0000 7f86d1200640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d2600640 1 -- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 msgr2=0x7f86cc14ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d2600640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc14ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d2600640 1 -- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86cc06f150 msgr2=0x7f86cc1f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d2600640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86cc06f150 0x7f86cc1f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d2600640 1 -- 172.21.15.97:0/3164358096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009a70 con 0x7f86cc160770 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d1200640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc14ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d2600640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 0x7f86cc0ad180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4039c90 con 0x7f86cc160770 2024-01-31T05:00:45.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c4039e10 con 0x7f86cc160770 2024-01-31T05:00:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4043640 con 0x7f86cc160770 2024-01-31T05:00:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c40094d0 con 0x7f86cc160770 2024-01-31T05:00:45.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.768+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc1f5190 con 0x7f86cc160770 2024-01-31T05:00:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.770+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc0b3fa0 con 0x7f86cc160770 2024-01-31T05:00:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.770+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f86c40437a0 con 0x7f86cc160770 2024-01-31T05:00:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.771+0000 7f86c9600640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86a406aed0 0x7f86a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.771+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f86c403e070 con 0x7f86cc160770 2024-01-31T05:00:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.772+0000 7f86d1c00640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86a406aed0 0x7f86a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:45.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.772+0000 7f86d1c00640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86a406aed0 0x7f86a406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0114e0 tx=0x7f86bc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:45.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.777+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f86c4077470 con 0x7f86cc160770 2024-01-31T05:00:45.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:45.999+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f86cc067dc0 con 0x7f86cc160770 2024-01-31T05:00:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.585+0000 7f86c9600640 1 -- 172.21.15.97:0/3164358096 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f86c40aae60 con 0x7f86cc160770 2024-01-31T05:00:46.585 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-01-31T05:00:46.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.590+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86a406aed0 msgr2=0x7f86a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:46.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.590+0000 7f86d3000640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86a406aed0 0x7f86a406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0114e0 tx=0x7f86bc00b040 comp rx=0 tx=0).stop 2024-01-31T05:00:46.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.590+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 msgr2=0x7f86cc0ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:46.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.590+0000 7f86d3000640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 0x7f86cc0ad180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009870 tx=0x7f86c4004730 comp rx=0 tx=0).stop 2024-01-31T05:00:46.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.592+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 shutdown_connections 2024-01-31T05:00:46.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.592+0000 7f86d3000640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86a406aed0 0x7f86a406d390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.592+0000 7f86d3000640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86cc160770 0x7f86cc0ad180 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.592+0000 7f86d3000640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86cc0b48d0 0x7f86cc14ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.592+0000 7f86d3000640 1 --2- 172.21.15.97:0/3164358096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86cc06f150 0x7f86cc1f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.593+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 >> 172.21.15.97:0/3164358096 conn(0x7f86cc0b1b00 msgr2=0x7f86cc06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:46.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.593+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 shutdown_connections 2024-01-31T05:00:46.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.593+0000 7f86d3000640 1 -- 172.21.15.97:0/3164358096 wait complete. 2024-01-31T05:00:46.618 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-01-31T05:00:46.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-01-31T05:00:46.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.791+0000 7f8ac3200640 1 Processor -- start 2024-01-31T05:00:46.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.792+0000 7f8ac3200640 1 -- start start 2024-01-31T05:00:46.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15b430 0x7f8abc0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8abc15b810 0x7f8abc0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8abc0b01b0 0x7f8abc0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac3200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0cf580 con 0x7f8abc0b01b0 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0d1c00 con 0x7f8abc15b810 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0d1ee0 con 0x7f8abc15b430 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac1200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8abc0b01b0 0x7f8abc0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac1200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8abc0b01b0 0x7f8abc0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46554/0 (socket says 172.21.15.97:46554) 2024-01-31T05:00:46.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.793+0000 7f8ac1200640 1 -- 172.21.15.97:0/2983466963 learned_addr learned my addr 172.21.15.97:0/2983466963 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8abbe00640 1 --2- 172.21.15.97:0/2983466963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15b430 0x7f8abc0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8abb400640 1 --2- 172.21.15.97:0/2983466963 >> v2:172.21.15.188:3300/0 conn(0x7f8abc15b810 0x7f8abc0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8ac1200640 1 -- 172.21.15.97:0/2983466963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15b430 msgr2=0x7f8abc0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8ac1200640 1 --2- 172.21.15.97:0/2983466963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15b430 0x7f8abc0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8ac1200640 1 -- 172.21.15.97:0/2983466963 >> v2:172.21.15.188:3300/0 conn(0x7f8abc15b810 msgr2=0x7f8abc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8ac1200640 1 --2- 172.21.15.97:0/2983466963 >> v2:172.21.15.188:3300/0 conn(0x7f8abc15b810 0x7f8abc0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.794+0000 7f8ac1200640 1 -- 172.21.15.97:0/2983466963 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cc310 con 0x7f8abc0b01b0 2024-01-31T05:00:46.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.795+0000 7f8ac1200640 1 --2- 172.21.15.97:0/2983466963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc0b01b0 0x7f8abc0cdaf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8abc06e560 tx=0x7f8ab402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48c91759f14f972 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:46.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.795+0000 7f8abaa00640 1 -- 172.21.15.97:0/2983466963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab402c9e0 con 0x7f8abc0b01b0 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.795+0000 7f8abaa00640 1 -- 172.21.15.97:0/2983466963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab402cb40 con 0x7f8abc0b01b0 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.795+0000 7f8abaa00640 1 -- 172.21.15.97:0/2983466963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40343f0 con 0x7f8abc0b01b0 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 -- 172.21.15.97:0/2983466963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc0b01b0 msgr2=0x7f8abc0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2983466963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc0b01b0 0x7f8abc0cdaf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8abc06e560 tx=0x7f8ab402be40 comp rx=0 tx=0).stop 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 -- 172.21.15.97:0/2983466963 shutdown_connections 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2983466963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc0b01b0 0x7f8abc0cdaf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2983466963 >> v2:172.21.15.188:3300/0 conn(0x7f8abc15b810 0x7f8abc0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2983466963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15b430 0x7f8abc0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 -- 172.21.15.97:0/2983466963 >> 172.21.15.97:0/2983466963 conn(0x7f8abc09a350 msgr2=0x7f8abc0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 -- 172.21.15.97:0/2983466963 shutdown_connections 2024-01-31T05:00:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.796+0000 7f8ac3200640 1 -- 172.21.15.97:0/2983466963 wait complete. 2024-01-31T05:00:46.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.798+0000 7f8ac3200640 1 Processor -- start 2024-01-31T05:00:46.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.798+0000 7f8ac3200640 1 -- start start 2024-01-31T05:00:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc0b01b0 0x7f8abc0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 0x7f8abc14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8ac3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc15b810 0x7f8abc147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8ac3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0c5000 con 0x7f8abc15b430 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8ac3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc14b700 con 0x7f8abc15b810 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8ac3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc148430 con 0x7f8abc0b01b0 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 0x7f8abc14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 0x7f8abc14b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46570/0 (socket says 172.21.15.97:46570) 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 -- 172.21.15.97:0/2700198204 learned_addr learned my addr 172.21.15.97:0/2700198204 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 -- 172.21.15.97:0/2700198204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc0b01b0 msgr2=0x7f8abc0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc0b01b0 0x7f8abc0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 -- 172.21.15.97:0/2700198204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc15b810 msgr2=0x7f8abc147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc15b810 0x7f8abc147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 -- 172.21.15.97:0/2700198204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009b30 con 0x7f8abc15b430 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.799+0000 7f8abb400640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 0x7f8abc14b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d5f0 tx=0x7f8aa400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.800+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40125f0 con 0x7f8abc15b430 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.800+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4012750 con 0x7f8abc15b430 2024-01-31T05:00:46.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.800+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4012b00 con 0x7f8abc15b430 2024-01-31T05:00:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.800+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0c6b40 con 0x7f8abc15b430 2024-01-31T05:00:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.800+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc0c7100 con 0x7f8abc15b430 2024-01-31T05:00:46.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.801+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc067830 con 0x7f8abc15b430 2024-01-31T05:00:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.803+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8aa4012c60 con 0x7f8abc15b430 2024-01-31T05:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.803+0000 7f8ab3e00640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a9406af00 0x7f8a9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.803+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aa408a550 con 0x7f8abc15b430 2024-01-31T05:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.804+0000 7f8abbe00640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a9406af00 0x7f8a9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.804+0000 7f8abbe00640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a9406af00 0x7f8a9406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009870 tx=0x7f8aac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.807+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8aa4055500 con 0x7f8abc15b430 2024-01-31T05:00:46.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:46.996+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f8abc0ce8f0 con 0x7f8abc15b430 2024-01-31T05:00:47.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.545+0000 7f8ab3e00640 1 -- 172.21.15.97:0/2700198204 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7f8aa4058b20 con 0x7f8abc15b430 2024-01-31T05:00:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.549+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a9406af00 msgr2=0x7f8a9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.549+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a9406af00 0x7f8a9406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009870 tx=0x7f8aac008040 comp rx=0 tx=0).stop 2024-01-31T05:00:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.549+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 msgr2=0x7f8abc14b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:47.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.549+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 0x7f8abc14b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400d5f0 tx=0x7f8aa400dab0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 shutdown_connections 2024-01-31T05:00:47.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc15b810 0x7f8abc147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a9406af00 0x7f8a9406d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8abc15b430 0x7f8abc14b1c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 --2- 172.21.15.97:0/2700198204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc0b01b0 0x7f8abc0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 >> 172.21.15.97:0/2700198204 conn(0x7f8abc09a350 msgr2=0x7f8abc15a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:47.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 shutdown_connections 2024-01-31T05:00:47.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.550+0000 7f8ac3200640 1 -- 172.21.15.97:0/2700198204 wait complete. 2024-01-31T05:00:47.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-01-31T05:00:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.772+0000 7f8e49400640 1 Processor -- start 2024-01-31T05:00:47.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.772+0000 7f8e49400640 1 -- start start 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e49400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e441449a0 0x7f8e4415b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e49400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8e440b0fd0 0x7f8e440b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e49400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e440705e0 con 0x7f8e44143ec0 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e49400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e441442a0 con 0x7f8e441449a0 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e49400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440b1920 con 0x7f8e440b0fd0 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e41e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e441449a0 0x7f8e4415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e41e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e441449a0 0x7f8e4415b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46580/0 (socket says 172.21.15.97:46580) 2024-01-31T05:00:47.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.774+0000 7f8e41e00640 1 -- 172.21.15.97:0/751252213 learned_addr learned my addr 172.21.15.97:0/751252213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.775+0000 7f8e43200640 1 --2- 172.21.15.97:0/751252213 >> v2:172.21.15.188:3300/0 conn(0x7f8e440b0fd0 0x7f8e440b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.775+0000 7f8e41e00640 1 -- 172.21.15.97:0/751252213 >> v2:172.21.15.188:3300/0 conn(0x7f8e440b0fd0 msgr2=0x7f8e440b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.775+0000 7f8e41e00640 1 --2- 172.21.15.97:0/751252213 >> v2:172.21.15.188:3300/0 conn(0x7f8e440b0fd0 0x7f8e440b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.775+0000 7f8e41e00640 1 -- 172.21.15.97:0/751252213 >> v1:172.21.15.188:6789/0 conn(0x7f8e44143ec0 legacy=0x7f8e440ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.775+0000 7f8e41e00640 1 -- 172.21.15.97:0/751252213 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44070850 con 0x7f8e441449a0 2024-01-31T05:00:47.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.776+0000 7f8e41e00640 1 --2- 172.21.15.97:0/751252213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415b740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e440cf070 tx=0x7f8e2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908fef48f10fe5ee server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.776+0000 7f8e41400640 1 -- 172.21.15.97:0/751252213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c031870 con 0x7f8e441449a0 2024-01-31T05:00:47.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.776+0000 7f8e41400640 1 -- 172.21.15.97:0/751252213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c0319d0 con 0x7f8e441449a0 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.776+0000 7f8e41400640 1 -- 172.21.15.97:0/751252213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c031d60 con 0x7f8e441449a0 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 -- 172.21.15.97:0/751252213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 msgr2=0x7f8e4415b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 --2- 172.21.15.97:0/751252213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415b740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8e440cf070 tx=0x7f8e2c02b910 comp rx=0 tx=0).stop 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 -- 172.21.15.97:0/751252213 shutdown_connections 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 --2- 172.21.15.97:0/751252213 >> v2:172.21.15.188:3300/0 conn(0x7f8e440b0fd0 0x7f8e440b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 --2- 172.21.15.97:0/751252213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415b740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 -- 172.21.15.97:0/751252213 >> 172.21.15.97:0/751252213 conn(0x7f8e440b45b0 msgr2=0x7f8e440b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 -- 172.21.15.97:0/751252213 shutdown_connections 2024-01-31T05:00:47.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.777+0000 7f8e49400640 1 -- 172.21.15.97:0/751252213 wait complete. 2024-01-31T05:00:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.778+0000 7f8e49400640 1 Processor -- start 2024-01-31T05:00:47.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 -- start start 2024-01-31T05:00:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e440b0fd0 0x7f8e4415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:47.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e44143ec0 0x7f8e4413c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4413c550 con 0x7f8e441449a0 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4413ba40 con 0x7f8e44143ec0 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.779+0000 7f8e49400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441f18b0 con 0x7f8e440b0fd0 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46588/0 (socket says 172.21.15.97:46588) 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 -- 172.21.15.97:0/401108647 learned_addr learned my addr 172.21.15.97:0/401108647 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 -- 172.21.15.97:0/401108647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e440b0fd0 msgr2=0x7f8e4415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e440b0fd0 0x7f8e4415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 -- 172.21.15.97:0/401108647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e44143ec0 msgr2=0x7f8e4413c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:47.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e44143ec0 0x7f8e4413c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 -- 172.21.15.97:0/401108647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c0094d0 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.780+0000 7f8e43200640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415d550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c007dc0 tx=0x7f8e3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.781+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c014420 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.781+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c0145a0 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.781+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c014950 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.781+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e441f1b90 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.781+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4413c170 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.782+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4413c690 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.783+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8e3c014ab0 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.783+0000 7f8e3aa00640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1c06aed0 0x7f8e1c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.784+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f8e3c089fe0 con 0x7f8e441449a0 2024-01-31T05:00:47.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.784+0000 7f8e42800640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1c06aed0 0x7f8e1c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:47.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.784+0000 7f8e42800640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1c06aed0 0x7f8e1c06d390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8e34003cc0 tx=0x7f8e340079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:47.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.788+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8e4413c690 con 0x7f8e441449a0 2024-01-31T05:00:47.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:47.977+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8e4413c690 con 0x7f8e441449a0 2024-01-31T05:00:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.596+0000 7f8e3aa00640 1 -- 172.21.15.97:0/401108647 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7f8e4413c690 con 0x7f8e441449a0 2024-01-31T05:00:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' created 2024-01-31T05:00:49.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1c06aed0 msgr2=0x7f8e1c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:49.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1c06aed0 0x7f8e1c06d390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8e34003cc0 tx=0x7f8e340079b0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 msgr2=0x7f8e4415d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:49.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415d550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c007dc0 tx=0x7f8e3c00e4b0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 shutdown_connections 2024-01-31T05:00:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1c06aed0 0x7f8e1c06d390 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e441449a0 0x7f8e4415d550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e44143ec0 0x7f8e4413c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 --2- 172.21.15.97:0/401108647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e440b0fd0 0x7f8e4415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.600+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 >> 172.21.15.97:0/401108647 conn(0x7f8e440b45b0 msgr2=0x7f8e440b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:49.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.601+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 shutdown_connections 2024-01-31T05:00:49.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.601+0000 7f8e49400640 1 -- 172.21.15.97:0/401108647 wait complete. 2024-01-31T05:00:49.632 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-01-31T05:00:49.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.829+0000 7fc5e7200640 1 Processor -- start 2024-01-31T05:00:49.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.829+0000 7fc5e7200640 1 -- start start 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.830+0000 7fc5e7200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5e015ed30 0x7fc5e00721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.830+0000 7fc5e7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5e015f110 0x7fc5e0070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.830+0000 7fc5e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e00c9280 0x7fc5e00c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.830+0000 7fc5e7200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e0072730 con 0x7fc5e015ed30 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.830+0000 7fc5e7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e006e730 con 0x7fc5e015f110 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.830+0000 7fc5e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0155c10 con 0x7fc5e00c9280 2024-01-31T05:00:49.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5e015ed30 0x7fc5e00721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5e015ed30 0x7fc5e00721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46600/0 (socket says 172.21.15.97:46600) 2024-01-31T05:00:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 -- 172.21.15.97:0/3650988381 learned_addr learned my addr 172.21.15.97:0/3650988381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e6800640 1 --2- 172.21.15.97:0/3650988381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e00c9280 0x7fc5e00c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5400640 1 --2- 172.21.15.97:0/3650988381 >> v2:172.21.15.188:3300/0 conn(0x7fc5e015f110 0x7fc5e0070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:49.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 -- 172.21.15.97:0/3650988381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e00c9280 msgr2=0x7fc5e00c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 --2- 172.21.15.97:0/3650988381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e00c9280 0x7fc5e00c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 -- 172.21.15.97:0/3650988381 >> v2:172.21.15.188:3300/0 conn(0x7fc5e015f110 msgr2=0x7fc5e0070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 --2- 172.21.15.97:0/3650988381 >> v2:172.21.15.188:3300/0 conn(0x7fc5e015f110 0x7fc5e0070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.831+0000 7fc5e5e00640 1 -- 172.21.15.97:0/3650988381 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e006e450 con 0x7fc5e015ed30 2024-01-31T05:00:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.832+0000 7fc5e5e00640 1 --2- 172.21.15.97:0/3650988381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e015ed30 0x7fc5e00721f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009870 tx=0x7fc5d002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6395de62fa20a003 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.833+0000 7fc5e4a00640 1 -- 172.21.15.97:0/3650988381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00317c0 con 0x7fc5e015ed30 2024-01-31T05:00:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.833+0000 7fc5e4a00640 1 -- 172.21.15.97:0/3650988381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d0031920 con 0x7fc5e015ed30 2024-01-31T05:00:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.833+0000 7fc5e4a00640 1 -- 172.21.15.97:0/3650988381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0031cb0 con 0x7fc5e015ed30 2024-01-31T05:00:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.833+0000 7fc5e7200640 1 -- 172.21.15.97:0/3650988381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e015ed30 msgr2=0x7fc5e00721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:49.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.833+0000 7fc5e7200640 1 --2- 172.21.15.97:0/3650988381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e015ed30 0x7fc5e00721f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009870 tx=0x7fc5d002b860 comp rx=0 tx=0).stop 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 -- 172.21.15.97:0/3650988381 shutdown_connections 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 --2- 172.21.15.97:0/3650988381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e00c9280 0x7fc5e00c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 --2- 172.21.15.97:0/3650988381 >> v2:172.21.15.188:3300/0 conn(0x7fc5e015f110 0x7fc5e0070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 --2- 172.21.15.97:0/3650988381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e015ed30 0x7fc5e00721f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 -- 172.21.15.97:0/3650988381 >> 172.21.15.97:0/3650988381 conn(0x7fc5e00cec30 msgr2=0x7fc5e00cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 -- 172.21.15.97:0/3650988381 shutdown_connections 2024-01-31T05:00:49.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.834+0000 7fc5e7200640 1 -- 172.21.15.97:0/3650988381 wait complete. 2024-01-31T05:00:49.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.835+0000 7fc5e7200640 1 Processor -- start 2024-01-31T05:00:49.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.836+0000 7fc5e7200640 1 -- start start 2024-01-31T05:00:49.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.836+0000 7fc5e7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 0x7fc5e01f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.836+0000 7fc5e7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5e015ed30 0x7fc5e01f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.836+0000 7fc5e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e015f110 0x7fc5e015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.836+0000 7fc5e7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015b7d0 con 0x7fc5e00c9280 2024-01-31T05:00:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00c5d00 con 0x7fc5e015ed30 2024-01-31T05:00:49.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00c5f50 con 0x7fc5e015f110 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 0x7fc5e01f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 0x7fc5e01f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46602/0 (socket says 172.21.15.97:46602) 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 -- 172.21.15.97:0/1439248429 learned_addr learned my addr 172.21.15.97:0/1439248429 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 -- 172.21.15.97:0/1439248429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e015f110 msgr2=0x7fc5e015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e015f110 0x7fc5e015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 -- 172.21.15.97:0/1439248429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5e015ed30 msgr2=0x7fc5e01f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5e015ed30 0x7fc5e01f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:49.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 -- 172.21.15.97:0/1439248429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00094d0 con 0x7fc5e00c9280 2024-01-31T05:00:49.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.837+0000 7fc5e5e00640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 0x7fc5e01f4210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5d002b8a0 tx=0x7fc5d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:49.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.838+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0032860 con 0x7fc5e00c9280 2024-01-31T05:00:49.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.838+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d00329e0 con 0x7fc5e00c9280 2024-01-31T05:00:49.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.838+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0032d90 con 0x7fc5e00c9280 2024-01-31T05:00:49.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.838+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e00c6230 con 0x7fc5e00c9280 2024-01-31T05:00:49.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.838+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e00c65c0 con 0x7fc5e00c9280 2024-01-31T05:00:49.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.839+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e00c6a30 con 0x7fc5e00c9280 2024-01-31T05:00:49.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.840+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc5d004e450 con 0x7fc5e00c9280 2024-01-31T05:00:49.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.841+0000 7fc5de000640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc5b806aed0 0x7fc5b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:49.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.841+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fc5d00abb40 con 0x7fc5e00c9280 2024-01-31T05:00:49.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.841+0000 7fc5e5400640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc5b806aed0 0x7fc5b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:49.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.842+0000 7fc5e5400640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc5b806aed0 0x7fc5b806d390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0154880 tx=0x7fc5c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:49.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:49.846+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc5d00768f0 con 0x7fc5e00c9280 2024-01-31T05:00:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.040+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fc5e00676b0 con 0x7fc5e00c9280 2024-01-31T05:00:50.334 INFO:tasks.ceph.mds.a.smithi097.stderr:2024-01-31T05:00:50.334+0000 7f8038e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:00:50.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.601+0000 7fc5de000640 1 -- 172.21.15.97:0/1439248429 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc5d0079f10 con 0x7fc5e00c9280 2024-01-31T05:00:50.601 INFO:teuthology.orchestra.run.smithi097.stderr:set pool 4 allow_ec_overwrites to true 2024-01-31T05:00:50.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.605+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc5b806aed0 msgr2=0x7fc5b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:50.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.605+0000 7fc5e7200640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc5b806aed0 0x7fc5b806d390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0154880 tx=0x7fc5c8008040 comp rx=0 tx=0).stop 2024-01-31T05:00:50.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.605+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 msgr2=0x7fc5e01f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:50.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.605+0000 7fc5e7200640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 0x7fc5e01f4210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc5d002b8a0 tx=0x7fc5d00046c0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.609+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 shutdown_connections 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.609+0000 7fc5e7200640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5e015f110 0x7fc5e015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.609+0000 7fc5e7200640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5e015ed30 0x7fc5e01f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.609+0000 7fc5e7200640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc5b806aed0 0x7fc5b806d390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.609+0000 7fc5e7200640 1 --2- 172.21.15.97:0/1439248429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00c9280 0x7fc5e01f4210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.609+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 >> 172.21.15.97:0/1439248429 conn(0x7fc5e00cec30 msgr2=0x7fc5e00afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.610+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 shutdown_connections 2024-01-31T05:00:50.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.610+0000 7fc5e7200640 1 -- 172.21.15.97:0/1439248429 wait complete. 2024-01-31T05:00:50.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-01-31T05:00:50.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.827+0000 7fd06b400640 1 Processor -- start 2024-01-31T05:00:50.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.827+0000 7fd06b400640 1 -- start start 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd06b400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd06406d580 0x7fd06406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd06b400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd06415d690 0x7fd0640ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd06b400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06406dea0 con 0x7fd06406c350 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd06b400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06415e7c0 con 0x7fd06406d580 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd06b400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06415db40 con 0x7fd06415d690 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd068a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd06406d580 0x7fd06406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:50.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd068a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd06406d580 0x7fd06406d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46616/0 (socket says 172.21.15.97:46616) 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd068a00640 1 -- 172.21.15.97:0/3821355556 learned_addr learned my addr 172.21.15.97:0/3821355556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd069400640 1 --1- 172.21.15.97:0/3821355556 >> v1:172.21.15.97:6789/0 conn(0x7fd06406c350 0x7fd06406c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43458/0 (socket says 172.21.15.97:43458) 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.829+0000 7fd063e00640 1 --2- 172.21.15.97:0/3821355556 >> v2:172.21.15.188:3300/0 conn(0x7fd06415d690 0x7fd0640ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.830+0000 7fd068a00640 1 -- 172.21.15.97:0/3821355556 >> v2:172.21.15.188:3300/0 conn(0x7fd06415d690 msgr2=0x7fd0640ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.830+0000 7fd068a00640 1 --2- 172.21.15.97:0/3821355556 >> v2:172.21.15.188:3300/0 conn(0x7fd06415d690 0x7fd0640ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.830+0000 7fd068a00640 1 -- 172.21.15.97:0/3821355556 >> v1:172.21.15.97:6789/0 conn(0x7fd06406c350 legacy=0x7fd06406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.830+0000 7fd068a00640 1 -- 172.21.15.97:0/3821355556 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06415de20 con 0x7fd06406d580 2024-01-31T05:00:50.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.830+0000 7fd063400640 1 -- 172.21.15.97:0/3821355556 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707287813 0 0) 0x7fd06406dea0 con 0x7fd06406c350 2024-01-31T05:00:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.831+0000 7fd068a00640 1 --2- 172.21.15.97:0/3821355556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06406d580 0x7fd06406d960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd0640cedd0 tx=0x7fd05402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3772ab27b86add server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.831+0000 7fd063400640 1 -- 172.21.15.97:0/3821355556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054004230 con 0x7fd06406d580 2024-01-31T05:00:50.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.831+0000 7fd063400640 1 -- 172.21.15.97:0/3821355556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd054004390 con 0x7fd06406d580 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.831+0000 7fd063400640 1 -- 172.21.15.97:0/3821355556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054030840 con 0x7fd06406d580 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 -- 172.21.15.97:0/3821355556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06406d580 msgr2=0x7fd06406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 --2- 172.21.15.97:0/3821355556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06406d580 0x7fd06406d960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd0640cedd0 tx=0x7fd05402b980 comp rx=0 tx=0).stop 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 -- 172.21.15.97:0/3821355556 shutdown_connections 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 --2- 172.21.15.97:0/3821355556 >> v2:172.21.15.188:3300/0 conn(0x7fd06415d690 0x7fd0640ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 --2- 172.21.15.97:0/3821355556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06406d580 0x7fd06406d960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 -- 172.21.15.97:0/3821355556 >> 172.21.15.97:0/3821355556 conn(0x7fd0640b0a10 msgr2=0x7fd0640b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 -- 172.21.15.97:0/3821355556 shutdown_connections 2024-01-31T05:00:50.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.832+0000 7fd06b400640 1 -- 172.21.15.97:0/3821355556 wait complete. 2024-01-31T05:00:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.834+0000 7fd06b400640 1 Processor -- start 2024-01-31T05:00:50.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.834+0000 7fd06b400640 1 -- start start 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd06b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06406c350 0x7fd0640ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd06b400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd06406d580 0x7fd0641e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd06b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 0x7fd0640716e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd06b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ad0b0 con 0x7fd06415d690 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd06b400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ad390 con 0x7fd06406d580 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd06b400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ad670 con 0x7fd06406c350 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 0x7fd0640716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 0x7fd0640716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46628/0 (socket says 172.21.15.97:46628) 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 -- 172.21.15.97:0/3868695653 learned_addr learned my addr 172.21.15.97:0/3868695653 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd063e00640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd06406d580 0x7fd0641e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:50.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 -- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06406c350 msgr2=0x7fd0640ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:50.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06406c350 0x7fd0640ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 -- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd06406d580 msgr2=0x7fd0641e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:50.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd06406d580 0x7fd0641e8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:50.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.835+0000 7fd069400640 1 -- 172.21.15.97:0/3868695653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd054009460 con 0x7fd06415d690 2024-01-31T05:00:50.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.836+0000 7fd069400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 0x7fd0640716e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c7a0 tx=0x7fd05c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.836+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00a640 con 0x7fd06415d690 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.836+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c00a7a0 con 0x7fd06415d690 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.836+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0640ad950 con 0x7fd06415d690 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.836+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00abd0 con 0x7fd06415d690 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.836+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0641e88d0 con 0x7fd06415d690 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.837+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0640ade70 con 0x7fd06415d690 2024-01-31T05:00:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.838+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd05c00ad30 con 0x7fd06415d690 2024-01-31T05:00:50.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.839+0000 7fd060c00640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd03c06af00 0x7fd03c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:50.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.839+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fd05c0890b0 con 0x7fd06415d690 2024-01-31T05:00:50.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.839+0000 7fd068a00640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd03c06af00 0x7fd03c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:50.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.840+0000 7fd068a00640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd03c06af00 0x7fd03c06d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd054035000 tx=0x7fd054030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:50.843+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd0640ade70 con 0x7fd06415d690 2024-01-31T05:00:51.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.074+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fd0640ade70 con 0x7fd06415d690 2024-01-31T05:00:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.679+0000 7fd060c00640 1 -- 172.21.15.97:0/3868695653 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7fd0640ade70 con 0x7fd06415d690 2024-01-31T05:00:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:added data pool 4 to fsmap 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd03c06af00 msgr2=0x7fd03c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd03c06af00 0x7fd03c06d3c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd054035000 tx=0x7fd054030000 comp rx=0 tx=0).stop 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 msgr2=0x7fd0640716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 0x7fd0640716e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c7a0 tx=0x7fd05c00cc60 comp rx=0 tx=0).stop 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 shutdown_connections 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd03c06af00 0x7fd03c06d3c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd06415d690 0x7fd0640716e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd06406d580 0x7fd0641e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 --2- 172.21.15.97:0/3868695653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06406c350 0x7fd0640ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 >> 172.21.15.97:0/3868695653 conn(0x7fd0640b0a10 msgr2=0x7fd0640aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:51.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.683+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 shutdown_connections 2024-01-31T05:00:51.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.684+0000 7fd06b400640 1 -- 172.21.15.97:0/3868695653 wait complete. 2024-01-31T05:00:51.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:00:51.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.895+0000 7fe0c7e00640 1 Processor -- start 2024-01-31T05:00:51.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.896+0000 7fe0c7e00640 1 -- start start 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cfae0 0x7fe0c00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe0c00d0de0 0x7fe0c00d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c7e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c015ec50 con 0x7fe0c015d590 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c7e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00d0460 con 0x7fe0c00d0de0 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c7e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c00cded0 con 0x7fe0c00cfae0 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c5400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe0c015d590 0x7fe0c015e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43468/0 (socket says 172.21.15.97:43468) 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c5e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe0c00d0de0 0x7fe0c00d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.897+0000 7fe0c5400640 1 -- 172.21.15.97:0/2756260121 learned_addr learned my addr 172.21.15.97:0/2756260121 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:51.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0c4a00640 1 --2- 172.21.15.97:0/2756260121 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cfae0 0x7fe0c00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2756260121 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263171561 0 0) 0x7fe0c015ec50 con 0x7fe0c015d590 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2756260121 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a0003520 con 0x7fe0c015d590 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0c5e00640 1 -- 172.21.15.97:0/2756260121 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cfae0 msgr2=0x7fe0c00cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0c5e00640 1 --2- 172.21.15.97:0/2756260121 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cfae0 0x7fe0c00cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0c5e00640 1 -- 172.21.15.97:0/2756260121 >> v1:172.21.15.97:6789/0 conn(0x7fe0c015d590 legacy=0x7fe0c015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0c5e00640 1 -- 172.21.15.97:0/2756260121 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c015d970 con 0x7fe0c00d0de0 2024-01-31T05:00:51.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.898+0000 7fe0c4a00640 1 --2- 172.21.15.97:0/2756260121 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cfae0 0x7fe0c00cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:51.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.899+0000 7fe0c5e00640 1 --2- 172.21.15.97:0/2756260121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c00d1250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00d3170 tx=0x7fe0b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf3a7adc33eedcf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:51.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.899+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2756260121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8030610 con 0x7fe0c00d0de0 2024-01-31T05:00:51.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.899+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2756260121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b8030770 con 0x7fe0c00d0de0 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.899+0000 7fe0bfe00640 1 -- 172.21.15.97:0/2756260121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8030ae0 con 0x7fe0c00d0de0 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2756260121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 msgr2=0x7fe0c00d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2756260121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c00d1250 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00d3170 tx=0x7fe0b802b960 comp rx=0 tx=0).stop 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2756260121 shutdown_connections 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2756260121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c00d1250 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2756260121 >> v2:172.21.15.188:3300/0 conn(0x7fe0c00cfae0 0x7fe0c00cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2756260121 >> 172.21.15.97:0/2756260121 conn(0x7fe0c009c560 msgr2=0x7fe0c00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2756260121 shutdown_connections 2024-01-31T05:00:51.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.900+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2756260121 wait complete. 2024-01-31T05:00:51.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.902+0000 7fe0c7e00640 1 Processor -- start 2024-01-31T05:00:51.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.902+0000 7fe0c7e00640 1 -- start start 2024-01-31T05:00:51.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00cfae0 0x7fe0c01f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:51.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c0157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:51.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c015d590 0x7fe0c00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0154220 con 0x7fe0c00d0de0 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0158c50 con 0x7fe0c00cfae0 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c01583b0 con 0x7fe0c015d590 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c0157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c0157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46638/0 (socket says 172.21.15.97:46638) 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 -- 172.21.15.97:0/2727982040 learned_addr learned my addr 172.21.15.97:0/2727982040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 -- 172.21.15.97:0/2727982040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c015d590 msgr2=0x7fe0c00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c015d590 0x7fe0c00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 -- 172.21.15.97:0/2727982040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00cfae0 msgr2=0x7fe0c01f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00cfae0 0x7fe0c01f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.903+0000 7fe0c4a00640 1 -- 172.21.15.97:0/2727982040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8009400 con 0x7fe0c00d0de0 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.904+0000 7fe0c4a00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c0157930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d7d0 tx=0x7fe0a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.904+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004030 con 0x7fe0c00d0de0 2024-01-31T05:00:51.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.904+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a8004190 con 0x7fe0c00d0de0 2024-01-31T05:00:51.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.904+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c0158690 con 0x7fe0c00d0de0 2024-01-31T05:00:51.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.904+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004520 con 0x7fe0c00d0de0 2024-01-31T05:00:51.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.904+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c00c6de0 con 0x7fe0c00d0de0 2024-01-31T05:00:51.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.905+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00c6c30 con 0x7fe0c00d0de0 2024-01-31T05:00:51.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.906+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe0a800b6c0 con 0x7fe0c00d0de0 2024-01-31T05:00:51.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.907+0000 7fe0bd600640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe09806aed0 0x7fe09806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:51.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.907+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe0a8089a10 con 0x7fe0c00d0de0 2024-01-31T05:00:51.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.908+0000 7fe0c5400640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe09806aed0 0x7fe09806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:51.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.908+0000 7fe0c5400640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe09806aed0 0x7fe09806d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00012e0 tx=0x7fe0b0001650 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:51.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:51.914+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe0a8054680 con 0x7fe0c00d0de0 2024-01-31T05:00:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.150+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0c00c6f70 con 0x7fe0c00d0de0 2024-01-31T05:00:52.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.151+0000 7fe0bd600640 1 -- 172.21.15.97:0/2727982040 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fe0a8057ca0 con 0x7fe0c00d0de0 2024-01-31T05:00:52.151 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:52.151 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:00:51.075919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-01-31T05:00:52.151 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe09806aed0 msgr2=0x7fe09806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe09806aed0 0x7fe09806d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00012e0 tx=0x7fe0b0001650 comp rx=0 tx=0).stop 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 msgr2=0x7fe0c0157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c0157930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d7d0 tx=0x7fe0a800dc90 comp rx=0 tx=0).stop 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 shutdown_connections 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c015d590 0x7fe0c00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe09806aed0 0x7fe09806d390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0c00d0de0 0x7fe0c0157930 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 --2- 172.21.15.97:0/2727982040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c00cfae0 0x7fe0c01f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 >> 172.21.15.97:0/2727982040 conn(0x7fe0c009c560 msgr2=0x7fe0c006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:52.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 shutdown_connections 2024-01-31T05:00:52.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.154+0000 7fe0c7e00640 1 -- 172.21.15.97:0/2727982040 wait complete. 2024-01-31T05:00:52.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:52.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.368+0000 7f2537400640 1 Processor -- start 2024-01-31T05:00:52.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.369+0000 7f2537400640 1 -- start start 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2537400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f25300cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2537400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f25300ce430 0x7f253006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2537400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25300abcf0 con 0x7f253015d520 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2537400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253015d900 con 0x7f25300ce430 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2537400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25300ce8c0 con 0x7f25301616d0 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2534a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f253015d520 0x7f2530160fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43474/0 (socket says 172.21.15.97:43474) 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2534a00640 1 -- 172.21.15.97:0/4283132651 learned_addr learned my addr 172.21.15.97:0/4283132651 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:52.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.370+0000 7f2535400640 1 --2- 172.21.15.97:0/4283132651 >> v2:172.21.15.97:3300/0 conn(0x7f25300ce430 0x7f253006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f2527c00640 1 --2- 172.21.15.97:0/4283132651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f25300cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f252fe00640 1 -- 172.21.15.97:0/4283132651 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352774944 0 0) 0x7f25300abcf0 con 0x7f253015d520 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f252fe00640 1 -- 172.21.15.97:0/4283132651 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2510003520 con 0x7f253015d520 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f2535400640 1 -- 172.21.15.97:0/4283132651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 msgr2=0x7f25300cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f2535400640 1 --2- 172.21.15.97:0/4283132651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f25300cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f2535400640 1 -- 172.21.15.97:0/4283132651 >> v1:172.21.15.97:6789/0 conn(0x7f253015d520 legacy=0x7f2530160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.371+0000 7f2535400640 1 -- 172.21.15.97:0/4283132651 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253006ca40 con 0x7f25300ce430 2024-01-31T05:00:52.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.372+0000 7f2535400640 1 --2- 172.21.15.97:0/4283132651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25300ce430 0x7f253006ced0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2528009d90 tx=0x7f252802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7efb3dc5457a3b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.372+0000 7f252fe00640 1 -- 172.21.15.97:0/4283132651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252802cc50 con 0x7f25300ce430 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.372+0000 7f252fe00640 1 -- 172.21.15.97:0/4283132651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252802cdb0 con 0x7f25300ce430 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.372+0000 7f252fe00640 1 -- 172.21.15.97:0/4283132651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528034660 con 0x7f25300ce430 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 -- 172.21.15.97:0/4283132651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25300ce430 msgr2=0x7f253006ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 --2- 172.21.15.97:0/4283132651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25300ce430 0x7f253006ced0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2528009d90 tx=0x7f252802bea0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 -- 172.21.15.97:0/4283132651 shutdown_connections 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 --2- 172.21.15.97:0/4283132651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25300ce430 0x7f253006ced0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 --2- 172.21.15.97:0/4283132651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f25300cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 -- 172.21.15.97:0/4283132651 >> 172.21.15.97:0/4283132651 conn(0x7f25300b0a10 msgr2=0x7f25300b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 -- 172.21.15.97:0/4283132651 shutdown_connections 2024-01-31T05:00:52.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.373+0000 7f2537400640 1 -- 172.21.15.97:0/4283132651 wait complete. 2024-01-31T05:00:52.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.375+0000 7f2537400640 1 Processor -- start 2024-01-31T05:00:52.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.375+0000 7f2537400640 1 -- start start 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2537400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25300ce430 0x7f2530159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2537400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 0x7f253015bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2537400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f253014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2537400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253015a370 con 0x7f253015d520 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2537400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253009bcc0 con 0x7f25300ce430 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2537400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253009bfa0 con 0x7f25301616d0 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 0x7f253015bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 0x7f253015bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46658/0 (socket says 172.21.15.97:46658) 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 -- 172.21.15.97:0/3689547062 learned_addr learned my addr 172.21.15.97:0/3689547062 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 -- 172.21.15.97:0/3689547062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 msgr2=0x7f253014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f253014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 -- 172.21.15.97:0/3689547062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25300ce430 msgr2=0x7f2530159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25300ce430 0x7f2530159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.376+0000 7f2527c00640 1 -- 172.21.15.97:0/3689547062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2528009a70 con 0x7f253015d520 2024-01-31T05:00:52.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.377+0000 7f2527c00640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 0x7f253015bad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f251800d5f0 tx=0x7f251800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:52.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.377+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180125f0 con 0x7f253015d520 2024-01-31T05:00:52.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.377+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518012750 con 0x7f253015d520 2024-01-31T05:00:52.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.377+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518012b00 con 0x7f253015d520 2024-01-31T05:00:52.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.377+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253009c280 con 0x7f253015d520 2024-01-31T05:00:52.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.377+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253015c010 con 0x7f253015d520 2024-01-31T05:00:52.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.378+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253015c1a0 con 0x7f253015d520 2024-01-31T05:00:52.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.379+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2518012c60 con 0x7f253015d520 2024-01-31T05:00:52.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.380+0000 7f252d600640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f250806af00 0x7f250806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.380+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2518089dd0 con 0x7f253015d520 2024-01-31T05:00:52.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.380+0000 7f2534a00640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f250806af00 0x7f250806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.381+0000 7f2534a00640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f250806af00 0x7f250806d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2520000f60 tx=0x7f25200078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:52.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.384+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f25180549b0 con 0x7f253015d520 2024-01-31T05:00:52.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.577+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25300cc590 con 0x7f253015d520 2024-01-31T05:00:52.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.580+0000 7f252d600640 1 -- 172.21.15.97:0/3689547062 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f251808b090 con 0x7f253015d520 2024-01-31T05:00:52.580 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:52.580 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.582+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f250806af00 msgr2=0x7f250806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.582+0000 7f2537400640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f250806af00 0x7f250806d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2520000f60 tx=0x7f25200078f0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.582+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 msgr2=0x7f253015bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.582+0000 7f2537400640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 0x7f253015bad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f251800d5f0 tx=0x7f251800dab0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 shutdown_connections 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25301616d0 0x7f253014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f250806af00 0x7f250806d3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f253015d520 0x7f253015bad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 --2- 172.21.15.97:0/3689547062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25300ce430 0x7f2530159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 >> 172.21.15.97:0/3689547062 conn(0x7f25300b0a10 msgr2=0x7f25300cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 shutdown_connections 2024-01-31T05:00:52.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.583+0000 7f2537400640 1 -- 172.21.15.97:0/3689547062 wait complete. 2024-01-31T05:00:52.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:52.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.802+0000 7f492c000640 1 Processor -- start 2024-01-31T05:00:52.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.802+0000 7f492c000640 1 -- start start 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492c000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f492415b400 0x7f492406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492c000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f492406db70 0x7f49240ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492c000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492406d1a0 con 0x7f49240cb810 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492c000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49240cc960 con 0x7f492406db70 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492c000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492406c600 con 0x7f492415b400 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492a000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f492406db70 0x7f49240ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492a000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f492406db70 0x7f49240ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46664/0 (socket says 172.21.15.97:46664) 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f492a000640 1 -- 172.21.15.97:0/1117054672 learned_addr learned my addr 172.21.15.97:0/1117054672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:52.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.804+0000 7f4928c00640 1 --2- 172.21.15.97:0/1117054672 >> v2:172.21.15.188:3300/0 conn(0x7f492415b400 0x7f492406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.805+0000 7f492a000640 1 -- 172.21.15.97:0/1117054672 >> v2:172.21.15.188:3300/0 conn(0x7f492415b400 msgr2=0x7f492406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:52.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.805+0000 7f492a000640 1 --2- 172.21.15.97:0/1117054672 >> v2:172.21.15.188:3300/0 conn(0x7f492415b400 0x7f492406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.805+0000 7f492a000640 1 -- 172.21.15.97:0/1117054672 >> v1:172.21.15.188:6789/0 conn(0x7f49240cb810 legacy=0x7f49240cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.805+0000 7f492a000640 1 -- 172.21.15.97:0/1117054672 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492406c870 con 0x7f492406db70 2024-01-31T05:00:52.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.806+0000 7f492a000640 1 --2- 172.21.15.97:0/1117054672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f492406db70 0x7f49240ccd00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49240ae830 tx=0x7f491c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207f99cdb0974289 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:52.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.806+0000 7f4923e00640 1 -- 172.21.15.97:0/1117054672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c02cb20 con 0x7f492406db70 2024-01-31T05:00:52.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.806+0000 7f4923e00640 1 -- 172.21.15.97:0/1117054672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c02cc80 con 0x7f492406db70 2024-01-31T05:00:52.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.806+0000 7f4923e00640 1 -- 172.21.15.97:0/1117054672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0343f0 con 0x7f492406db70 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 -- 172.21.15.97:0/1117054672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f492406db70 msgr2=0x7f49240ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 --2- 172.21.15.97:0/1117054672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f492406db70 0x7f49240ccd00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49240ae830 tx=0x7f491c02bf80 comp rx=0 tx=0).stop 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 -- 172.21.15.97:0/1117054672 shutdown_connections 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 --2- 172.21.15.97:0/1117054672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f492406db70 0x7f49240ccd00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 --2- 172.21.15.97:0/1117054672 >> v2:172.21.15.188:3300/0 conn(0x7f492415b400 0x7f492406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 -- 172.21.15.97:0/1117054672 >> 172.21.15.97:0/1117054672 conn(0x7f49240b0a10 msgr2=0x7f49240b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 -- 172.21.15.97:0/1117054672 shutdown_connections 2024-01-31T05:00:52.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.807+0000 7f492c000640 1 -- 172.21.15.97:0/1117054672 wait complete. 2024-01-31T05:00:52.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.809+0000 7f492c000640 1 Processor -- start 2024-01-31T05:00:52.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.809+0000 7f492c000640 1 -- start start 2024-01-31T05:00:52.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f492c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f492406db70 0x7f4924152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f492c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 0x7f492409a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f492c000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f492415b400 0x7f492409aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f492c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492415c050 con 0x7f49240cb810 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f492c000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49241f1a00 con 0x7f492415b400 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f492c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49241f1720 con 0x7f492406db70 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f4928c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 0x7f492409a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f4928c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 0x7f492409a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46670/0 (socket says 172.21.15.97:46670) 2024-01-31T05:00:52.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.810+0000 7f4928c00640 1 -- 172.21.15.97:0/2269439052 learned_addr learned my addr 172.21.15.97:0/2269439052 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:52.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f492a000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f492415b400 0x7f492409aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4929600640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f492406db70 0x7f4924152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4928c00640 1 -- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f492406db70 msgr2=0x7f4924152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4928c00640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f492406db70 0x7f4924152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4928c00640 1 -- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f492415b400 msgr2=0x7f492409aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4928c00640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f492415b400 0x7f492409aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4928c00640 1 -- 172.21.15.97:0/2269439052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c009b30 con 0x7f49240cb810 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4928c00640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 0x7f492409a4e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e870 tx=0x7f490c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0094f0 con 0x7f49240cb810 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c00c940 con 0x7f49240cb810 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492415b990 con 0x7f49240cb810 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c00ccd0 con 0x7f49240cb810 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.811+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4924153120 con 0x7f49240cb810 2024-01-31T05:00:52.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.812+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492415c190 con 0x7f49240cb810 2024-01-31T05:00:52.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.813+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f490c013470 con 0x7f49240cb810 2024-01-31T05:00:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.815+0000 7f4921600640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f48fc06af00 0x7f48fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.815+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f490c0899f0 con 0x7f49240cb810 2024-01-31T05:00:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.815+0000 7f4929600640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f48fc06af00 0x7f48fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:52.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.815+0000 7f4929600640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f48fc06af00 0x7f48fc06d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f492415fb90 tx=0x7f4914009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:52.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:52.819+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f492415c190 con 0x7f49240cb810 2024-01-31T05:00:53.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.012+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f492415c190 con 0x7f49240cb810 2024-01-31T05:00:53.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.015+0000 7f4921600640 1 -- 172.21.15.97:0/2269439052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f492415c190 con 0x7f49240cb810 2024-01-31T05:00:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:53.015 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.017+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f48fc06af00 msgr2=0x7f48fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.017+0000 7f492c000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f48fc06af00 0x7f48fc06d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f492415fb90 tx=0x7f4914009040 comp rx=0 tx=0).stop 2024-01-31T05:00:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.017+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 msgr2=0x7f492409a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.017+0000 7f492c000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 0x7f492409a4e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f490c00e870 tx=0x7f490c00ed30 comp rx=0 tx=0).stop 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 shutdown_connections 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f492415b400 0x7f492409aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f48fc06af00 0x7f48fc06d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49240cb810 0x7f492409a4e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 --2- 172.21.15.97:0/2269439052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f492406db70 0x7f4924152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 >> 172.21.15.97:0/2269439052 conn(0x7f49240b0a10 msgr2=0x7f49240ac490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 shutdown_connections 2024-01-31T05:00:53.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.018+0000 7f492c000640 1 -- 172.21.15.97:0/2269439052 wait complete. 2024-01-31T05:00:53.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:00:53.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.254+0000 7f2fdc200640 1 Processor -- start 2024-01-31T05:00:53.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.254+0000 7f2fdc200640 1 -- start start 2024-01-31T05:00:53.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fdc200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2fd415d520 0x7f2fd40cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fdc200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fd40ce430 0x7f2fd4160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fdc200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd40abcf0 con 0x7f2fd4161500 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fdc200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd415d900 con 0x7f2fd40ce430 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fdc200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd4161970 con 0x7f2fd415d520 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fd8e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fd40ce430 0x7f2fd4160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fd8e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fd40ce430 0x7f2fd4160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46672/0 (socket says 172.21.15.97:46672) 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fd8e00640 1 -- 172.21.15.97:0/3692250394 learned_addr learned my addr 172.21.15.97:0/3692250394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:53.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.256+0000 7f2fd9800640 1 --2- 172.21.15.97:0/3692250394 >> v2:172.21.15.188:3300/0 conn(0x7f2fd415d520 0x7f2fd40cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.257+0000 7f2fd8e00640 1 -- 172.21.15.97:0/3692250394 >> v2:172.21.15.188:3300/0 conn(0x7f2fd415d520 msgr2=0x7f2fd40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:53.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.257+0000 7f2fd8e00640 1 --2- 172.21.15.97:0/3692250394 >> v2:172.21.15.188:3300/0 conn(0x7f2fd415d520 0x7f2fd40cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.257+0000 7f2fd8e00640 1 -- 172.21.15.97:0/3692250394 >> v1:172.21.15.97:6789/0 conn(0x7f2fd4161500 legacy=0x7f2fd40b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.257+0000 7f2fd8e00640 1 -- 172.21.15.97:0/3692250394 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd406ca40 con 0x7f2fd40ce430 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.257+0000 7f2fd9800640 1 --2- 172.21.15.97:0/3692250394 >> v2:172.21.15.188:3300/0 conn(0x7f2fd415d520 0x7f2fd40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.258+0000 7f2fd8e00640 1 --2- 172.21.15.97:0/3692250394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd4160fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4072ef0 tx=0x7f2fbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fd6f5f0140881d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.258+0000 7f2fd3e00640 1 -- 172.21.15.97:0/3692250394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc030860 con 0x7f2fd40ce430 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.258+0000 7f2fd3e00640 1 -- 172.21.15.97:0/3692250394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc0309c0 con 0x7f2fd40ce430 2024-01-31T05:00:53.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.258+0000 7f2fd3e00640 1 -- 172.21.15.97:0/3692250394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc030d50 con 0x7f2fd40ce430 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 -- 172.21.15.97:0/3692250394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 msgr2=0x7f2fd4160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 --2- 172.21.15.97:0/3692250394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd4160fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4072ef0 tx=0x7f2fbc02b9a0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 -- 172.21.15.97:0/3692250394 shutdown_connections 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 --2- 172.21.15.97:0/3692250394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd4160fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 --2- 172.21.15.97:0/3692250394 >> v2:172.21.15.188:3300/0 conn(0x7f2fd415d520 0x7f2fd40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 -- 172.21.15.97:0/3692250394 >> 172.21.15.97:0/3692250394 conn(0x7f2fd40b0a10 msgr2=0x7f2fd40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 -- 172.21.15.97:0/3692250394 shutdown_connections 2024-01-31T05:00:53.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.259+0000 7f2fdc200640 1 -- 172.21.15.97:0/3692250394 wait complete. 2024-01-31T05:00:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.260+0000 7f2fdc200640 1 Processor -- start 2024-01-31T05:00:53.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.261+0000 7f2fdc200640 1 -- start start 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fdc200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd41478a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fdc200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd415d520 0x7f2fd415da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fdc200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd4161500 0x7f2fd415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fdc200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd409add0 con 0x7f2fd40ce430 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fdc200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4147de0 con 0x7f2fd4161500 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fdc200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd415dfc0 con 0x7f2fd415d520 2024-01-31T05:00:53.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd41478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd41478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46684/0 (socket says 172.21.15.97:46684) 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 -- 172.21.15.97:0/2205584163 learned_addr learned my addr 172.21.15.97:0/2205584163 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 -- 172.21.15.97:0/2205584163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd415d520 msgr2=0x7f2fd415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd415d520 0x7f2fd415da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 -- 172.21.15.97:0/2205584163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd4161500 msgr2=0x7f2fd415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd4161500 0x7f2fd415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 -- 172.21.15.97:0/2205584163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0094d0 con 0x7f2fd40ce430 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.262+0000 7f2fd9800640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd41478a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e810 tx=0x7f2fc400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.263+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004030 con 0x7f2fd40ce430 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.263+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4004190 con 0x7f2fd40ce430 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.263+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd409a4a0 con 0x7f2fd40ce430 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.263+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4004520 con 0x7f2fd40ce430 2024-01-31T05:00:53.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.263+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd409aa60 con 0x7f2fd40ce430 2024-01-31T05:00:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.264+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd40677c0 con 0x7f2fd40ce430 2024-01-31T05:00:53.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.265+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2fc400c6d0 con 0x7f2fd40ce430 2024-01-31T05:00:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.266+0000 7f2fd1600640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fac06af00 0x7f2fac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.266+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2fc40896a0 con 0x7f2fd40ce430 2024-01-31T05:00:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.266+0000 7f2fd8e00640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fac06af00 0x7f2fac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.267+0000 7f2fd8e00640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fac06af00 0x7f2fac06d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc036000 tx=0x7f2fbc03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:53.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.270+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2fc40541e0 con 0x7f2fd40ce430 2024-01-31T05:00:53.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.502+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd40cc590 con 0x7f2fd40ce430 2024-01-31T05:00:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.503+0000 7f2fd1600640 1 -- 172.21.15.97:0/2205584163 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f2fc4057800 con 0x7f2fd40ce430 2024-01-31T05:00:53.502 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:53.502 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:00:51.075919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-01-31T05:00:53.502 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fac06af00 msgr2=0x7f2fac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fac06af00 0x7f2fac06d3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc036000 tx=0x7f2fbc03b040 comp rx=0 tx=0).stop 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 msgr2=0x7f2fd41478a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd41478a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400e810 tx=0x7f2fc400ecd0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 shutdown_connections 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd4161500 0x7f2fd415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd415d520 0x7f2fd415da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fac06af00 0x7f2fac06d3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 --2- 172.21.15.97:0/2205584163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd40ce430 0x7f2fd41478a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.505+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 >> 172.21.15.97:0/2205584163 conn(0x7f2fd40b0a10 msgr2=0x7f2fd40b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.506+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 shutdown_connections 2024-01-31T05:00:53.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.506+0000 7f2fdc200640 1 -- 172.21.15.97:0/2205584163 wait complete. 2024-01-31T05:00:53.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:00:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.727+0000 7f047e400640 1 Processor -- start 2024-01-31T05:00:53.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.728+0000 7f047e400640 1 -- start start 2024-01-31T05:00:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.729+0000 7f047e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 0x7f04780d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.729+0000 7f047e400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047815ec50 con 0x7f04780cfae0 2024-01-31T05:00:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.729+0000 7f047e400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04780cded0 con 0x7f047815d590 2024-01-31T05:00:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.729+0000 7f047e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047815d970 con 0x7f04780cff20 2024-01-31T05:00:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.729+0000 7f0476a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f04780cfae0 0x7f04780d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43498/0 (socket says 172.21.15.97:43498) 2024-01-31T05:00:53.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.729+0000 7f0476a00640 1 -- 172.21.15.97:0/2273304791 learned_addr learned my addr 172.21.15.97:0/2273304791 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0477e00640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 0x7f04780d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384035579 0 0) 0x7f047815ec50 con 0x7f04780cfae0 2024-01-31T05:00:53.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0458003520 con 0x7f04780cfae0 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719380927 0 0) 0x7f0458003520 con 0x7f04780cfae0 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 msgr2=0x7f04780d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 0x7f04780d14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 >> v1:172.21.15.188:6789/0 conn(0x7f047815d590 legacy=0x7f047815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04780cf210 con 0x7f04780cfae0 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f0460001670 con 0x7f04780cfae0 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 >> v1:172.21.15.97:6789/0 conn(0x7f04780cfae0 legacy=0x7f04780d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04580039a0 0x7f0458003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 0x7f04580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047815d970 con 0x7f04580039a0 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045800bac0 con 0x7f0458004d00 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.730+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045800bc80 con 0x7f04780cff20 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477400640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04580039a0 0x7f0458003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477400640 1 -- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 msgr2=0x7f04580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477e00640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0476a00640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 0x7f04580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477400640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04780cff20 0x7f04580047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477400640 1 -- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0458004d00 msgr2=0x7f0458005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477400640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0458004d00 0x7f0458005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.731+0000 7f0477400640 1 -- 172.21.15.97:0/2273304791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458003520 con 0x7f04580039a0 2024-01-31T05:00:53.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.732+0000 7f0477400640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04580039a0 0x7f0458003dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0468001720 tx=0x7f04580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11df911511de030e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.732+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04680052d0 con 0x7f04580039a0 2024-01-31T05:00:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.732+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046802e050 con 0x7f04580039a0 2024-01-31T05:00:53.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.732+0000 7f0476000640 1 -- 172.21.15.97:0/2273304791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04680035f0 con 0x7f04580039a0 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f047e400640 1 -- 172.21.15.97:0/2273304791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04580039a0 msgr2=0x7f0458003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f047e400640 1 --2- 172.21.15.97:0/2273304791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04580039a0 0x7f0458003dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0468001720 tx=0x7f04580057f0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f0477e00640 1 -- 172.21.15.97:0/2273304791 reap_dead start 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f047e400640 1 -- 172.21.15.97:0/2273304791 shutdown_connections 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f047e400640 1 -- 172.21.15.97:0/2273304791 >> 172.21.15.97:0/2273304791 conn(0x7f047809c560 msgr2=0x7f04780d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f047e400640 1 -- 172.21.15.97:0/2273304791 shutdown_connections 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.733+0000 7f047e400640 1 -- 172.21.15.97:0/2273304791 wait complete. 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.735+0000 7f047e400640 1 Processor -- start 2024-01-31T05:00:53.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.735+0000 7f047e400640 1 -- start start 2024-01-31T05:00:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f047e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04580039a0 0x7f0478158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f047e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 0x7f04780cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f047e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd2d0 0x7f04780cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f047e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0478158cb0 con 0x7f04781f4070 2024-01-31T05:00:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f047e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04781f5030 con 0x7f04780cd2d0 2024-01-31T05:00:53.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f047e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04781f5310 con 0x7f04580039a0 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0476a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 0x7f04780cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0476a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 0x7f04780cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46704/0 (socket says 172.21.15.97:46704) 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0476a00640 1 -- 172.21.15.97:0/3687203318 learned_addr learned my addr 172.21.15.97:0/3687203318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0477e00640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd2d0 0x7f04780cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0476a00640 1 -- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04580039a0 msgr2=0x7f0478158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0477400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04580039a0 0x7f0478158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.736+0000 7f0476a00640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04580039a0 0x7f0478158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f0476a00640 1 -- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd2d0 msgr2=0x7f04780cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f0476a00640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd2d0 0x7f04780cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f0476a00640 1 -- 172.21.15.97:0/3687203318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04781f55f0 con 0x7f04781f4070 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f0477e00640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd2d0 0x7f04780cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f0476a00640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 0x7f04780cb0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f046000d7a0 tx=0x7f046000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600074e0 con 0x7f04781f4070 2024-01-31T05:00:53.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0460007640 con 0x7f04781f4070 2024-01-31T05:00:53.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600049e0 con 0x7f04781f4070 2024-01-31T05:00:53.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04781f58d0 con 0x7f04781f4070 2024-01-31T05:00:53.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.737+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04781f60d0 con 0x7f04781f4070 2024-01-31T05:00:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.738+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04781f5df0 con 0x7f04781f4070 2024-01-31T05:00:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.739+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f0460004b40 con 0x7f04781f4070 2024-01-31T05:00:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.740+0000 7f046f400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f045006af00 0x7f045006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.740+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f04600894f0 con 0x7f04781f4070 2024-01-31T05:00:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.740+0000 7f0477400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f045006af00 0x7f045006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:53.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.741+0000 7f0477400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f045006af00 0x7f045006d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0468033000 tx=0x7f0468009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:53.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.744+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f04781f5df0 con 0x7f04781f4070 2024-01-31T05:00:53.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.939+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04781f5df0 con 0x7f04781f4070 2024-01-31T05:00:53.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.941+0000 7f046f400640 1 -- 172.21.15.97:0/3687203318 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f04781f5df0 con 0x7f04781f4070 2024-01-31T05:00:53.941 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:53.941 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:00:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.943+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f045006af00 msgr2=0x7f045006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.943+0000 7f047e400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f045006af00 0x7f045006d3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0468033000 tx=0x7f0468009030 comp rx=0 tx=0).stop 2024-01-31T05:00:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.943+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 msgr2=0x7f04780cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:53.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.943+0000 7f047e400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 0x7f04780cb0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f046000d7a0 tx=0x7f046000dc60 comp rx=0 tx=0).stop 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 shutdown_connections 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04780cd2d0 0x7f04780cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f045006af00 0x7f045006d3c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04781f4070 0x7f04780cb0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 --2- 172.21.15.97:0/3687203318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04580039a0 0x7f0478158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 >> 172.21.15.97:0/3687203318 conn(0x7f047809c560 msgr2=0x7f04780b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 shutdown_connections 2024-01-31T05:00:53.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:53.944+0000 7f047e400640 1 -- 172.21.15.97:0/3687203318 wait complete. 2024-01-31T05:00:54.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-31T05:00:46.002409+0000', 'modified': '2024-01-31T05:00:51.075919+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4257}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4257': {'gid': 4257, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/294460102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 294460102}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 294460102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-01-31T05:00:54.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-31T05:00:54.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-31T05:00:54.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.585+0000 7f5f25c00640 1 Processor -- start 2024-01-31T05:00:54.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.585+0000 7f5f25c00640 1 -- start start 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f25c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5f20157a90 0x7f5f2006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f25c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2015e7b0 con 0x7f5f201576b0 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f25c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2015e4d0 con 0x7f5f2006c640 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f25c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2006cad0 con 0x7f5f20157a90 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f1fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5f201576b0 0x7f5f20070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43510/0 (socket says 172.21.15.97:43510) 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f1fe00640 1 -- 172.21.15.97:0/2371170187 learned_addr learned my addr 172.21.15.97:0/2371170187 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:54.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.587+0000 7f5f1f400640 1 --2- 172.21.15.97:0/2371170187 >> v2:172.21.15.97:3300/0 conn(0x7f5f20157a90 0x7f5f2006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.588+0000 7f5f1f400640 1 -- 172.21.15.97:0/2371170187 >> v1:172.21.15.188:6789/0 conn(0x7f5f2006c640 legacy=0x7f5f200b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.588+0000 7f5f1f400640 1 -- 172.21.15.97:0/2371170187 >> v1:172.21.15.97:6789/0 conn(0x7f5f201576b0 legacy=0x7f5f20070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.588+0000 7f5f1f400640 1 -- 172.21.15.97:0/2371170187 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20070420 con 0x7f5f20157a90 2024-01-31T05:00:54.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.589+0000 7f5f1f400640 1 --2- 172.21.15.97:0/2371170187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20157a90 0x7f5f2006c100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f080097a0 tx=0x7f5f0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e569fd758d5a21ec server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f1ea00640 1 -- 172.21.15.97:0/2371170187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08030610 con 0x7f5f20157a90 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f1ea00640 1 -- 172.21.15.97:0/2371170187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08030770 con 0x7f5f20157a90 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f1ea00640 1 -- 172.21.15.97:0/2371170187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08030b00 con 0x7f5f20157a90 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 -- 172.21.15.97:0/2371170187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20157a90 msgr2=0x7f5f2006c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 --2- 172.21.15.97:0/2371170187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20157a90 0x7f5f2006c100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f080097a0 tx=0x7f5f0802b960 comp rx=0 tx=0).stop 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 -- 172.21.15.97:0/2371170187 shutdown_connections 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 --2- 172.21.15.97:0/2371170187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f20157a90 0x7f5f2006c100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 -- 172.21.15.97:0/2371170187 >> 172.21.15.97:0/2371170187 conn(0x7f5f2006ad40 msgr2=0x7f5f200b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 -- 172.21.15.97:0/2371170187 shutdown_connections 2024-01-31T05:00:54.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.590+0000 7f5f25c00640 1 -- 172.21.15.97:0/2371170187 wait complete. 2024-01-31T05:00:54.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.592+0000 7f5f25c00640 1 Processor -- start 2024-01-31T05:00:54.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.592+0000 7f5f25c00640 1 -- start start 2024-01-31T05:00:54.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.593+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2006c640 0x7f5f200af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.593+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 0x7f5f200adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.593+0000 7f5f25c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f20157a90 0x7f5f2014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.593+0000 7f5f25c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200afc90 con 0x7f5f201576b0 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.593+0000 7f5f25c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200ae450 con 0x7f5f20157a90 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.593+0000 7f5f25c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200ae6b0 con 0x7f5f2006c640 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 0x7f5f200adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 0x7f5f200adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46708/0 (socket says 172.21.15.97:46708) 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 -- 172.21.15.97:0/1226941093 learned_addr learned my addr 172.21.15.97:0/1226941093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 -- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2006c640 msgr2=0x7f5f200af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2006c640 0x7f5f200af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f25200640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f20157a90 0x7f5f2014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2006c640 0x7f5f200af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 -- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f20157a90 msgr2=0x7f5f2014e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f20157a90 0x7f5f2014e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 -- 172.21.15.97:0/1226941093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08009400 con 0x7f5f201576b0 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.594+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 0x7f5f200adf10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f0802bf70 tx=0x7f5f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.595+0000 7f5f17e00640 1 -- 172.21.15.97:0/1226941093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0803e070 con 0x7f5f201576b0 2024-01-31T05:00:54.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.595+0000 7f5f17e00640 1 -- 172.21.15.97:0/1226941093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08039a10 con 0x7f5f201576b0 2024-01-31T05:00:54.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.595+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f200ae990 con 0x7f5f201576b0 2024-01-31T05:00:54.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.595+0000 7f5f17e00640 1 -- 172.21.15.97:0/1226941093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08039da0 con 0x7f5f201576b0 2024-01-31T05:00:54.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.595+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200aef50 con 0x7f5f201576b0 2024-01-31T05:00:54.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.597+0000 7f5f17e00640 1 -- 172.21.15.97:0/1226941093 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5f0804b400 con 0x7f5f201576b0 2024-01-31T05:00:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.598+0000 7f5f17e00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ef806aed0 0x7f5ef806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.598+0000 7f5f17e00640 1 -- 172.21.15.97:0/1226941093 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f080abe60 con 0x7f5f201576b0 2024-01-31T05:00:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.598+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ef806aed0 0x7f5ef806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.599+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ef806aed0 0x7f5ef806d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f10005880 tx=0x7f5f10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.599+0000 7f5f2a1bd740 1 Processor -- start 2024-01-31T05:00:54.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.599+0000 7f5f2a1bd740 1 -- start start 2024-01-31T05:00:54.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28e1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 0x558ff28c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28bb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2805f50 con 0x558ff28bfe70 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2803640 con 0x558ff28c2bb0 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff27fef30 con 0x558ff28c34d0 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f1f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 0x558ff28c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f1f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 0x558ff28c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46722/0 (socket says 172.21.15.97:46722) 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.600+0000 7f5f1f400640 1 -- 172.21.15.97:0/2362449506 learned_addr learned my addr 172.21.15.97:0/2362449506 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f25200640 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28bb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1f400640 1 -- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 msgr2=0x558ff28bb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1f400640 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28bb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1f400640 1 -- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 msgr2=0x558ff28e1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1f400640 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28e1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1f400640 1 -- 172.21.15.97:0/2362449506 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ff27c12d0 con 0x558ff28bfe70 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28e1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f1f400640 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 0x558ff28c2f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5f080824d0 tx=0x7f5f0807a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9defe6c76a75ca4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f14c00640 1 -- 172.21.15.97:0/2362449506 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08082a00 con 0x558ff28bfe70 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f14c00640 1 -- 172.21.15.97:0/2362449506 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08078aa0 con 0x558ff28bfe70 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.601+0000 7f5f14c00640 1 -- 172.21.15.97:0/2362449506 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08082570 con 0x558ff28bfe70 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/2362449506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 msgr2=0x558ff28c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 0x558ff28c2f90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5f080824d0 tx=0x7f5f0807a370 comp rx=0 tx=0).stop 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/2362449506 shutdown_connections 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28bb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28bfe70 0x558ff28c2f90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/2362449506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28e1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/2362449506 >> 172.21.15.97:0/2362449506 conn(0x558ff2894e40 msgr2=0x558ff28e0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/2362449506 shutdown_connections 2024-01-31T05:00:54.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.602+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/2362449506 wait complete. 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.603+0000 7f5f2a1bd740 1 -- start start 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.603+0000 7f5f2a1bd740 1 Processor -- start 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 0x558ff28dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff27b86b0 con 0x558ff28c2bb0 2024-01-31T05:00:54.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff27b8350 con 0x558ff28b80f0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff2560f20 con 0x558ff28c34d0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f2a1bd740 10 client.0 _ll_register_callbacks cb 0x558ff27ba9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f1fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 0x558ff28dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f1fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 0x558ff28dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46730/0 (socket says 172.21.15.97:46730) 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.604+0000 7f5f1fe00640 1 -- 172.21.15.97:0/1276837658 learned_addr learned my addr 172.21.15.97:0/1276837658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f25200640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1fe00640 1 -- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 msgr2=0x558ff28e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1fe00640 1 -- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 msgr2=0x558ff28c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1fe00640 1 -- 172.21.15.97:0/1276837658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f200aef50 con 0x558ff28c2bb0 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:54.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 0x558ff28dfcd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f10007380 tx=0x7f5f10000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f50eba2f5faf514 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f0d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f0d600640 1 -- 172.21.15.97:0/1276837658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10021630 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f0d600640 1 -- 172.21.15.97:0/1276837658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f10020d70 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.605+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ff2826e60 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f0d600640 1 -- 172.21.15.97:0/1276837658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f10021790 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 10 client.4419 fetch_fsmap learned FSMap version 6 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x558ff27fd5b0 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f0d600640 1 -- 172.21.15.97:0/1276837658 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ff2826e60 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f0d600640 1 -- 172.21.15.97:0/1276837658 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5f100218f0 con 0x558ff28c2bb0 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 10 client.4419 fetch_fsmap finished waiting for FSMap version 6 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 10 client.4419 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 20 client.4419 populate_metadata read hostname 'smithi097' 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d8b90 0x558ff28d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 4 client.4419 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:00:54.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 --> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ff27d0c40 con 0x558ff28d8b90 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.606+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d8b90 0x558ff28d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.607+0000 7f5f0d600640 10 client.4419 ms_handle_connect on v2:172.21.15.97:6835/294460102 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.607+0000 7f5f1f400640 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d8b90 0x558ff28d8f70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78574f5776d8797a server_cookie=61380833c6676c0 in_seq=0 out_seq=0 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f0d600640 1 -- 172.21.15.97:0/1276837658 <== mds.0 v2:172.21.15.97:6835/294460102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ff27d0c40 con 0x558ff28d8b90 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f0d600640 10 client.4419 handle_command_reply: tid=0 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 client.4419 shutdown 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 msgr2=0x558ff28dfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 0x558ff28dfcd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f10007380 tx=0x7f5f10000ed0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 shutdown_connections 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d8b90 0x558ff28d8f70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c34d0 0x558ff28e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/1276837658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c2bb0 0x558ff28dfcd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f0d600640 0 client.4419 ms_handle_reset on v2:172.21.15.97:6835/294460102 2024-01-31T05:00:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 >> 172.21.15.97:0/1276837658 conn(0x558ff28bfe70 msgr2=0x558ff28b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 shutdown_connections 2024-01-31T05:00:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/1276837658 wait complete. 2024-01-31T05:00:54.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.608+0000 7f5f2a1bd740 20 client.4419 trim_cache size 0 max 16384 2024-01-31T05:00:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.625+0000 7f5f2a1bd740 1 Processor -- start 2024-01-31T05:00:54.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.625+0000 7f5f2a1bd740 1 -- start start 2024-01-31T05:00:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c2bb0 0x558ff2897ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff2898010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff27fe0d0 con 0x558ff28c34d0 2024-01-31T05:00:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff28b7ae0 con 0x558ff28b80f0 2024-01-31T05:00:54.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff28db090 con 0x558ff28c2bb0 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f25200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff2898010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f25200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff2898010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46740/0 (socket says 172.21.15.97:46740) 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.626+0000 7f5f25200640 1 -- 172.21.15.97:0/827569633 learned_addr learned my addr 172.21.15.97:0/827569633 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f1f400640 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c2bb0 0x558ff2897ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f25200640 1 -- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c2bb0 msgr2=0x558ff2897ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f25200640 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c2bb0 0x558ff2897ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f25200640 1 -- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 msgr2=0x558ff28bf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f25200640 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28bf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f25200640 1 -- 172.21.15.97:0/827569633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ff279a1e0 con 0x558ff28c34d0 2024-01-31T05:00:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f25200640 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff2898010 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f180012e0 tx=0x7f5f18001eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dd6dcc4d321b1d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f0fe00640 1 -- 172.21.15.97:0/827569633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18002100 con 0x558ff28c34d0 2024-01-31T05:00:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f0fe00640 1 -- 172.21.15.97:0/827569633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800d040 con 0x558ff28c34d0 2024-01-31T05:00:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.627+0000 7f5f0fe00640 1 -- 172.21.15.97:0/827569633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180107c0 con 0x558ff28c34d0 2024-01-31T05:00:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/827569633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 msgr2=0x558ff2898010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff2898010 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f180012e0 tx=0x7f5f18001eb0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/827569633 shutdown_connections 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff2898010 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28c2bb0 0x558ff2897ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/827569633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28b80f0 0x558ff28bf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/827569633 >> 172.21.15.97:0/827569633 conn(0x558ff28bfe70 msgr2=0x558ff28d9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/827569633 shutdown_connections 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.628+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/827569633 wait complete. 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.629+0000 7f5f2a1bd740 1 -- start start 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.629+0000 7f5f2a1bd740 1 Processor -- start 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28b80f0 0x558ff28ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff28df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff2948470 con 0x558ff28c34d0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ff28bac60 con 0x558ff28c2bb0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ff28de5c0 con 0x558ff28b80f0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f2a1bd740 10 client.0 _ll_register_callbacks cb 0x7f5f10001420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f25200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff28df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f25200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff28df0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46748/0 (socket says 172.21.15.97:46748) 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.630+0000 7f5f25200640 1 -- 172.21.15.97:0/3443094673 learned_addr learned my addr 172.21.15.97:0/3443094673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f1f400640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28b80f0 0x558ff28ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f25200640 1 -- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28b80f0 msgr2=0x558ff28ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f25200640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28b80f0 0x558ff28ba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f25200640 1 -- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 msgr2=0x558ff28debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f25200640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28debb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f25200640 1 -- 172.21.15.97:0/3443094673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2015e4d0 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f25200640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff28df0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5f18016040 tx=0x7f5f18001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6caafe94e9b58dc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f0d600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f0d600640 1 -- 172.21.15.97:0/3443094673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18002100 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f0d600640 1 -- 172.21.15.97:0/3443094673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5f1800c050 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.631+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f08078aa0 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f0d600640 1 -- 172.21.15.97:0/3443094673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f18007dd0 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 10 client.4425 fetch_fsmap learned FSMap version 6 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x558ff27bfb30 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f0d600640 1 -- 172.21.15.97:0/3443094673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08078aa0 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f0d600640 1 -- 172.21.15.97:0/3443094673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5f18012040 con 0x558ff28c34d0 2024-01-31T05:00:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 10 client.4425 fetch_fsmap finished waiting for FSMap version 6 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 10 client.4425 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 20 client.4425 populate_metadata read hostname 'smithi097' 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d7340 0x558ff2946ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 4 client.4425 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 --> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558ff27f7dc0 con 0x558ff28d7340 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.632+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d7340 0x558ff2946ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f1fe00640 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d7340 0x558ff2946ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fead1eb2b4c4501d server_cookie=337821eeda17d42e in_seq=0 out_seq=0 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f0d600640 10 client.4425 ms_handle_connect on v2:172.21.15.97:6835/294460102 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f0d600640 1 -- 172.21.15.97:0/3443094673 <== mds.0 v2:172.21.15.97:6835/294460102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x558ff27f7dc0 con 0x558ff28d7340 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f0d600640 10 client.4425 handle_command_reply: tid=0 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 client.4425 shutdown 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 msgr2=0x558ff28df0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff28df0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5f18016040 tx=0x7f5f18001250 comp rx=0 tx=0).stop 2024-01-31T05:00:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 shutdown_connections 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x558ff28d7340 0x558ff2946ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ff28c34d0 0x558ff28df0f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558ff28c2bb0 0x558ff28debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 --2- 172.21.15.97:0/3443094673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558ff28b80f0 0x558ff28ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f0d600640 0 client.4425 ms_handle_reset on v2:172.21.15.97:6835/294460102 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 >> 172.21.15.97:0/3443094673 conn(0x558ff28bfe70 msgr2=0x558ff28d9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 shutdown_connections 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.633+0000 7f5f2a1bd740 1 -- 172.21.15.97:0/3443094673 wait complete. 2024-01-31T05:00:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f2a1bd740 20 client.4425 trim_cache size 0 max 16384 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "dc34a409-d027-49db-a591-dbd89a074581", 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4257, 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 8.0457068459999999, 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 6, 2024-01-31T05:00:54.639 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 26, 2024-01-31T05:00:54.640 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 26, 2024-01-31T05:00:54.640 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 15.327013968999999 2024-01-31T05:00:54.640 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ef806aed0 msgr2=0x7f5ef806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ef806aed0 0x7f5ef806d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5f10005880 tx=0x7f5f10009040 comp rx=0 tx=0).stop 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 msgr2=0x7f5f200adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 0x7f5f200adf10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f0802bf70 tx=0x7f5f080046c0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 shutdown_connections 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f20157a90 0x7f5f2014e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ef806aed0 0x7f5ef806d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f201576b0 0x7f5f200adf10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 --2- 172.21.15.97:0/1226941093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f2006c640 0x7f5f200af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 >> 172.21.15.97:0/1226941093 conn(0x7f5f2006ad40 msgr2=0x7f5f200b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 shutdown_connections 2024-01-31T05:00:54.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.634+0000 7f5f25c00640 1 -- 172.21.15.97:0/1226941093 wait complete. 2024-01-31T05:00:54.649 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-31T05:00:54.649 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:54.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:00:54.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.842+0000 7f4eb2c00640 1 Processor -- start 2024-01-31T05:00:54.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.842+0000 7f4eb2c00640 1 -- start start 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eb2c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4eac15a600 0x7f4eac15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eb2c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4eac06d090 0x7f4eac06d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eb2c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac06c8b0 con 0x7f4eac0b1d10 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eb2c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac15b120 con 0x7f4eac15a600 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eb2c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac15b400 con 0x7f4eac06d090 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eabe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4eac0b1d10 0x7f4eac0b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43526/0 (socket says 172.21.15.97:43526) 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eabe00640 1 -- 172.21.15.97:0/3846718428 learned_addr learned my addr 172.21.15.97:0/3846718428 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:00:54.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4ea3200640 1 --2- 172.21.15.97:0/3846718428 >> v2:172.21.15.97:3300/0 conn(0x7f4eac15a600 0x7f4eac15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.844+0000 7f4eab400640 1 -- 172.21.15.97:0/3846718428 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145605018 0 0) 0x7f4eac06c8b0 con 0x7f4eac0b1d10 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.845+0000 7f4eab400640 1 -- 172.21.15.97:0/3846718428 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e8c003520 con 0x7f4eac0b1d10 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.845+0000 7f4eb0c00640 1 --2- 172.21.15.97:0/3846718428 >> v2:172.21.15.188:3300/0 conn(0x7f4eac06d090 0x7f4eac06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.845+0000 7f4ea3200640 1 -- 172.21.15.97:0/3846718428 >> v2:172.21.15.188:3300/0 conn(0x7f4eac06d090 msgr2=0x7f4eac06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.845+0000 7f4ea3200640 1 --2- 172.21.15.97:0/3846718428 >> v2:172.21.15.188:3300/0 conn(0x7f4eac06d090 0x7f4eac06d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.845+0000 7f4ea3200640 1 -- 172.21.15.97:0/3846718428 >> v1:172.21.15.97:6789/0 conn(0x7f4eac0b1d10 legacy=0x7f4eac0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.845+0000 7f4ea3200640 1 -- 172.21.15.97:0/3846718428 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac06cb70 con 0x7f4eac15a600 2024-01-31T05:00:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.846+0000 7f4ea3200640 1 --2- 172.21.15.97:0/3846718428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac15a600 0x7f4eac15aa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0b09a0 tx=0x7f4e9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b01c85af80c41b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.846+0000 7f4eab400640 1 -- 172.21.15.97:0/3846718428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94031830 con 0x7f4eac15a600 2024-01-31T05:00:54.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.846+0000 7f4eab400640 1 -- 172.21.15.97:0/3846718428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94031990 con 0x7f4eac15a600 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.846+0000 7f4eab400640 1 -- 172.21.15.97:0/3846718428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94031d20 con 0x7f4eac15a600 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.846+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3846718428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac15a600 msgr2=0x7f4eac15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3846718428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac15a600 0x7f4eac15aa60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0b09a0 tx=0x7f4e9402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3846718428 shutdown_connections 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3846718428 >> v2:172.21.15.188:3300/0 conn(0x7f4eac06d090 0x7f4eac06d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3846718428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac15a600 0x7f4eac15aa60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3846718428 >> 172.21.15.97:0/3846718428 conn(0x7f4eac0b0a10 msgr2=0x7f4eac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3846718428 shutdown_connections 2024-01-31T05:00:54.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.847+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3846718428 wait complete. 2024-01-31T05:00:54.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.848+0000 7f4eb2c00640 1 Processor -- start 2024-01-31T05:00:54.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.848+0000 7f4eb2c00640 1 -- start start 2024-01-31T05:00:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.849+0000 7f4eb2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 0x7f4eac14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.849+0000 7f4eb2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eac0b1d10 0x7f4eac152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.849+0000 7f4eb2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eac15a600 0x7f4eac15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.849+0000 7f4eb2c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac157f50 con 0x7f4eac06d090 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.849+0000 7f4eb2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac14cb70 con 0x7f4eac15a600 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.849+0000 7f4eb2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac157340 con 0x7f4eac0b1d10 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 0x7f4eac14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 0x7f4eac14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46768/0 (socket says 172.21.15.97:46768) 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 -- 172.21.15.97:0/3492793053 learned_addr learned my addr 172.21.15.97:0/3492793053 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eb0c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eac15a600 0x7f4eac15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 -- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eac0b1d10 msgr2=0x7f4eac152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eac0b1d10 0x7f4eac152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 -- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eac15a600 msgr2=0x7f4eac15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eac15a600 0x7f4eac15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 -- 172.21.15.97:0/3492793053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009400 con 0x7f4eac06d090 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eb0c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eac15a600 0x7f4eac15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:00:54.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4eabe00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 0x7f4eac14c630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c430 tx=0x7f4e9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.850+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c005ce0 con 0x7f4eac06d090 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.851+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c005e40 con 0x7f4eac06d090 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.851+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac157620 con 0x7f4eac06d090 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.851+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0072f0 con 0x7f4eac06d090 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.851+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac157be0 con 0x7f4eac06d090 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.852+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac0ae900 con 0x7f4eac06d090 2024-01-31T05:00:54.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.853+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4e9c007450 con 0x7f4eac06d090 2024-01-31T05:00:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.854+0000 7f4ea8c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4e8406aed0 0x7f4e8406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:00:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.855+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4e9c088ff0 con 0x7f4eac06d090 2024-01-31T05:00:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.855+0000 7f4ea3200640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4e8406aed0 0x7f4e8406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:00:54.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.855+0000 7f4ea3200640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4e8406aed0 0x7f4e8406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e94036000 tx=0x7f4e94030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:00:54.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:54.859+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4e9c057640 con 0x7f4eac06d090 2024-01-31T05:00:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.092+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4eac0b3ee0 con 0x7f4eac06d090 2024-01-31T05:00:55.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.093+0000 7f4ea8c00640 1 -- 172.21.15.97:0/3492793053 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4e9c017070 con 0x7f4eac06d090 2024-01-31T05:00:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-31T05:00:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.095+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4e8406aed0 msgr2=0x7f4e8406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.095+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4e8406aed0 0x7f4e8406d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e94036000 tx=0x7f4e94030000 comp rx=0 tx=0).stop 2024-01-31T05:00:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.095+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 msgr2=0x7f4eac14c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.095+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 0x7f4eac14c630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c430 tx=0x7f4e9c00c8f0 comp rx=0 tx=0).stop 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 shutdown_connections 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eac15a600 0x7f4eac15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eac0b1d10 0x7f4eac152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4e8406aed0 0x7f4e8406d390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 --2- 172.21.15.97:0/3492793053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eac06d090 0x7f4eac14c630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 >> 172.21.15.97:0/3492793053 conn(0x7f4eac0b0a10 msgr2=0x7f4eac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 shutdown_connections 2024-01-31T05:00:55.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:55.096+0000 7f4eb2c00640 1 -- 172.21.15.97:0/3492793053 wait complete. 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:00:55.114 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-01-31T05:00:55.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:55.114 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:00:55.161 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:00:55.162 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:00:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:00:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:00:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42469sec preferred_lft 42469sec 2024-01-31T05:00:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:00:55.163 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:00:55.164 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-31T05:00:55.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:55.164 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:00:55.164 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:00:55.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-01-31T05:00:55.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-01-31T05:00:55.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-01-31T05:00:55.165 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-31T05:00:55.165 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:00:55.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:55.637 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:00:55.712 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:00:55.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:55.717 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:00:55.770 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:00:55.770 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:00:55.770 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:00:55.770 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:00:55.770 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:00:55.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:55.771 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:00:55.771 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:00:55.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:00:55.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:00:55.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:00:55.771 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:00:56.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:56.478 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:00:56.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:56.520 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-01-31T05:00:56.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:56.608 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:00:56.608 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:00:56.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:56.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-01-31T05:00:56.608 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:00:56.779 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-01-31T05:00:56.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:56.779 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:00:56.779 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:00:56.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-01-31T05:00:56.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-31T05:00:56.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-01-31T05:00:56.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-01-31T05:00:56.780 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-01-31T05:00:56.780 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:00:57.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:57.034 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:00:57.034 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:00:57.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:00:57.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-01-31T05:00:57.034 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:00:57.185 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-01-31T05:00:57.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:00:57.186 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:57.226 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-01-31T05:00:57.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:00:57.227 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:57.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:00:57.445 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:00:57.491 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:00:57.492 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:00:57.493 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:00:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:00:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:00:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:00:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:00:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:57.494 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:57.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:57.495 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:00:57.586 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:00:57.587 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-01-31T05:00:57.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:00:57.659 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:00:57.699 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:00:57.699 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:00:57.700 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:00:57.700 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:00:57.700 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:00:57.700 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:00:57.700 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:00:57.700 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:00:57.701 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:00:57.701 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:00:57.701 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:00:57.701 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:00:57.701 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:00:57.701 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:00:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:00:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:00:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:00:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:00:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:00:57.702 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:00:57.703 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:00:57.703 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:00:57.703 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:00:57.703 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:00:57.703 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:00:57.703 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:00:57.704 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:00:57.704 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:00:57.704 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:00:57.704 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:57.704 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:57.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:57.706 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:00:57.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:2024-01-31T05:00:57.787+0000 7f7954e3c480 -1 init, newargv = 0x555f7ce97820 newargc=15 2024-01-31T05:00:57.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[42983]: starting ceph client 2024-01-31T05:00:57.798 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[42983]: starting fuse 2024-01-31T05:00:58.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:00:58.995 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:00:59.049 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:00:59.049 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:00:59.049 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:00:59.049 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:00:59.050 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:00:59.051 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:59.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:59.053 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:00:59.102 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:00:59.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:00:59.103 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.admin")) 2024-01-31T05:00:59.104 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:00:59.274 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.admin.42983.asok 2024-01-31T05:00:59.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:00:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:00:59.291 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42983.asok 2024-01-31T05:00:59.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:59.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42983.asok status 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "admin", 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "42983", 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:00:59.527 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4440, 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4440 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2813941730 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:00:59.528 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2813941730 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4440 192.168.0.1:0/2813941730", 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2813941730", 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 6, 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 26, 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:00:59.529 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:00:59.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:59.543 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:00:59.543 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:59.597 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:00:59.597 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:59.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:59.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:00:59.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:00:59.692 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-01-31T05:00:59.780 INFO:teuthology.orchestra.run.smithi097.stdout:# file: . 2024-01-31T05:00:59.781 INFO:teuthology.orchestra.run.smithi097.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-01-31T05:00:59.781 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:00:59.782 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:00:59.782 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:00:59.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:00.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.042+0000 7f6b51600640 1 Processor -- start 2024-01-31T05:01:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.042+0000 7f6b51600640 1 -- start start 2024-01-31T05:01:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b51600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b4c0b1d10 0x7f6b4c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b51600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b4c15a600 0x7f6b4c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b51600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4c0b2630 con 0x7f6b4c15dc50 2024-01-31T05:01:00.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b51600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c06c8b0 con 0x7f6b4c0b1d10 2024-01-31T05:01:00.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b51600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4c15b120 con 0x7f6b4c15a600 2024-01-31T05:01:00.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b4aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b4c0b1d10 0x7f6b4c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b4aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b4c0b1d10 0x7f6b4c0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43168/0 (socket says 172.21.15.97:43168) 2024-01-31T05:01:00.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.044+0000 7f6b4aa00640 1 -- 172.21.15.97:0/2948827819 learned_addr learned my addr 172.21.15.97:0/2948827819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:00.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.045+0000 7f6b4aa00640 1 -- 172.21.15.97:0/2948827819 >> v2:172.21.15.188:3300/0 conn(0x7f6b4c15a600 msgr2=0x7f6b4c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:00.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.045+0000 7f6b4aa00640 1 --2- 172.21.15.97:0/2948827819 >> v2:172.21.15.188:3300/0 conn(0x7f6b4c15a600 0x7f6b4c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.045+0000 7f6b4aa00640 1 -- 172.21.15.97:0/2948827819 >> v1:172.21.15.188:6789/0 conn(0x7f6b4c15dc50 legacy=0x7f6b4c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:00.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.045+0000 7f6b4aa00640 1 -- 172.21.15.97:0/2948827819 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c15b400 con 0x7f6b4c0b1d10 2024-01-31T05:01:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.046+0000 7f6b4aa00640 1 --2- 172.21.15.97:0/2948827819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c0b1d10 0x7f6b4c0b20f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c158250 tx=0x7f6b3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d487c9f5eb9cad9a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.046+0000 7f6b49600640 1 -- 172.21.15.97:0/2948827819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c031870 con 0x7f6b4c0b1d10 2024-01-31T05:01:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.046+0000 7f6b49600640 1 -- 172.21.15.97:0/2948827819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0319d0 con 0x7f6b4c0b1d10 2024-01-31T05:01:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.046+0000 7f6b49600640 1 -- 172.21.15.97:0/2948827819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c031d60 con 0x7f6b4c0b1d10 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 -- 172.21.15.97:0/2948827819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c0b1d10 msgr2=0x7f6b4c0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 --2- 172.21.15.97:0/2948827819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c0b1d10 0x7f6b4c0b20f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c158250 tx=0x7f6b3c02b910 comp rx=0 tx=0).stop 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 -- 172.21.15.97:0/2948827819 shutdown_connections 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 --2- 172.21.15.97:0/2948827819 >> v2:172.21.15.188:3300/0 conn(0x7f6b4c15a600 0x7f6b4c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 --2- 172.21.15.97:0/2948827819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c0b1d10 0x7f6b4c0b20f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 -- 172.21.15.97:0/2948827819 >> 172.21.15.97:0/2948827819 conn(0x7f6b4c0b0a10 msgr2=0x7f6b4c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 -- 172.21.15.97:0/2948827819 shutdown_connections 2024-01-31T05:01:00.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.047+0000 7f6b51600640 1 -- 172.21.15.97:0/2948827819 wait complete. 2024-01-31T05:01:00.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.049+0000 7f6b51600640 1 Processor -- start 2024-01-31T05:01:00.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.049+0000 7f6b51600640 1 -- start start 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b51600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b4c0b1d10 0x7f6b4c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b51600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 0x7f6b4c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b51600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c15dc50 0x7f6b4c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b51600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c0722f0 con 0x7f6b4c15a600 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b51600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c158250 con 0x7f6b4c15dc50 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b51600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c15bc20 con 0x7f6b4c0b1d10 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b4a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 0x7f6b4c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b4a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 0x7f6b4c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43170/0 (socket says 172.21.15.97:43170) 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b4a000640 1 -- 172.21.15.97:0/13649750 learned_addr learned my addr 172.21.15.97:0/13649750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b4aa00640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b4c0b1d10 0x7f6b4c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.050+0000 7f6b4b400640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c15dc50 0x7f6b4c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4a000640 1 -- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b4c0b1d10 msgr2=0x7f6b4c1499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4a000640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b4c0b1d10 0x7f6b4c1499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4a000640 1 -- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c15dc50 msgr2=0x7f6b4c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4a000640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c15dc50 0x7f6b4c158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4a000640 1 -- 172.21.15.97:0/13649750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c0094d0 con 0x7f6b4c15a600 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4b400640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c15dc50 0x7f6b4c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b4a000640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 0x7f6b4c15b6e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400b4a0 tx=0x7f6b3400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34010610 con 0x7f6b4c15a600 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b34010770 con 0x7f6b4c15a600 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34010b20 con 0x7f6b4c15a600 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c0adb80 con 0x7f6b4c15a600 2024-01-31T05:01:00.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.051+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c15d530 con 0x7f6b4c15a600 2024-01-31T05:01:00.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.052+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c0677c0 con 0x7f6b4c15a600 2024-01-31T05:01:00.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.053+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6b34010c80 con 0x7f6b4c15a600 2024-01-31T05:01:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.055+0000 7f6b42a00640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b2406aed0 0x7f6b2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.055+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6b340895c0 con 0x7f6b4c15a600 2024-01-31T05:01:00.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.055+0000 7f6b4aa00640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b2406aed0 0x7f6b2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.056+0000 7f6b4aa00640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b2406aed0 0x7f6b2406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c036000 tx=0x7f6b3c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.060+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6b34054190 con 0x7f6b4c15a600 2024-01-31T05:01:00.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.255+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b4c06e740 con 0x7f6b4c15a600 2024-01-31T05:01:00.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.259+0000 7f6b42a00640 1 -- 172.21.15.97:0/13649750 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f6b340577b0 con 0x7f6b4c15a600 2024-01-31T05:01:00.258 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:00.259 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b2406aed0 msgr2=0x7f6b2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b2406aed0 0x7f6b2406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c036000 tx=0x7f6b3c03c040 comp rx=0 tx=0).stop 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 msgr2=0x7f6b4c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 0x7f6b4c15b6e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400b4a0 tx=0x7f6b3400b960 comp rx=0 tx=0).stop 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 shutdown_connections 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b4c15dc50 0x7f6b4c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b2406aed0 0x7f6b2406d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b4c15a600 0x7f6b4c15b6e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 --2- 172.21.15.97:0/13649750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b4c0b1d10 0x7f6b4c1499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.261+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 >> 172.21.15.97:0/13649750 conn(0x7f6b4c0b0a10 msgr2=0x7f6b4c0acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:00.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.262+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 shutdown_connections 2024-01-31T05:01:00.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.262+0000 7f6b51600640 1 -- 172.21.15.97:0/13649750 wait complete. 2024-01-31T05:01:00.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:00.280 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.493+0000 7f23d9a00640 1 Processor -- start 2024-01-31T05:01:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.493+0000 7f23d9a00640 1 -- start start 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d9a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23d415a350 0x7f23d40abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d9a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23d415a730 0x7f23d406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d9a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d415abc0 con 0x7f23d406dfc0 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d9a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40b2c90 con 0x7f23d415a730 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d9a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d415aea0 con 0x7f23d415a350 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23cbe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f23d415a730 0x7f23d406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d3400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f23d406dfc0 0x7f23d40cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46322/0 (socket says 172.21.15.97:46322) 2024-01-31T05:01:00.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d3400640 1 -- 172.21.15.97:0/289708272 learned_addr learned my addr 172.21.15.97:0/289708272 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:00.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.495+0000 7f23d2a00640 1 --2- 172.21.15.97:0/289708272 >> v2:172.21.15.188:3300/0 conn(0x7f23d415a350 0x7f23d40abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.496+0000 7f23cbe00640 1 -- 172.21.15.97:0/289708272 >> v2:172.21.15.188:3300/0 conn(0x7f23d415a350 msgr2=0x7f23d40abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:00.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.496+0000 7f23cbe00640 1 --2- 172.21.15.97:0/289708272 >> v2:172.21.15.188:3300/0 conn(0x7f23d415a350 0x7f23d40abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.496+0000 7f23cbe00640 1 -- 172.21.15.97:0/289708272 >> v1:172.21.15.97:6789/0 conn(0x7f23d406dfc0 legacy=0x7f23d40cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.496+0000 7f23cbe00640 1 -- 172.21.15.97:0/289708272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d415b180 con 0x7f23d415a730 2024-01-31T05:01:00.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.497+0000 7f23cbe00640 1 --2- 172.21.15.97:0/289708272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a730 0x7f23d406da80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097d0 tx=0x7f23bc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c65e58659ea9859 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.497+0000 7f23d2000640 1 -- 172.21.15.97:0/289708272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc031780 con 0x7f23d415a730 2024-01-31T05:01:00.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.497+0000 7f23d2000640 1 -- 172.21.15.97:0/289708272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0318e0 con 0x7f23d415a730 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.497+0000 7f23d2000640 1 -- 172.21.15.97:0/289708272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc031c70 con 0x7f23d415a730 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 -- 172.21.15.97:0/289708272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a730 msgr2=0x7f23d406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 --2- 172.21.15.97:0/289708272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a730 0x7f23d406da80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097d0 tx=0x7f23bc02b820 comp rx=0 tx=0).stop 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 -- 172.21.15.97:0/289708272 shutdown_connections 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 --2- 172.21.15.97:0/289708272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a730 0x7f23d406da80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 --2- 172.21.15.97:0/289708272 >> v2:172.21.15.188:3300/0 conn(0x7f23d415a350 0x7f23d40abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 -- 172.21.15.97:0/289708272 >> 172.21.15.97:0/289708272 conn(0x7f23d40b0a10 msgr2=0x7f23d40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:00.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 -- 172.21.15.97:0/289708272 shutdown_connections 2024-01-31T05:01:00.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.498+0000 7f23d9a00640 1 -- 172.21.15.97:0/289708272 wait complete. 2024-01-31T05:01:00.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.500+0000 7f23d9a00640 1 Processor -- start 2024-01-31T05:01:00.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.500+0000 7f23d9a00640 1 -- start start 2024-01-31T05:01:00.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d9a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23d406dfc0 0x7f23d4165e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 0x7f23d40b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d9a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23d40ae7f0 0x7f23d4141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d9a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41663b0 con 0x7f23d415a350 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d9a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41495a0 con 0x7f23d406dfc0 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d9a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41665a0 con 0x7f23d40ae7f0 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 0x7f23d40b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 0x7f23d40b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43178/0 (socket says 172.21.15.97:43178) 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 -- 172.21.15.97:0/1350019494 learned_addr learned my addr 172.21.15.97:0/1350019494 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d2a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23d406dfc0 0x7f23d4165e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d3400640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23d40ae7f0 0x7f23d4141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 -- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23d40ae7f0 msgr2=0x7f23d4141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23d40ae7f0 0x7f23d4141f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 -- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23d406dfc0 msgr2=0x7f23d4165e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23d406dfc0 0x7f23d4165e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23cbe00640 1 -- 172.21.15.97:0/1350019494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc009400 con 0x7f23d415a350 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d2a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23d406dfc0 0x7f23d4165e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.501+0000 7f23d3400640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23d40ae7f0 0x7f23d4141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.502+0000 7f23cbe00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 0x7f23d40b0180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097d0 tx=0x7f23bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.502+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc032820 con 0x7f23d415a350 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.502+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0329a0 con 0x7f23d415a350 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.502+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc032d50 con 0x7f23d415a350 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.502+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4148990 con 0x7f23d415a350 2024-01-31T05:01:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.502+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d4148f50 con 0x7f23d415a350 2024-01-31T05:01:00.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.503+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d4067830 con 0x7f23d415a350 2024-01-31T05:01:00.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.504+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f23bc04e440 con 0x7f23d415a350 2024-01-31T05:01:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.505+0000 7f23caa00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f23ac06aed0 0x7f23ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.505+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23bc0aba50 con 0x7f23d415a350 2024-01-31T05:01:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.505+0000 7f23d2a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f23ac06aed0 0x7f23ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.506+0000 7f23d2a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f23ac06aed0 0x7f23ac06d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009820 tx=0x7f23c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.509+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f23bc0766e0 con 0x7f23d415a350 2024-01-31T05:01:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.702+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23d4067830 con 0x7f23d415a350 2024-01-31T05:01:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.703+0000 7f23caa00640 1 -- 172.21.15.97:0/1350019494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23bc079d00 con 0x7f23d415a350 2024-01-31T05:01:00.703 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f23ac06aed0 msgr2=0x7f23ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f23ac06aed0 0x7f23ac06d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009820 tx=0x7f23c4008040 comp rx=0 tx=0).stop 2024-01-31T05:01:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 msgr2=0x7f23d40b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 0x7f23d40b0180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0097d0 tx=0x7f23bc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 shutdown_connections 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23d40ae7f0 0x7f23d4141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f23ac06aed0 0x7f23ac06d390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d415a350 0x7f23d40b0180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 --2- 172.21.15.97:0/1350019494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23d406dfc0 0x7f23d4165e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 >> 172.21.15.97:0/1350019494 conn(0x7f23d40b0a10 msgr2=0x7f23d40cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 shutdown_connections 2024-01-31T05:01:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.706+0000 7f23d9a00640 1 -- 172.21.15.97:0/1350019494 wait complete. 2024-01-31T05:01:00.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:00.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.932+0000 7fa9a7000640 1 Processor -- start 2024-01-31T05:01:00.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.932+0000 7fa9a7000640 1 -- start start 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a7000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9a015af90 0x7fa9a006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a7000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9a00ac770 0x7fa9a00acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a7000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a006db30 con 0x7fa9a006d210 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a7000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a015b440 con 0x7fa9a00ac770 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a7000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a00714c0 con 0x7fa9a015af90 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9a00ac770 0x7fa9a00acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9a00ac770 0x7fa9a00acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43190/0 (socket says 172.21.15.97:43190) 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 -- 172.21.15.97:0/3133721439 learned_addr learned my addr 172.21.15.97:0/3133721439 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 -- 172.21.15.97:0/3133721439 >> v2:172.21.15.188:3300/0 conn(0x7fa9a015af90 msgr2=0x7fa9a006d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 --2- 172.21.15.97:0/3133721439 >> v2:172.21.15.188:3300/0 conn(0x7fa9a015af90 0x7fa9a006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 -- 172.21.15.97:0/3133721439 >> v1:172.21.15.188:6789/0 conn(0x7fa9a006d210 legacy=0x7fa9a0071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:00.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.934+0000 7fa9a6600640 1 -- 172.21.15.97:0/3133721439 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a0070420 con 0x7fa9a00ac770 2024-01-31T05:01:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.935+0000 7fa9a6600640 1 --2- 172.21.15.97:0/3133721439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00ac770 0x7fa9a00acbe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00cebd0 tx=0x7fa99802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6e6c50595c456d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.936+0000 7fa99fe00640 1 -- 172.21.15.97:0/3133721439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99802ca80 con 0x7fa9a00ac770 2024-01-31T05:01:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.936+0000 7fa99fe00640 1 -- 172.21.15.97:0/3133721439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99802cbe0 con 0x7fa9a00ac770 2024-01-31T05:01:00.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.936+0000 7fa99fe00640 1 -- 172.21.15.97:0/3133721439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9980343f0 con 0x7fa9a00ac770 2024-01-31T05:01:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.936+0000 7fa9a7000640 1 -- 172.21.15.97:0/3133721439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00ac770 msgr2=0x7fa9a00acbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.936+0000 7fa9a7000640 1 --2- 172.21.15.97:0/3133721439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00ac770 0x7fa9a00acbe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00cebd0 tx=0x7fa99802bee0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.937+0000 7fa9a7000640 1 -- 172.21.15.97:0/3133721439 shutdown_connections 2024-01-31T05:01:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.937+0000 7fa9a7000640 1 --2- 172.21.15.97:0/3133721439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a00ac770 0x7fa9a00acbe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.937+0000 7fa9a7000640 1 --2- 172.21.15.97:0/3133721439 >> v2:172.21.15.188:3300/0 conn(0x7fa9a015af90 0x7fa9a006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.937+0000 7fa9a7000640 1 -- 172.21.15.97:0/3133721439 >> 172.21.15.97:0/3133721439 conn(0x7fa9a006ad40 msgr2=0x7fa9a00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.937+0000 7fa9a7000640 1 -- 172.21.15.97:0/3133721439 shutdown_connections 2024-01-31T05:01:00.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.937+0000 7fa9a7000640 1 -- 172.21.15.97:0/3133721439 wait complete. 2024-01-31T05:01:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.938+0000 7fa9a7000640 1 Processor -- start 2024-01-31T05:01:00.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 -- start start 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 0x7fa9a01f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a00ac770 0x7fa9a00999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015af90 0x7fa9a00c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00cad00 con 0x7fa9a006d210 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a0099f00 con 0x7fa9a00ac770 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.939+0000 7fa9a7000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a009ad20 con 0x7fa9a015af90 2024-01-31T05:01:00.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 0x7fa9a01f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 0x7fa9a01f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43200/0 (socket says 172.21.15.97:43200) 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 -- 172.21.15.97:0/2734647815 learned_addr learned my addr 172.21.15.97:0/2734647815 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a6600640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015af90 0x7fa9a00c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5200640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a00ac770 0x7fa9a00999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 -- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015af90 msgr2=0x7fa9a00c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015af90 0x7fa9a00c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 -- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a00ac770 msgr2=0x7fa9a00999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a00ac770 0x7fa9a00999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5c00640 1 -- 172.21.15.97:0/2734647815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa998009b30 con 0x7fa9a006d210 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a6600640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015af90 0x7fa9a00c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.940+0000 7fa9a5200640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a00ac770 0x7fa9a00999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.941+0000 7fa9a5c00640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 0x7fa9a01f2db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b410 tx=0x7fa99000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.941+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990011830 con 0x7fa9a006d210 2024-01-31T05:01:00.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.941+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990011990 con 0x7fa9a006d210 2024-01-31T05:01:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.941+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990011d40 con 0x7fa9a006d210 2024-01-31T05:01:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.941+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a009a110 con 0x7fa9a006d210 2024-01-31T05:01:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.941+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a009a6d0 con 0x7fa9a006d210 2024-01-31T05:01:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.942+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a00676b0 con 0x7fa9a006d210 2024-01-31T05:01:00.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.943+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa99001a400 con 0x7fa9a006d210 2024-01-31T05:01:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.944+0000 7fa99d600640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa97806aed0 0x7fa97806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:00.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.944+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa990089b20 con 0x7fa9a006d210 2024-01-31T05:01:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.944+0000 7fa9a5200640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa97806aed0 0x7fa97806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.945+0000 7fa9a5200640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa97806aed0 0x7fa97806d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa988006fd0 tx=0x7fa988008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:00.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:00.948+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa990054660 con 0x7fa9a006d210 2024-01-31T05:01:01.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.140+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9a0071640 con 0x7fa9a006d210 2024-01-31T05:01:01.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.141+0000 7fa99d600640 1 -- 172.21.15.97:0/2734647815 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9900150a0 con 0x7fa9a006d210 2024-01-31T05:01:01.140 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa97806aed0 msgr2=0x7fa97806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa97806aed0 0x7fa97806d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa988006fd0 tx=0x7fa988008040 comp rx=0 tx=0).stop 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 msgr2=0x7fa9a01f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 0x7fa9a01f2db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b410 tx=0x7fa99000b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 shutdown_connections 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015af90 0x7fa9a00c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a00ac770 0x7fa9a00999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa97806aed0 0x7fa97806d390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 --2- 172.21.15.97:0/2734647815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a006d210 0x7fa9a01f2db0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 >> 172.21.15.97:0/2734647815 conn(0x7fa9a006ad40 msgr2=0x7fa9a00ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.143+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 shutdown_connections 2024-01-31T05:01:01.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.144+0000 7fa9a7000640 1 -- 172.21.15.97:0/2734647815 wait complete. 2024-01-31T05:01:01.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:01.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.376+0000 7ff0a7400640 1 Processor -- start 2024-01-31T05:01:01.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.377+0000 7ff0a7400640 1 -- start start 2024-01-31T05:01:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.378+0000 7ff0a7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00cbf20 0x7ff0a00722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.378+0000 7ff0a7400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0a00abcf0 0x7ff0a00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.378+0000 7ff0a7400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a006cd20 con 0x7ff0a015b470 2024-01-31T05:01:01.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.378+0000 7ff0a7400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a00ce970 con 0x7ff0a00abcf0 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.378+0000 7ff0a7400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a015b850 con 0x7ff0a00cbf20 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.378+0000 7ff0a5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0a00abcf0 0x7ff0a00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff0a5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0a00abcf0 0x7ff0a00ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43212/0 (socket says 172.21.15.97:43212) 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff0a5400640 1 -- 172.21.15.97:0/3638330523 learned_addr learned my addr 172.21.15.97:0/3638330523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff097c00640 1 --2- 172.21.15.97:0/3638330523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00cbf20 0x7ff0a00722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff0a5400640 1 -- 172.21.15.97:0/3638330523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00cbf20 msgr2=0x7ff0a00722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff0a5400640 1 --2- 172.21.15.97:0/3638330523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00cbf20 0x7ff0a00722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff0a5400640 1 -- 172.21.15.97:0/3638330523 >> v1:172.21.15.188:6789/0 conn(0x7ff0a015b470 legacy=0x7ff0a00cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.379+0000 7ff0a5400640 1 -- 172.21.15.97:0/3638330523 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00ceb60 con 0x7ff0a00abcf0 2024-01-31T05:01:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.380+0000 7ff0a5400640 1 --2- 172.21.15.97:0/3638330523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a00abcf0 0x7ff0a00ce430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff098009eb0 tx=0x7ff09802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8f6dd8baad3352 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.380+0000 7ff09fe00640 1 -- 172.21.15.97:0/3638330523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09802cad0 con 0x7ff0a00abcf0 2024-01-31T05:01:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.380+0000 7ff09fe00640 1 -- 172.21.15.97:0/3638330523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09802cc30 con 0x7ff0a00abcf0 2024-01-31T05:01:01.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff09fe00640 1 -- 172.21.15.97:0/3638330523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980343f0 con 0x7ff0a00abcf0 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 -- 172.21.15.97:0/3638330523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a00abcf0 msgr2=0x7ff0a00ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 --2- 172.21.15.97:0/3638330523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a00abcf0 0x7ff0a00ce430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff098009eb0 tx=0x7ff09802bf30 comp rx=0 tx=0).stop 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 -- 172.21.15.97:0/3638330523 shutdown_connections 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 --2- 172.21.15.97:0/3638330523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a00abcf0 0x7ff0a00ce430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 --2- 172.21.15.97:0/3638330523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00cbf20 0x7ff0a00722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 -- 172.21.15.97:0/3638330523 >> 172.21.15.97:0/3638330523 conn(0x7ff0a00b0a10 msgr2=0x7ff0a00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 -- 172.21.15.97:0/3638330523 shutdown_connections 2024-01-31T05:01:01.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.381+0000 7ff0a7400640 1 -- 172.21.15.97:0/3638330523 wait complete. 2024-01-31T05:01:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.383+0000 7ff0a7400640 1 Processor -- start 2024-01-31T05:01:01.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.383+0000 7ff0a7400640 1 -- start start 2024-01-31T05:01:01.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00abcf0 0x7ff0a0145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a7400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a00cbf20 0x7ff0a01e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 0x7ff0a00acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a7400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01f58b0 con 0x7ff0a015b470 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a7400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0146310 con 0x7ff0a00cbf20 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a7400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00aecd0 con 0x7ff0a00abcf0 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 0x7ff0a00acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 0x7ff0a00acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43214/0 (socket says 172.21.15.97:43214) 2024-01-31T05:01:01.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.384+0000 7ff0a5400640 1 -- 172.21.15.97:0/2230368298 learned_addr learned my addr 172.21.15.97:0/2230368298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a4a00640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00abcf0 0x7ff0a0145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff097c00640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a00cbf20 0x7ff0a01e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a5400640 1 -- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00abcf0 msgr2=0x7ff0a0145dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a5400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00abcf0 0x7ff0a0145dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a5400640 1 -- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a00cbf20 msgr2=0x7ff0a01e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a5400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a00cbf20 0x7ff0a01e39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a5400640 1 -- 172.21.15.97:0/2230368298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff098009b30 con 0x7ff0a015b470 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff097c00640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a00cbf20 0x7ff0a01e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a5400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 0x7ff0a00acca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff09802bf70 tx=0x7ff098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff098009510 con 0x7ff0a015b470 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09803e450 con 0x7ff0a015b470 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09803e800 con 0x7ff0a015b470 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.385+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a01f55d0 con 0x7ff0a015b470 2024-01-31T05:01:01.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.386+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a00af250 con 0x7ff0a015b470 2024-01-31T05:01:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.387+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a00b0df0 con 0x7ff0a015b470 2024-01-31T05:01:01.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.388+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff09803e960 con 0x7ff0a015b470 2024-01-31T05:01:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.388+0000 7ff09d600640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff07806af00 0x7ff07806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.388+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff0980ac720 con 0x7ff0a015b470 2024-01-31T05:01:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.389+0000 7ff0a4a00640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff07806af00 0x7ff07806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.389+0000 7ff0a4a00640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff07806af00 0x7ff07806d3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff090001630 tx=0x7ff090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:01.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.393+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff098077260 con 0x7ff0a015b470 2024-01-31T05:01:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.592+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0a0067830 con 0x7ff0a015b470 2024-01-31T05:01:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.593+0000 7ff09d600640 1 -- 172.21.15.97:0/2230368298 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff09807a880 con 0x7ff0a015b470 2024-01-31T05:01:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff07806af00 msgr2=0x7ff07806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff07806af00 0x7ff07806d3c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff090001630 tx=0x7ff090009040 comp rx=0 tx=0).stop 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 msgr2=0x7ff0a00acca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 0x7ff0a00acca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff09802bf70 tx=0x7ff098004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 shutdown_connections 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff07806af00 0x7ff07806d3c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0a015b470 0x7ff0a00acca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0a00cbf20 0x7ff0a01e39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 --2- 172.21.15.97:0/2230368298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0a00abcf0 0x7ff0a0145dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 >> 172.21.15.97:0/2230368298 conn(0x7ff0a00b0a10 msgr2=0x7ff0a015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:01.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 shutdown_connections 2024-01-31T05:01:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.596+0000 7ff0a7400640 1 -- 172.21.15.97:0/2230368298 wait complete. 2024-01-31T05:01:01.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:01.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.816+0000 7f7ee2600640 1 Processor -- start 2024-01-31T05:01:01.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.817+0000 7f7ee2600640 1 -- start start 2024-01-31T05:01:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.818+0000 7f7ee2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.818+0000 7f7ee2600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7edc15edf0 0x7f7edc06eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.818+0000 7f7ee2600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0cc760 con 0x7f7edc0715a0 2024-01-31T05:01:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.818+0000 7f7ee2600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc06b6b0 con 0x7f7edc15edf0 2024-01-31T05:01:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.818+0000 7f7ee2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc06ae10 con 0x7f7edc06fc70 2024-01-31T05:01:01.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7edc15edf0 0x7f7edc06eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7edc15edf0 0x7f7edc06eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43220/0 (socket says 172.21.15.97:43220) 2024-01-31T05:01:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 -- 172.21.15.97:0/3781512654 learned_addr learned my addr 172.21.15.97:0/3781512654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edb400640 1 --2- 172.21.15.97:0/3781512654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 -- 172.21.15.97:0/3781512654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 msgr2=0x7f7edc06a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 --2- 172.21.15.97:0/3781512654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc06a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 -- 172.21.15.97:0/3781512654 >> v1:172.21.15.188:6789/0 conn(0x7f7edc0715a0 legacy=0x7f7edc0719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.819+0000 7f7edaa00640 1 -- 172.21.15.97:0/3781512654 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc06b0f0 con 0x7f7edc15edf0 2024-01-31T05:01:01.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.820+0000 7f7edaa00640 1 --2- 172.21.15.97:0/3781512654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc06eb10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009870 tx=0x7f7ec402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e554ae939fd3ae82 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.820+0000 7f7eda000640 1 -- 172.21.15.97:0/3781512654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4030840 con 0x7f7edc15edf0 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7eda000640 1 -- 172.21.15.97:0/3781512654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec40309a0 con 0x7f7edc15edf0 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7eda000640 1 -- 172.21.15.97:0/3781512654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4030d30 con 0x7f7edc15edf0 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 -- 172.21.15.97:0/3781512654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 msgr2=0x7f7edc06eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 --2- 172.21.15.97:0/3781512654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc06eb10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009870 tx=0x7f7ec402b980 comp rx=0 tx=0).stop 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 -- 172.21.15.97:0/3781512654 shutdown_connections 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 --2- 172.21.15.97:0/3781512654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc06eb10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 --2- 172.21.15.97:0/3781512654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc06a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 -- 172.21.15.97:0/3781512654 >> 172.21.15.97:0/3781512654 conn(0x7f7edc0cc970 msgr2=0x7f7edc0ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 -- 172.21.15.97:0/3781512654 shutdown_connections 2024-01-31T05:01:01.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.821+0000 7f7ee2600640 1 -- 172.21.15.97:0/3781512654 wait complete. 2024-01-31T05:01:01.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.823+0000 7f7ee2600640 1 Processor -- start 2024-01-31T05:01:01.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.823+0000 7f7ee2600640 1 -- start start 2024-01-31T05:01:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.824+0000 7f7ee2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.824+0000 7f7ee2600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7edc0715a0 0x7f7edc0cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.824+0000 7f7ee2600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc09e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.824+0000 7f7ee2600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0c44d0 con 0x7f7edc15edf0 2024-01-31T05:01:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.824+0000 7f7ee2600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0c47b0 con 0x7f7edc0715a0 2024-01-31T05:01:01.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.824+0000 7f7ee2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0c4a90 con 0x7f7edc06fc70 2024-01-31T05:01:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc09e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc09e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43234/0 (socket says 172.21.15.97:43234) 2024-01-31T05:01:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 -- 172.21.15.97:0/2096342125 learned_addr learned my addr 172.21.15.97:0/2096342125 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edaa00640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7edc0715a0 0x7f7edc0cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edb400640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 -- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 msgr2=0x7f7edc0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc0b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 -- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7edc0715a0 msgr2=0x7f7edc0cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7edc0715a0 0x7f7edc0cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.825+0000 7f7edbe00640 1 -- 172.21.15.97:0/2096342125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec40094d0 con 0x7f7edc15edf0 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.826+0000 7f7edbe00640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc09e290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400c7a0 tx=0x7f7ed400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.826+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4003b70 con 0x7f7edc15edf0 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.826+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4003cf0 con 0x7f7edc15edf0 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.826+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc0c4d70 con 0x7f7edc15edf0 2024-01-31T05:01:01.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.826+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4003ea0 con 0x7f7edc15edf0 2024-01-31T05:01:01.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.826+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc0c4160 con 0x7f7edc15edf0 2024-01-31T05:01:01.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.827+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc0c5290 con 0x7f7edc15edf0 2024-01-31T05:01:01.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.828+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7ed400aa90 con 0x7f7edc15edf0 2024-01-31T05:01:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.829+0000 7f7ed3400640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7eb406aed0 0x7f7eb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.829+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7ed4089700 con 0x7f7edc15edf0 2024-01-31T05:01:01.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.829+0000 7f7edb400640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7eb406aed0 0x7f7eb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:01.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.830+0000 7f7edb400640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7eb406aed0 0x7f7eb406d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc006fd0 tx=0x7f7ecc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:01.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:01.834+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7ed40542f0 con 0x7f7edc15edf0 2024-01-31T05:01:02.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.029+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7edc067830 con 0x7f7edc15edf0 2024-01-31T05:01:02.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.030+0000 7f7ed3400640 1 -- 172.21.15.97:0/2096342125 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ed4057910 con 0x7f7edc15edf0 2024-01-31T05:01:02.029 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.032+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7eb406aed0 msgr2=0x7f7eb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.032+0000 7f7ee2600640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7eb406aed0 0x7f7eb406d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc006fd0 tx=0x7f7ecc008040 comp rx=0 tx=0).stop 2024-01-31T05:01:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.032+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 msgr2=0x7f7edc09e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.032+0000 7f7ee2600640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc09e290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400c7a0 tx=0x7f7ed400cc60 comp rx=0 tx=0).stop 2024-01-31T05:01:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 shutdown_connections 2024-01-31T05:01:02.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7eb406aed0 0x7f7eb406d390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7edc15edf0 0x7f7edc09e290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7edc0715a0 0x7f7edc0cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 --2- 172.21.15.97:0/2096342125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7edc06fc70 0x7f7edc0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 >> 172.21.15.97:0/2096342125 conn(0x7f7edc0cc970 msgr2=0x7f7edc0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 shutdown_connections 2024-01-31T05:01:02.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.033+0000 7f7ee2600640 1 -- 172.21.15.97:0/2096342125 wait complete. 2024-01-31T05:01:02.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:01:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.259+0000 7f8b55a00640 1 Processor -- start 2024-01-31T05:01:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.260+0000 7f8b55a00640 1 -- start start 2024-01-31T05:01:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.261+0000 7f8b55a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 0x7f8b500abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.261+0000 7f8b55a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5015abc0 con 0x7f8b500cde10 2024-01-31T05:01:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.261+0000 7f8b55a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5015aea0 con 0x7f8b5015a730 2024-01-31T05:01:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.261+0000 7f8b55a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500b2c90 con 0x7f8b5015a350 2024-01-31T05:01:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.261+0000 7f8b4f400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8b500cde10 0x7f8b500cbaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46338/0 (socket says 172.21.15.97:46338) 2024-01-31T05:01:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.261+0000 7f8b4f400640 1 -- 172.21.15.97:0/878432636 learned_addr learned my addr 172.21.15.97:0/878432636 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742074403 0 0) 0x7f8b5015abc0 con 0x7f8b500cde10 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b30005730 con 0x7f8b500cde10 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4ea00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 0x7f8b500abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1801003838 0 0) 0x7f8b30005730 con 0x7f8b500cde10 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 msgr2=0x7f8b500abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 0x7f8b500abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 >> v1:172.21.15.188:6789/0 conn(0x7f8b5015a730 legacy=0x7f8b5006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5015b180 con 0x7f8b500cde10 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f8b480019f0 con 0x7f8b500cde10 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 >> v1:172.21.15.97:6789/0 conn(0x7f8b500cde10 legacy=0x7f8b500cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 0x7f8b30005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300061e0 0x7f8b300069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b30006f10 0x7f8b30007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500b2c90 con 0x7f8b300061e0 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3000ddd0 con 0x7f8b30006f10 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.262+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3000df90 con 0x7f8b5015a350 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b45e00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300061e0 0x7f8b300069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b4ea00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 0x7f8b30005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b4f400640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b30006f10 0x7f8b30007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b45e00640 1 -- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 msgr2=0x7f8b30005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b45e00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b5015a350 0x7f8b30005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b45e00640 1 -- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b30006f10 msgr2=0x7f8b30007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b45e00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b30006f10 0x7f8b30007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.263+0000 7f8b45e00640 1 -- 172.21.15.97:0/878432636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30005730 con 0x7f8b300061e0 2024-01-31T05:01:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.264+0000 7f8b45e00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300061e0 0x7f8b300069d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8b38001750 tx=0x7f8b30007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf69d1291accec server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.264+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b380052d0 con 0x7f8b300061e0 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.264+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3802e050 con 0x7f8b300061e0 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.264+0000 7f8b4e000640 1 -- 172.21.15.97:0/878432636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38003830 con 0x7f8b300061e0 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b55a00640 1 -- 172.21.15.97:0/878432636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300061e0 msgr2=0x7f8b300069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b55a00640 1 --2- 172.21.15.97:0/878432636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b300061e0 0x7f8b300069d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8b38001750 tx=0x7f8b30007a00 comp rx=0 tx=0).stop 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b4f400640 1 -- 172.21.15.97:0/878432636 reap_dead start 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b55a00640 1 -- 172.21.15.97:0/878432636 shutdown_connections 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b55a00640 1 -- 172.21.15.97:0/878432636 >> 172.21.15.97:0/878432636 conn(0x7f8b500b0a10 msgr2=0x7f8b500b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b55a00640 1 -- 172.21.15.97:0/878432636 shutdown_connections 2024-01-31T05:01:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.265+0000 7f8b55a00640 1 -- 172.21.15.97:0/878432636 wait complete. 2024-01-31T05:01:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.267+0000 7f8b55a00640 1 Processor -- start 2024-01-31T05:01:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.267+0000 7f8b55a00640 1 -- start start 2024-01-31T05:01:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b55a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b300061e0 0x7f8b500ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b55a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b50157280 0x7f8b500999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b55a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 0x7f8b501589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b55a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50157700 con 0x7f8b50099f00 2024-01-31T05:01:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b55a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50158520 con 0x7f8b50157280 2024-01-31T05:01:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b55a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50157910 con 0x7f8b300061e0 2024-01-31T05:01:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.268+0000 7f8b4f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 0x7f8b501589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 0x7f8b501589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43250/0 (socket says 172.21.15.97:43250) 2024-01-31T05:01:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 -- 172.21.15.97:0/849027251 learned_addr learned my addr 172.21.15.97:0/849027251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b45e00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b50157280 0x7f8b500999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4ea00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b300061e0 0x7f8b500ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 -- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b300061e0 msgr2=0x7f8b500ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b300061e0 0x7f8b500ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 -- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b50157280 msgr2=0x7f8b500999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b50157280 0x7f8b500999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 -- 172.21.15.97:0/849027251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50157bf0 con 0x7f8b50099f00 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b4f400640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 0x7f8b501589b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b48001370 tx=0x7f8b4800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.269+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48004d60 con 0x7f8b50099f00 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.270+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4800bb80 con 0x7f8b50099f00 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.270+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48007070 con 0x7f8b50099f00 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.270+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b50157ed0 con 0x7f8b50099f00 2024-01-31T05:01:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.270+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b50153410 con 0x7f8b50099f00 2024-01-31T05:01:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.271+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b50158690 con 0x7f8b50099f00 2024-01-31T05:01:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.272+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8b480071d0 con 0x7f8b50099f00 2024-01-31T05:01:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.272+0000 7f8b47400640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8b2806af00 0x7f8b2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.273+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8b48089b70 con 0x7f8b50099f00 2024-01-31T05:01:02.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.273+0000 7f8b4ea00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8b2806af00 0x7f8b2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.273+0000 7f8b4ea00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8b2806af00 0x7f8b2806d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000ba80 tx=0x7f8b40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:02.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.277+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8b48054770 con 0x7f8b50099f00 2024-01-31T05:01:02.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.486+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8b50067830 con 0x7f8b50099f00 2024-01-31T05:01:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.486+0000 7f8b47400640 1 -- 172.21.15.97:0/849027251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8b48057d90 con 0x7f8b50099f00 2024-01-31T05:01:02.486 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:01:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8b2806af00 msgr2=0x7f8b2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8b2806af00 0x7f8b2806d3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000ba80 tx=0x7f8b40008040 comp rx=0 tx=0).stop 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 msgr2=0x7f8b501589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 0x7f8b501589b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b48001370 tx=0x7f8b4800b4a0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 shutdown_connections 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8b2806af00 0x7f8b2806d3c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b50099f00 0x7f8b501589b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b50157280 0x7f8b500999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 --2- 172.21.15.97:0/849027251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b300061e0 0x7f8b500ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 >> 172.21.15.97:0/849027251 conn(0x7f8b500b0a10 msgr2=0x7f8b500ac5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 shutdown_connections 2024-01-31T05:01:02.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.490+0000 7f8b55a00640 1 -- 172.21.15.97:0/849027251 wait complete. 2024-01-31T05:01:02.508 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:01:02.508 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:02.555 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.809+0000 7f5d5a600640 1 Processor -- start 2024-01-31T05:01:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.809+0000 7f5d5a600640 1 -- start start 2024-01-31T05:01:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d5a600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d540cb7d0 0x7f5d540b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d5a600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d540ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d5a600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5406e0d0 con 0x7f5d5406dc40 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d5a600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5406cba0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d5a600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5406ce80 con 0x7f5d5406c350 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d53400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d540cb7d0 0x7f5d540b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d53400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d540cb7d0 0x7f5d540b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43260/0 (socket says 172.21.15.97:43260) 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d53400640 1 -- 172.21.15.97:0/3776759069 learned_addr learned my addr 172.21.15.97:0/3776759069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.811+0000 7f5d53e00640 1 --2- 172.21.15.97:0/3776759069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d540ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.812+0000 7f5d53400640 1 -- 172.21.15.97:0/3776759069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 msgr2=0x7f5d540ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.812+0000 7f5d53400640 1 --2- 172.21.15.97:0/3776759069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d540ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.812+0000 7f5d53400640 1 -- 172.21.15.97:0/3776759069 >> v1:172.21.15.188:6789/0 conn(0x7f5d5406dc40 legacy=0x7f5d540cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.812+0000 7f5d53400640 1 -- 172.21.15.97:0/3776759069 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5406d160 con 0x7f5d540cb7d0 2024-01-31T05:01:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.813+0000 7f5d53400640 1 --2- 172.21.15.97:0/3776759069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d540b2e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009870 tx=0x7f5d4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c8586f9d2beadb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.813+0000 7f5d52000640 1 -- 172.21.15.97:0/3776759069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44031640 con 0x7f5d540cb7d0 2024-01-31T05:01:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.813+0000 7f5d52000640 1 -- 172.21.15.97:0/3776759069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440317a0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.813+0000 7f5d52000640 1 -- 172.21.15.97:0/3776759069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44031b30 con 0x7f5d540cb7d0 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 -- 172.21.15.97:0/3776759069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 msgr2=0x7f5d540b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 --2- 172.21.15.97:0/3776759069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d540b2e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009870 tx=0x7f5d4402b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 -- 172.21.15.97:0/3776759069 shutdown_connections 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 --2- 172.21.15.97:0/3776759069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d540ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 --2- 172.21.15.97:0/3776759069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d540b2e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 -- 172.21.15.97:0/3776759069 >> 172.21.15.97:0/3776759069 conn(0x7f5d540b0a10 msgr2=0x7f5d540b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 -- 172.21.15.97:0/3776759069 shutdown_connections 2024-01-31T05:01:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.814+0000 7f5d5a600640 1 -- 172.21.15.97:0/3776759069 wait complete. 2024-01-31T05:01:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.816+0000 7f5d5a600640 1 Processor -- start 2024-01-31T05:01:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.816+0000 7f5d5a600640 1 -- start start 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d5a600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d5415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d5a600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d5406dc40 0x7f5d54164100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d5a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d541601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d5a600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5415ece0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d5a600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5409a140 con 0x7f5d5406dc40 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d5a600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5415b9a0 con 0x7f5d5406c350 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d541601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d541601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43270/0 (socket says 172.21.15.97:43270) 2024-01-31T05:01:02.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 -- 172.21.15.97:0/2351652599 learned_addr learned my addr 172.21.15.97:0/2351652599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53400640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d5415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 -- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 msgr2=0x7f5d5415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d5415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d52a00640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d5406dc40 0x7f5d54164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 -- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d5406dc40 msgr2=0x7f5d54164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d5406dc40 0x7f5d54164100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53e00640 1 -- 172.21.15.97:0/2351652599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440094d0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.817+0000 7f5d53400640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d5415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.818+0000 7f5d53e00640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d541601a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c007e60 tx=0x7f5d4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.818+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c014420 con 0x7f5d540cb7d0 2024-01-31T05:01:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.818+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c0145a0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.818+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c014950 con 0x7f5d540cb7d0 2024-01-31T05:01:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.818+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5415ea00 con 0x7f5d540cb7d0 2024-01-31T05:01:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.818+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d54099dd0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.819+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5409a2b0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.820+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5d4c014ab0 con 0x7f5d540cb7d0 2024-01-31T05:01:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.821+0000 7f5d4b400640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d2c06aed0 0x7f5d2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.821+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5d4c08a260 con 0x7f5d540cb7d0 2024-01-31T05:01:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.822+0000 7f5d53400640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d2c06aed0 0x7f5d2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:02.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.822+0000 7f5d53400640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d2c06aed0 0x7f5d2c06d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d44036000 tx=0x7f5d4403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:02.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:02.826+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5d4c054e30 con 0x7f5d540cb7d0 2024-01-31T05:01:03.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.020+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d540ae1d0 con 0x7f5d540cb7d0 2024-01-31T05:01:03.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.023+0000 7f5d4b400640 1 -- 172.21.15.97:0/2351652599 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f5d4c058450 con 0x7f5d540cb7d0 2024-01-31T05:01:03.023 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:03.023 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d2c06aed0 msgr2=0x7f5d2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d2c06aed0 0x7f5d2c06d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d44036000 tx=0x7f5d4403c040 comp rx=0 tx=0).stop 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 msgr2=0x7f5d541601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d541601a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c007e60 tx=0x7f5d4c00e4b0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 shutdown_connections 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d2c06aed0 0x7f5d2c06d390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d540cb7d0 0x7f5d541601a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d5406dc40 0x7f5d54164100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 --2- 172.21.15.97:0/2351652599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5406c350 0x7f5d5415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.026+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 >> 172.21.15.97:0/2351652599 conn(0x7f5d540b0a10 msgr2=0x7f5d54165320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.027+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 shutdown_connections 2024-01-31T05:01:03.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.027+0000 7f5d5a600640 1 -- 172.21.15.97:0/2351652599 wait complete. 2024-01-31T05:01:03.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:03.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:03.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.255+0000 7f953aa00640 1 Processor -- start 2024-01-31T05:01:03.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.256+0000 7f953aa00640 1 -- start start 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.257+0000 7f953aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95340ce680 0x7f95340b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.257+0000 7f953aa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95340b3f60 con 0x7f95340ae360 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.257+0000 7f953aa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953415c540 con 0x7f953415b080 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.257+0000 7f953aa00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95340b41c0 con 0x7f95340ce680 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f953a000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f95340ae360 0x7f95340b21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46344/0 (socket says 172.21.15.97:46344) 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f953a000640 1 -- 172.21.15.97:0/2110559210 learned_addr learned my addr 172.21.15.97:0/2110559210 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:03.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f9538c00640 1 --2- 172.21.15.97:0/2110559210 >> v2:172.21.15.97:3300/0 conn(0x7f95340ce680 0x7f95340b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f9533e00640 1 -- 172.21.15.97:0/2110559210 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483487774 0 0) 0x7f95340b3f60 con 0x7f95340ae360 2024-01-31T05:01:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f9533e00640 1 -- 172.21.15.97:0/2110559210 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9514003520 con 0x7f95340ae360 2024-01-31T05:01:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f9538c00640 1 -- 172.21.15.97:0/2110559210 >> v1:172.21.15.188:6789/0 conn(0x7f953415b080 legacy=0x7f95340cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f9538c00640 1 -- 172.21.15.97:0/2110559210 >> v1:172.21.15.97:6789/0 conn(0x7f95340ae360 legacy=0x7f95340b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.258+0000 7f9538c00640 1 -- 172.21.15.97:0/2110559210 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953415b780 con 0x7f95340ce680 2024-01-31T05:01:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.259+0000 7f9538c00640 1 --2- 172.21.15.97:0/2110559210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95340ce680 0x7f95340b3a20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f951c0097a0 tx=0x7f951c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4559bd02f7aa799a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f9533e00640 1 -- 172.21.15.97:0/2110559210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c031830 con 0x7f95340ce680 2024-01-31T05:01:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f9533e00640 1 -- 172.21.15.97:0/2110559210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c031990 con 0x7f95340ce680 2024-01-31T05:01:03.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f9533e00640 1 -- 172.21.15.97:0/2110559210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c031d20 con 0x7f95340ce680 2024-01-31T05:01:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f953aa00640 1 -- 172.21.15.97:0/2110559210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95340ce680 msgr2=0x7f95340b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f953aa00640 1 --2- 172.21.15.97:0/2110559210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95340ce680 0x7f95340b3a20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f951c0097a0 tx=0x7f951c02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f953aa00640 1 -- 172.21.15.97:0/2110559210 shutdown_connections 2024-01-31T05:01:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f953aa00640 1 --2- 172.21.15.97:0/2110559210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95340ce680 0x7f95340b3a20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.260+0000 7f953aa00640 1 -- 172.21.15.97:0/2110559210 >> 172.21.15.97:0/2110559210 conn(0x7f953406ad40 msgr2=0x7f95340b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:03.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.261+0000 7f953aa00640 1 -- 172.21.15.97:0/2110559210 shutdown_connections 2024-01-31T05:01:03.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.261+0000 7f953aa00640 1 -- 172.21.15.97:0/2110559210 wait complete. 2024-01-31T05:01:03.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.262+0000 7f953aa00640 1 Processor -- start 2024-01-31T05:01:03.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.262+0000 7f953aa00640 1 -- start start 2024-01-31T05:01:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95340ae360 0x7f95341568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95340ce680 0x7f953415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 0x7f953415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9534154df0 con 0x7f953415b080 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95341f6d60 con 0x7f95340ae360 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95341f5e70 con 0x7f95340ce680 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.263+0000 7f953a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 0x7f953415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 0x7f953415e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43278/0 (socket says 172.21.15.97:43278) 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 -- 172.21.15.97:0/3682938223 learned_addr learned my addr 172.21.15.97:0/3682938223 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f9539600640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95340ae360 0x7f95341568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f9538c00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95340ce680 0x7f953415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 -- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95340ce680 msgr2=0x7f953415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95340ce680 0x7f953415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 -- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95340ae360 msgr2=0x7f95341568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95340ae360 0x7f95341568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 -- 172.21.15.97:0/3682938223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009400 con 0x7f953415b080 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f9538c00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95340ce680 0x7f953415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:03.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f953a000640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 0x7f953415e070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f952c00cae0 tx=0x7f952c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.264+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00ac40 con 0x7f953415b080 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.265+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c00ada0 con 0x7f953415b080 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.265+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c012600 con 0x7f953415b080 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.265+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95341f6150 con 0x7f953415b080 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.265+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95341f6710 con 0x7f953415b080 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.266+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9534153620 con 0x7f953415b080 2024-01-31T05:01:03.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.267+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f952c007070 con 0x7f953415b080 2024-01-31T05:01:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.268+0000 7f9531600640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f950c06aed0 0x7f950c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.268+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f952c0897f0 con 0x7f953415b080 2024-01-31T05:01:03.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.268+0000 7f9539600640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f950c06aed0 0x7f950c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.269+0000 7f9539600640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f950c06aed0 0x7f950c06d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9524007fd0 tx=0x7f9524009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:03.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.272+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9534153620 con 0x7f953415b080 2024-01-31T05:01:03.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.465+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9534153620 con 0x7f953415b080 2024-01-31T05:01:03.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.466+0000 7f9531600640 1 -- 172.21.15.97:0/3682938223 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9534153620 con 0x7f953415b080 2024-01-31T05:01:03.466 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f950c06aed0 msgr2=0x7f950c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f950c06aed0 0x7f950c06d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9524007fd0 tx=0x7f9524009040 comp rx=0 tx=0).stop 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 msgr2=0x7f953415e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 0x7f953415e070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f952c00cae0 tx=0x7f952c00a060 comp rx=0 tx=0).stop 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 shutdown_connections 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f950c06aed0 0x7f950c06d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953415b080 0x7f953415e070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95340ce680 0x7f953415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 --2- 172.21.15.97:0/3682938223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95340ae360 0x7f95341568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 >> 172.21.15.97:0/3682938223 conn(0x7f953406ad40 msgr2=0x7f95340b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:03.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 shutdown_connections 2024-01-31T05:01:03.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.469+0000 7f953aa00640 1 -- 172.21.15.97:0/3682938223 wait complete. 2024-01-31T05:01:03.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.700+0000 7f178ae00640 1 Processor -- start 2024-01-31T05:01:03.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.701+0000 7f178ae00640 1 -- start start 2024-01-31T05:01:03.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.702+0000 7f178ae00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f17840cfae0 0x7f17840cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.702+0000 7f178ae00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17840d0de0 0x7f17840d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.702+0000 7f178ae00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178415ec50 con 0x7f178415d590 2024-01-31T05:01:03.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.702+0000 7f178ae00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17840d0460 con 0x7f17840cfae0 2024-01-31T05:01:03.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.702+0000 7f178ae00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840cded0 con 0x7f17840d0de0 2024-01-31T05:01:03.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.703+0000 7f1783e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f178415d590 0x7f178415e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:40092/0 (socket says 172.21.15.97:40092) 2024-01-31T05:01:03.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.703+0000 7f1783e00640 1 -- 172.21.15.97:0/2839565304 learned_addr learned my addr 172.21.15.97:0/2839565304 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:03.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.703+0000 7f1783400640 1 --2- 172.21.15.97:0/2839565304 >> v2:172.21.15.188:3300/0 conn(0x7f17840cfae0 0x7f17840cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.703+0000 7f1788e00640 1 --2- 172.21.15.97:0/2839565304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17840d0de0 0x7f17840d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1782a00640 1 -- 172.21.15.97:0/2839565304 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806114105 0 0) 0x7f178415ec50 con 0x7f178415d590 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1782a00640 1 -- 172.21.15.97:0/2839565304 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1764003520 con 0x7f178415d590 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1788e00640 1 -- 172.21.15.97:0/2839565304 >> v2:172.21.15.188:3300/0 conn(0x7f17840cfae0 msgr2=0x7f17840cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1788e00640 1 --2- 172.21.15.97:0/2839565304 >> v2:172.21.15.188:3300/0 conn(0x7f17840cfae0 0x7f17840cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1788e00640 1 -- 172.21.15.97:0/2839565304 >> v1:172.21.15.188:6789/0 conn(0x7f178415d590 legacy=0x7f178415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1788e00640 1 -- 172.21.15.97:0/2839565304 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178415d970 con 0x7f17840d0de0 2024-01-31T05:01:03.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.704+0000 7f1783400640 1 --2- 172.21.15.97:0/2839565304 >> v2:172.21.15.188:3300/0 conn(0x7f17840cfae0 0x7f17840cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:03.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.706+0000 7f1788e00640 1 --2- 172.21.15.97:0/2839565304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17840d0de0 0x7f17840d1270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f177c0097a0 tx=0x7f177c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52cd3b26f49969ce server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:03.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.706+0000 7f1782a00640 1 -- 172.21.15.97:0/2839565304 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c030800 con 0x7f17840d0de0 2024-01-31T05:01:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.706+0000 7f1782a00640 1 -- 172.21.15.97:0/2839565304 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c030960 con 0x7f17840d0de0 2024-01-31T05:01:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.706+0000 7f1782a00640 1 -- 172.21.15.97:0/2839565304 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c030cd0 con 0x7f17840d0de0 2024-01-31T05:01:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 -- 172.21.15.97:0/2839565304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17840d0de0 msgr2=0x7f17840d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 --2- 172.21.15.97:0/2839565304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17840d0de0 0x7f17840d1270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f177c0097a0 tx=0x7f177c02b940 comp rx=0 tx=0).stop 2024-01-31T05:01:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 -- 172.21.15.97:0/2839565304 shutdown_connections 2024-01-31T05:01:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 --2- 172.21.15.97:0/2839565304 >> v2:172.21.15.188:3300/0 conn(0x7f17840cfae0 0x7f17840cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 --2- 172.21.15.97:0/2839565304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17840d0de0 0x7f17840d1270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 -- 172.21.15.97:0/2839565304 >> 172.21.15.97:0/2839565304 conn(0x7f178409c560 msgr2=0x7f17840d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 -- 172.21.15.97:0/2839565304 shutdown_connections 2024-01-31T05:01:03.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.707+0000 7f178ae00640 1 -- 172.21.15.97:0/2839565304 wait complete. 2024-01-31T05:01:03.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.709+0000 7f178ae00640 1 Processor -- start 2024-01-31T05:01:03.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.709+0000 7f178ae00640 1 -- start start 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f178ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 0x7f17840cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f178ae00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17840d0de0 0x7f178415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f178ae00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f178415d590 0x7f17840b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f178ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840b8970 con 0x7f17840cfae0 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f178ae00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840b7cf0 con 0x7f17840d0de0 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f178ae00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840b7fd0 con 0x7f178415d590 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 0x7f17840cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 0x7f17840cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43286/0 (socket says 172.21.15.97:43286) 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 -- 172.21.15.97:0/2439228102 learned_addr learned my addr 172.21.15.97:0/2439228102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1788e00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f178415d590 0x7f17840b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783400640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17840d0de0 0x7f178415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 -- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f178415d590 msgr2=0x7f17840b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f178415d590 0x7f17840b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 -- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17840d0de0 msgr2=0x7f178415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17840d0de0 0x7f178415b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.710+0000 7f1783e00640 1 -- 172.21.15.97:0/2439228102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c009a70 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.711+0000 7f1783e00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 0x7f17840cb670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f177400c390 tx=0x7f177400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.711+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774005ce0 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.711+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1774005e40 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.711+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740072f0 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.711+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177c009400 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.711+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17840b8590 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.712+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1784067830 con 0x7f17840cfae0 2024-01-31T05:01:03.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.713+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1774007450 con 0x7f17840cfae0 2024-01-31T05:01:03.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.714+0000 7f177be00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f175c06aed0 0x7f175c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.714+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1774088f10 con 0x7f17840cfae0 2024-01-31T05:01:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.714+0000 7f1783400640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f175c06aed0 0x7f175c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:03.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.715+0000 7f1783400640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f175c06aed0 0x7f175c06d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17840b3470 tx=0x7f176c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:03.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.719+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1774053ae0 con 0x7f17840cfae0 2024-01-31T05:01:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.913+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17840cea00 con 0x7f17840cfae0 2024-01-31T05:01:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.914+0000 7f177be00640 1 -- 172.21.15.97:0/2439228102 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17740170a0 con 0x7f17840cfae0 2024-01-31T05:01:03.914 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f175c06aed0 msgr2=0x7f175c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f175c06aed0 0x7f175c06d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17840b3470 tx=0x7f176c00b040 comp rx=0 tx=0).stop 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 msgr2=0x7f17840cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 0x7f17840cb670 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f177400c390 tx=0x7f177400c850 comp rx=0 tx=0).stop 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 shutdown_connections 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f178415d590 0x7f17840b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17840d0de0 0x7f178415b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f175c06aed0 0x7f175c06d390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 --2- 172.21.15.97:0/2439228102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17840cfae0 0x7f17840cb670 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 >> 172.21.15.97:0/2439228102 conn(0x7f178409c560 msgr2=0x7f178406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 shutdown_connections 2024-01-31T05:01:03.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:03.917+0000 7f178ae00640 1 -- 172.21.15.97:0/2439228102 wait complete. 2024-01-31T05:01:03.935 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.143+0000 7ff89a200640 1 Processor -- start 2024-01-31T05:01:04.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.143+0000 7ff89a200640 1 -- start start 2024-01-31T05:01:04.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff89a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 0x7ff8940cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff89a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff8940d1b90 0x7ff89415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff89a200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940cb760 con 0x7ff8940b2570 2024-01-31T05:01:04.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff89a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89406ce00 con 0x7ff8940d1b90 2024-01-31T05:01:04.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff89a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940cd4f0 con 0x7ff8940ce000 2024-01-31T05:01:04.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff893400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff8940b2570 0x7ff8940cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46354/0 (socket says 172.21.15.97:46354) 2024-01-31T05:01:04.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.144+0000 7ff893400640 1 -- 172.21.15.97:0/1277206191 learned_addr learned my addr 172.21.15.97:0/1277206191 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892a00640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 0x7ff8940cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff893e00640 1 --2- 172.21.15.97:0/1277206191 >> v2:172.21.15.188:3300/0 conn(0x7ff8940d1b90 0x7ff89415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010075803 0 0) 0x7ff8940cb760 con 0x7ff8940b2570 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff874003520 con 0x7ff8940b2570 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106890620 0 0) 0x7ff874003520 con 0x7ff8940b2570 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 msgr2=0x7ff8940cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:04.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 0x7ff8940cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 >> v2:172.21.15.188:3300/0 conn(0x7ff8940d1b90 msgr2=0x7ff89415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 --2- 172.21.15.97:0/1277206191 >> v2:172.21.15.188:3300/0 conn(0x7ff8940d1b90 0x7ff89415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940b2950 con 0x7ff8940b2570 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff884001670 con 0x7ff8940b2570 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 >> v1:172.21.15.97:6789/0 conn(0x7ff8940b2570 legacy=0x7ff8940cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff874003970 0x7ff874003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8740047c0 0x7ff874004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 0x7ff874005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89406ce00 con 0x7ff8740047c0 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940cd4f0 con 0x7ff874003970 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.145+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87400b980 con 0x7ff8940ce000 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff892a00640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8740047c0 0x7ff874004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff893400640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff874003970 0x7ff874003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff893e00640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 0x7ff874005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff892a00640 1 -- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 msgr2=0x7ff874005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:04.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff892a00640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8940ce000 0x7ff874005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff892a00640 1 -- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff874003970 msgr2=0x7ff874003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:04.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff892a00640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff874003970 0x7ff874003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.146+0000 7ff892a00640 1 -- 172.21.15.97:0/1277206191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874003520 con 0x7ff8740047c0 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.147+0000 7ff892a00640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8740047c0 0x7ff874004c40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009ac0 tx=0x7ff8740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bd5a1659d74cf8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.147+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c02c6b0 con 0x7ff8740047c0 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.147+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c02c810 con 0x7ff8740047c0 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.147+0000 7ff892000640 1 -- 172.21.15.97:0/1277206191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c02cba0 con 0x7ff8740047c0 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff89a200640 1 -- 172.21.15.97:0/1277206191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8740047c0 msgr2=0x7ff874004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff89a200640 1 --2- 172.21.15.97:0/1277206191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8740047c0 0x7ff874004c40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff87c009ac0 tx=0x7ff8740057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff893e00640 1 -- 172.21.15.97:0/1277206191 reap_dead start 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff89a200640 1 -- 172.21.15.97:0/1277206191 shutdown_connections 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff89a200640 1 -- 172.21.15.97:0/1277206191 >> 172.21.15.97:0/1277206191 conn(0x7ff8940d1070 msgr2=0x7ff8940d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:04.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff89a200640 1 -- 172.21.15.97:0/1277206191 shutdown_connections 2024-01-31T05:01:04.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.148+0000 7ff89a200640 1 -- 172.21.15.97:0/1277206191 wait complete. 2024-01-31T05:01:04.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.150+0000 7ff89a200640 1 Processor -- start 2024-01-31T05:01:04.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.150+0000 7ff89a200640 1 -- start start 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff89a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8740047c0 0x7ff8940c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff89a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940c8b50 0x7ff8941dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff89a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 0x7ff8941e5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff89a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940c9360 con 0x7ff8941e5310 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff89a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940c9640 con 0x7ff8940c8b50 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff89a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940c9920 con 0x7ff8740047c0 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.151+0000 7ff893e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 0x7ff8941e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 0x7ff8941e5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43312/0 (socket says 172.21.15.97:43312) 2024-01-31T05:01:04.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 -- 172.21.15.97:0/4040644331 learned_addr learned my addr 172.21.15.97:0/4040644331 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893400640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8740047c0 0x7ff8940c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff892a00640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940c8b50 0x7ff8941dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 -- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8740047c0 msgr2=0x7ff8940c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8740047c0 0x7ff8940c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 -- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940c8b50 msgr2=0x7ff8941dfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940c8b50 0x7ff8941dfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 -- 172.21.15.97:0/4040644331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940c9c00 con 0x7ff8941e5310 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff893e00640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 0x7ff8941e5790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8940cee50 tx=0x7ff88c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c00e070 con 0x7ff8941e5310 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.152+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c011e10 con 0x7ff8941e5310 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.153+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8940c9ee0 con 0x7ff8941e5310 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.153+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c003990 con 0x7ff8941e5310 2024-01-31T05:01:04.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.153+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8940c8fd0 con 0x7ff8941e5310 2024-01-31T05:01:04.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.154+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8940c7f00 con 0x7ff8941e5310 2024-01-31T05:01:04.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.155+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff88c003af0 con 0x7ff8941e5310 2024-01-31T05:01:04.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.156+0000 7ff88b400640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff86c06aed0 0x7ff86c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.156+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff88c089fd0 con 0x7ff8941e5310 2024-01-31T05:01:04.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.156+0000 7ff893400640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff86c06aed0 0x7ff86c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.156+0000 7ff893400640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff86c06aed0 0x7ff86c06d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8941e5100 tx=0x7ff884009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:04.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.160+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff88c054c40 con 0x7ff8941e5310 2024-01-31T05:01:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.352+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff894067830 con 0x7ff8941e5310 2024-01-31T05:01:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.353+0000 7ff88b400640 1 -- 172.21.15.97:0/4040644331 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff88c058260 con 0x7ff8941e5310 2024-01-31T05:01:04.353 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff86c06aed0 msgr2=0x7ff86c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff86c06aed0 0x7ff86c06d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff8941e5100 tx=0x7ff884009040 comp rx=0 tx=0).stop 2024-01-31T05:01:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 msgr2=0x7ff8941e5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 0x7ff8941e5790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff8940cee50 tx=0x7ff88c0115a0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 shutdown_connections 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff86c06aed0 0x7ff86c06d390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8941e5310 0x7ff8941e5790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8940c8b50 0x7ff8941dfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 --2- 172.21.15.97:0/4040644331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff8740047c0 0x7ff8940c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.356+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 >> 172.21.15.97:0/4040644331 conn(0x7ff8940d1070 msgr2=0x7ff8941e4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.357+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 shutdown_connections 2024-01-31T05:01:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.357+0000 7ff89a200640 1 -- 172.21.15.97:0/4040644331 wait complete. 2024-01-31T05:01:04.375 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:04.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.581+0000 7fbbd2400640 1 Processor -- start 2024-01-31T05:01:04.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.581+0000 7fbbd2400640 1 -- start start 2024-01-31T05:01:04.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.582+0000 7fbbd2400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbd2400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbcc0ce680 0x7fbbcc0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbd2400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc0b3f60 con 0x7fbbcc0ae360 2024-01-31T05:01:04.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbd2400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc0b41c0 con 0x7fbbcc0ce680 2024-01-31T05:01:04.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbd2400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc15c540 con 0x7fbbcc15b080 2024-01-31T05:01:04.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbd1a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbbcc0ce680 0x7fbbcc0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbcbe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbbcc0ae360 0x7fbbcc0cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46360/0 (socket says 172.21.15.97:46360) 2024-01-31T05:01:04.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbcbe00640 1 -- 172.21.15.97:0/3983459588 learned_addr learned my addr 172.21.15.97:0/3983459588 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbd1000640 1 --2- 172.21.15.97:0/3983459588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbcb400640 1 -- 172.21.15.97:0/3983459588 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898628558 0 0) 0x7fbbcc0b3f60 con 0x7fbbcc0ae360 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.583+0000 7fbbcb400640 1 -- 172.21.15.97:0/3983459588 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbac003520 con 0x7fbbcc0ae360 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.584+0000 7fbbd1a00640 1 -- 172.21.15.97:0/3983459588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 msgr2=0x7fbbcc0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.584+0000 7fbbd1a00640 1 --2- 172.21.15.97:0/3983459588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.584+0000 7fbbd1a00640 1 -- 172.21.15.97:0/3983459588 >> v1:172.21.15.97:6789/0 conn(0x7fbbcc0ae360 legacy=0x7fbbcc0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.584+0000 7fbbd1a00640 1 -- 172.21.15.97:0/3983459588 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc15b780 con 0x7fbbcc0ce680 2024-01-31T05:01:04.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.585+0000 7fbbd1a00640 1 --2- 172.21.15.97:0/3983459588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc0b21c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009d90 tx=0x7fbbc402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975dd5ec459a42ac server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:04.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.585+0000 7fbbcb400640 1 -- 172.21.15.97:0/3983459588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc402cc50 con 0x7fbbcc0ce680 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.585+0000 7fbbcb400640 1 -- 172.21.15.97:0/3983459588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc402cdb0 con 0x7fbbcc0ce680 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.585+0000 7fbbcb400640 1 -- 172.21.15.97:0/3983459588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4034640 con 0x7fbbcc0ce680 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 -- 172.21.15.97:0/3983459588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ce680 msgr2=0x7fbbcc0b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 --2- 172.21.15.97:0/3983459588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc0b21c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4009d90 tx=0x7fbbc402bea0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 -- 172.21.15.97:0/3983459588 shutdown_connections 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 --2- 172.21.15.97:0/3983459588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc0b21c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 --2- 172.21.15.97:0/3983459588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc0b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 -- 172.21.15.97:0/3983459588 >> 172.21.15.97:0/3983459588 conn(0x7fbbcc06ad40 msgr2=0x7fbbcc0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 -- 172.21.15.97:0/3983459588 shutdown_connections 2024-01-31T05:01:04.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.586+0000 7fbbd2400640 1 -- 172.21.15.97:0/3983459588 wait complete. 2024-01-31T05:01:04.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.588+0000 7fbbd2400640 1 Processor -- start 2024-01-31T05:01:04.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.588+0000 7fbbd2400640 1 -- start start 2024-01-31T05:01:04.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd2400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 0x7fbbcc06d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd2400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc09a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd2400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd2400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1f4000 con 0x7fbbcc0ae360 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd2400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1f4250 con 0x7fbbcc0ce680 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd2400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1f4530 con 0x7fbbcc15b080 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 0x7fbbcc06d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 0x7fbbcc06d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43322/0 (socket says 172.21.15.97:43322) 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd1000640 1 -- 172.21.15.97:0/2951438065 learned_addr learned my addr 172.21.15.97:0/2951438065 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.589+0000 7fbbd1a00640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbcbe00640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc09a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd1000640 1 -- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 msgr2=0x7fbbcc156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd1000640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd1000640 1 -- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbcc0ce680 msgr2=0x7fbbcc09a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd1000640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc09a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd1000640 1 -- 172.21.15.97:0/2951438065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4009a70 con 0x7fbbcc0ae360 2024-01-31T05:01:04.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd1000640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 0x7fbbcc06d940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e9b0 tx=0x7fbbbc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:04.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc009670 con 0x7fbbcc0ae360 2024-01-31T05:01:04.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc00c9a0 con 0x7fbbcc0ae360 2024-01-31T05:01:04.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc1f4810 con 0x7fbbcc0ae360 2024-01-31T05:01:04.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.590+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc00cd50 con 0x7fbbcc0ae360 2024-01-31T05:01:04.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.591+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc099bd0 con 0x7fbbcc0ae360 2024-01-31T05:01:04.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.592+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc0676b0 con 0x7fbbcc0ae360 2024-01-31T05:01:04.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.592+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fbbbc013470 con 0x7fbbcc0ae360 2024-01-31T05:01:04.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.593+0000 7fbbc8c00640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbba406aed0 0x7fbba406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:04.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.593+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbbbc089b00 con 0x7fbbcc0ae360 2024-01-31T05:01:04.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.594+0000 7fbbcbe00640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbba406aed0 0x7fbba406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:04.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.594+0000 7fbbcbe00640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbba406aed0 0x7fbba406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007fd0 tx=0x7fbbb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:04.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.597+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fbbbc054770 con 0x7fbbcc0ae360 2024-01-31T05:01:04.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.791+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbbcc1533f0 con 0x7fbbcc0ae360 2024-01-31T05:01:04.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.792+0000 7fbbc8c00640 1 -- 172.21.15.97:0/2951438065 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbbbc0170a0 con 0x7fbbcc0ae360 2024-01-31T05:01:04.791 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:04.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.794+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbba406aed0 msgr2=0x7fbba406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.794+0000 7fbbd2400640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbba406aed0 0x7fbba406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007fd0 tx=0x7fbbb4009040 comp rx=0 tx=0).stop 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.794+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 msgr2=0x7fbbcc06d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.794+0000 7fbbd2400640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 0x7fbbcc06d940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e9b0 tx=0x7fbbbc00ee70 comp rx=0 tx=0).stop 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 shutdown_connections 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbcc15b080 0x7fbbcc156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbcc0ce680 0x7fbbcc09a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbba406aed0 0x7fbba406d390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 --2- 172.21.15.97:0/2951438065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbbcc0ae360 0x7fbbcc06d940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 >> 172.21.15.97:0/2951438065 conn(0x7fbbcc06ad40 msgr2=0x7fbbcc153910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 shutdown_connections 2024-01-31T05:01:04.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:04.795+0000 7fbbd2400640 1 -- 172.21.15.97:0/2951438065 wait complete. 2024-01-31T05:01:04.813 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:01:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.017+0000 7f40dec00640 1 Processor -- start 2024-01-31T05:01:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.018+0000 7f40dec00640 1 -- start start 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.019+0000 7f40dec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80ce1e0 0x7f40d80b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.019+0000 7f40dec00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40d8161b10 0x7f40d80ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.019+0000 7f40dec00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d80ceb00 con 0x7f40d80cb7d0 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.019+0000 7f40dec00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d8162c40 con 0x7f40d8161b10 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.019+0000 7f40dec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8161fc0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40d8161b10 0x7f40d80ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40d8161b10 0x7f40d80ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42898/0 (socket says 172.21.15.97:42898) 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 -- 172.21.15.97:0/1292100683 learned_addr learned my addr 172.21.15.97:0/1292100683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7e00640 1 --2- 172.21.15.97:0/1292100683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80ce1e0 0x7f40d80b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 -- 172.21.15.97:0/1292100683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80ce1e0 msgr2=0x7f40d80b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 --2- 172.21.15.97:0/1292100683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80ce1e0 0x7f40d80b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 -- 172.21.15.97:0/1292100683 >> v1:172.21.15.188:6789/0 conn(0x7f40d80cb7d0 legacy=0x7f40d80cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.020+0000 7f40d7400640 1 -- 172.21.15.97:0/1292100683 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d81622a0 con 0x7f40d8161b10 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.021+0000 7f40d7400640 1 --2- 172.21.15.97:0/1292100683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d8161b10 0x7f40d80ce5c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009e50 tx=0x7f40c002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91278a20b11eef54 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40d6a00640 1 -- 172.21.15.97:0/1292100683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c002c950 con 0x7f40d8161b10 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40d6a00640 1 -- 172.21.15.97:0/1292100683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c002cab0 con 0x7f40d8161b10 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40d6a00640 1 -- 172.21.15.97:0/1292100683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c002ce40 con 0x7f40d8161b10 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 -- 172.21.15.97:0/1292100683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d8161b10 msgr2=0x7f40d80ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 --2- 172.21.15.97:0/1292100683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d8161b10 0x7f40d80ce5c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009e50 tx=0x7f40c002bdb0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 -- 172.21.15.97:0/1292100683 shutdown_connections 2024-01-31T05:01:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 --2- 172.21.15.97:0/1292100683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d8161b10 0x7f40d80ce5c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 --2- 172.21.15.97:0/1292100683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80ce1e0 0x7f40d80b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 -- 172.21.15.97:0/1292100683 >> 172.21.15.97:0/1292100683 conn(0x7f40d80b0a10 msgr2=0x7f40d80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 -- 172.21.15.97:0/1292100683 shutdown_connections 2024-01-31T05:01:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.022+0000 7f40dec00640 1 -- 172.21.15.97:0/1292100683 wait complete. 2024-01-31T05:01:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.024+0000 7f40dec00640 1 Processor -- start 2024-01-31T05:01:05.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.024+0000 7f40dec00640 1 -- start start 2024-01-31T05:01:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40dec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80cb7d0 0x7f40d8157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40dec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 0x7f40d8160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40dec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d8161b10 0x7f40d8163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40dec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d809aa00 con 0x7f40d80ce1e0 2024-01-31T05:01:05.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40dec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8099d80 con 0x7f40d8161b10 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40dec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d809a060 con 0x7f40d80cb7d0 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.025+0000 7f40d7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 0x7f40d8160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 0x7f40d8160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42914/0 (socket says 172.21.15.97:42914) 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 -- 172.21.15.97:0/2380723748 learned_addr learned my addr 172.21.15.97:0/2380723748 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7e00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80cb7d0 0x7f40d8157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 -- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80cb7d0 msgr2=0x7f40d8157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80cb7d0 0x7f40d8157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40dcc00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d8161b10 0x7f40d8163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 -- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d8161b10 msgr2=0x7f40d8163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d8161b10 0x7f40d8163f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 -- 172.21.15.97:0/2380723748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c0009b30 con 0x7f40d80ce1e0 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7e00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80cb7d0 0x7f40d8157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.026+0000 7f40d7400640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 0x7f40d8160c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009e50 tx=0x7f40c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:05.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.027+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c003e450 con 0x7f40d80ce1e0 2024-01-31T05:01:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.027+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c003e5b0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.027+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c003e960 con 0x7f40d80ce1e0 2024-01-31T05:01:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.027+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d809a340 con 0x7f40d80ce1e0 2024-01-31T05:01:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.027+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d8157560 con 0x7f40d80ce1e0 2024-01-31T05:01:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.028+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d809a1a0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.029+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f40c003eac0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.030+0000 7f40cfe00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f40b006aed0 0x7f40b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.030+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f40c00ac810 con 0x7f40d80ce1e0 2024-01-31T05:01:05.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.030+0000 7f40d7e00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f40b006aed0 0x7f40b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.031+0000 7f40d7e00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f40b006aed0 0x7f40b006d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009820 tx=0x7f40c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:05.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.035+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f40d809a1a0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.239+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f40d809a1a0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.240+0000 7f40cfe00640 1 -- 172.21.15.97:0/2380723748 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f40d809a1a0 con 0x7f40d80ce1e0 2024-01-31T05:01:05.240 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.242+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f40b006aed0 msgr2=0x7f40b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f40b006aed0 0x7f40b006d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009820 tx=0x7f40c8008040 comp rx=0 tx=0).stop 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 msgr2=0x7f40d8160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 0x7f40d8160c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009e50 tx=0x7f40c0004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 shutdown_connections 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40d8161b10 0x7f40d8163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f40b006aed0 0x7f40b006d390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80ce1e0 0x7f40d8160c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 --2- 172.21.15.97:0/2380723748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40d80cb7d0 0x7f40d8157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 >> 172.21.15.97:0/2380723748 conn(0x7f40d80b0a10 msgr2=0x7f40d80b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 shutdown_connections 2024-01-31T05:01:05.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.243+0000 7f40dec00640 1 -- 172.21.15.97:0/2380723748 wait complete. 2024-01-31T05:01:05.259 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:01:05.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:05.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-01-31T05:01:05.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi097.stderr:ceph-fuse[42983]: fuse finished with error 0 and tester_r 0 2024-01-31T05:01:05.346 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:01:05.379 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:01:05.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:05.380 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-01-31T05:01:05.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:05.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.695+0000 7f192d600640 1 Processor -- start 2024-01-31T05:01:05.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.695+0000 7f192d600640 1 -- start start 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.696+0000 7f192d600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19280b1d10 0x7f19280b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f192d600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f192815a600 0x7f192815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f192d600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280b2630 con 0x7f192815dc50 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f192d600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192806c8b0 con 0x7f19280b1d10 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f192d600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192815b120 con 0x7f192815a600 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19280b1d10 0x7f19280b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19280b1d10 0x7f19280b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42926/0 (socket says 172.21.15.97:42926) 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 -- 172.21.15.97:0/1259150387 learned_addr learned my addr 172.21.15.97:0/1259150387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 -- 172.21.15.97:0/1259150387 >> v2:172.21.15.188:3300/0 conn(0x7f192815a600 msgr2=0x7f192815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 --2- 172.21.15.97:0/1259150387 >> v2:172.21.15.188:3300/0 conn(0x7f192815a600 0x7f192815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 -- 172.21.15.97:0/1259150387 >> v1:172.21.15.188:6789/0 conn(0x7f192815dc50 legacy=0x7f192815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.697+0000 7f1926a00640 1 -- 172.21.15.97:0/1259150387 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192815b400 con 0x7f19280b1d10 2024-01-31T05:01:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.698+0000 7f1926a00640 1 --2- 172.21.15.97:0/1259150387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f19280b20f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1928158250 tx=0x7f191802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46587a301c1b30a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f1925600640 1 -- 172.21.15.97:0/1259150387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918031870 con 0x7f19280b1d10 2024-01-31T05:01:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f1925600640 1 -- 172.21.15.97:0/1259150387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19180319d0 con 0x7f19280b1d10 2024-01-31T05:01:05.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f1925600640 1 -- 172.21.15.97:0/1259150387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918031d60 con 0x7f19280b1d10 2024-01-31T05:01:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f192d600640 1 -- 172.21.15.97:0/1259150387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 msgr2=0x7f19280b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f192d600640 1 --2- 172.21.15.97:0/1259150387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f19280b20f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1928158250 tx=0x7f191802b910 comp rx=0 tx=0).stop 2024-01-31T05:01:05.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f192d600640 1 -- 172.21.15.97:0/1259150387 shutdown_connections 2024-01-31T05:01:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f192d600640 1 --2- 172.21.15.97:0/1259150387 >> v2:172.21.15.188:3300/0 conn(0x7f192815a600 0x7f192815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f192d600640 1 --2- 172.21.15.97:0/1259150387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f19280b20f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.699+0000 7f192d600640 1 -- 172.21.15.97:0/1259150387 >> 172.21.15.97:0/1259150387 conn(0x7f19280b0a10 msgr2=0x7f19280b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.700+0000 7f192d600640 1 -- 172.21.15.97:0/1259150387 shutdown_connections 2024-01-31T05:01:05.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.700+0000 7f192d600640 1 -- 172.21.15.97:0/1259150387 wait complete. 2024-01-31T05:01:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.701+0000 7f192d600640 1 Processor -- start 2024-01-31T05:01:05.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.701+0000 7f192d600640 1 -- start start 2024-01-31T05:01:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.702+0000 7f192d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f192815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.702+0000 7f192d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f192815a600 0x7f1928154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.702+0000 7f192d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192815dc50 0x7f192815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.702+0000 7f192d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192814c1b0 con 0x7f19280b1d10 2024-01-31T05:01:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.702+0000 7f192d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192815c830 con 0x7f192815a600 2024-01-31T05:01:05.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.702+0000 7f192d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192815bc20 con 0x7f192815dc50 2024-01-31T05:01:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f192815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f192815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42936/0 (socket says 172.21.15.97:42936) 2024-01-31T05:01:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 -- 172.21.15.97:0/2527470567 learned_addr learned my addr 172.21.15.97:0/2527470567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1927400640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192815dc50 0x7f192815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926000640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f192815a600 0x7f1928154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 -- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192815dc50 msgr2=0x7f192815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192815dc50 0x7f192815d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 -- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f192815a600 msgr2=0x7f1928154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f192815a600 0x7f1928154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.703+0000 7f1926a00640 1 -- 172.21.15.97:0/2527470567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180094d0 con 0x7f19280b1d10 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.704+0000 7f1926a00640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f192815b640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1918000c00 tx=0x7f19180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.704+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191803ad10 con 0x7f19280b1d10 2024-01-31T05:01:05.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.704+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191803ae70 con 0x7f19280b1d10 2024-01-31T05:01:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.704+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918044610 con 0x7f19280b1d10 2024-01-31T05:01:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.704+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19280adb80 con 0x7f19280b1d10 2024-01-31T05:01:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.704+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192815c1e0 con 0x7f19280b1d10 2024-01-31T05:01:05.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.706+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19280726a0 con 0x7f19280b1d10 2024-01-31T05:01:05.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.706+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f191804d450 con 0x7f19280b1d10 2024-01-31T05:01:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.707+0000 7f191ea00640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f190006aed0 0x7f190006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.707+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f191802d040 con 0x7f19280b1d10 2024-01-31T05:01:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.708+0000 7f1926000640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f190006aed0 0x7f190006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:05.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.708+0000 7f1926000640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f190006aed0 0x7f190006d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1928072670 tx=0x7f1910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:05.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.712+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1918076c30 con 0x7f19280b1d10 2024-01-31T05:01:05.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.907+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19280ac690 con 0x7f19280b1d10 2024-01-31T05:01:05.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.910+0000 7f191ea00640 1 -- 172.21.15.97:0/2527470567 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f191807a250 con 0x7f19280b1d10 2024-01-31T05:01:05.910 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:05.911 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f190006aed0 msgr2=0x7f190006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f190006aed0 0x7f190006d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1928072670 tx=0x7f1910008040 comp rx=0 tx=0).stop 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 msgr2=0x7f192815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f192815b640 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1918000c00 tx=0x7f19180046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 shutdown_connections 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192815dc50 0x7f192815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f192815a600 0x7f1928154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f190006aed0 0x7f190006d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 --2- 172.21.15.97:0/2527470567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19280b1d10 0x7f192815b640 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:05.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 >> 172.21.15.97:0/2527470567 conn(0x7f19280b0a10 msgr2=0x7f19280abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:05.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 shutdown_connections 2024-01-31T05:01:05.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:05.913+0000 7f192d600640 1 -- 172.21.15.97:0/2527470567 wait complete. 2024-01-31T05:01:05.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.141+0000 7f3783800640 1 Processor -- start 2024-01-31T05:01:06.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.141+0000 7f3783800640 1 -- start start 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f3783800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c0cb810 0x7f377c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f3783800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f377c15b400 0x7f377c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f3783800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377c06d1a0 con 0x7f377c06db70 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f3783800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377c0cc960 con 0x7f377c15b400 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f3783800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c06c600 con 0x7f377c0cb810 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f377be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f377c15b400 0x7f377c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f377be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f377c15b400 0x7f377c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42944/0 (socket says 172.21.15.97:42944) 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f377be00640 1 -- 172.21.15.97:0/3411476051 learned_addr learned my addr 172.21.15.97:0/3411476051 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:06.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.143+0000 7f3780e00640 1 --2- 172.21.15.97:0/3411476051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c0cb810 0x7f377c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.144+0000 7f377be00640 1 -- 172.21.15.97:0/3411476051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c0cb810 msgr2=0x7f377c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.144+0000 7f377be00640 1 --2- 172.21.15.97:0/3411476051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c0cb810 0x7f377c0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.144+0000 7f377be00640 1 -- 172.21.15.97:0/3411476051 >> v1:172.21.15.97:6789/0 conn(0x7f377c06db70 legacy=0x7f377c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.144+0000 7f377be00640 1 -- 172.21.15.97:0/3411476051 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c0cc130 con 0x7f377c15b400 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.144+0000 7f377b400640 1 -- 172.21.15.97:0/3411476051 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099791343 0 0) 0x7f377c06d1a0 con 0x7f377c06db70 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.145+0000 7f377be00640 1 --2- 172.21.15.97:0/3411476051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c06d630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3764009870 tx=0x7f376402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664b262e8644861d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.145+0000 7f377b400640 1 -- 172.21.15.97:0/3411476051 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764030840 con 0x7f377c15b400 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.145+0000 7f377b400640 1 -- 172.21.15.97:0/3411476051 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37640309a0 con 0x7f377c15b400 2024-01-31T05:01:06.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.145+0000 7f377b400640 1 -- 172.21.15.97:0/3411476051 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764030d30 con 0x7f377c15b400 2024-01-31T05:01:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 -- 172.21.15.97:0/3411476051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 msgr2=0x7f377c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 --2- 172.21.15.97:0/3411476051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c06d630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3764009870 tx=0x7f376402b980 comp rx=0 tx=0).stop 2024-01-31T05:01:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 -- 172.21.15.97:0/3411476051 shutdown_connections 2024-01-31T05:01:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 --2- 172.21.15.97:0/3411476051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c06d630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 --2- 172.21.15.97:0/3411476051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c0cb810 0x7f377c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 -- 172.21.15.97:0/3411476051 >> 172.21.15.97:0/3411476051 conn(0x7f377c0b0a10 msgr2=0x7f377c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:06.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 -- 172.21.15.97:0/3411476051 shutdown_connections 2024-01-31T05:01:06.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.146+0000 7f3783800640 1 -- 172.21.15.97:0/3411476051 wait complete. 2024-01-31T05:01:06.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.148+0000 7f3783800640 1 Processor -- start 2024-01-31T05:01:06.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.148+0000 7f3783800640 1 -- start start 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3783800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c06db70 0x7f377c0ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3783800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0cb810 0x7f377c0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3783800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3783800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1457d0 con 0x7f377c15b400 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3783800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c099f00 con 0x7f377c0cb810 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3783800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c147cc0 con 0x7f377c06db70 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3781800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3781800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42958/0 (socket says 172.21.15.97:42958) 2024-01-31T05:01:06.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3781800640 1 -- 172.21.15.97:0/3503946056 learned_addr learned my addr 172.21.15.97:0/3503946056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f3780e00640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c06db70 0x7f377c0ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.149+0000 7f377be00640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0cb810 0x7f377c0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3781800640 1 -- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c06db70 msgr2=0x7f377c0ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3781800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c06db70 0x7f377c0ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3781800640 1 -- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0cb810 msgr2=0x7f377c0999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3781800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0cb810 0x7f377c0999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3781800640 1 -- 172.21.15.97:0/3503946056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640094d0 con 0x7f377c15b400 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3781800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c147780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f377400c730 tx=0x7f377400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774003da0 con 0x7f377c15b400 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377400ce40 con 0x7f377c15b400 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37740052b0 con 0x7f377c15b400 2024-01-31T05:01:06.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c09a140 con 0x7f377c15b400 2024-01-31T05:01:06.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.150+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c09a700 con 0x7f377c15b400 2024-01-31T05:01:06.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.152+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c145910 con 0x7f377c15b400 2024-01-31T05:01:06.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.152+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3774005410 con 0x7f377c15b400 2024-01-31T05:01:06.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.153+0000 7f3778c00640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f375406aed0 0x7f375406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.153+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f37740896d0 con 0x7f377c15b400 2024-01-31T05:01:06.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.154+0000 7f3780e00640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f375406aed0 0x7f375406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.154+0000 7f3780e00640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f375406aed0 0x7f375406d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f376c009ad0 tx=0x7f376c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:06.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.158+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f377c145910 con 0x7f377c15b400 2024-01-31T05:01:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.350+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f377c145910 con 0x7f377c15b400 2024-01-31T05:01:06.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.352+0000 7f3778c00640 1 -- 172.21.15.97:0/3503946056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f377c145910 con 0x7f377c15b400 2024-01-31T05:01:06.352 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:06.352 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f375406aed0 msgr2=0x7f375406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f375406aed0 0x7f375406d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f376c009ad0 tx=0x7f376c008040 comp rx=0 tx=0).stop 2024-01-31T05:01:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 msgr2=0x7f377c147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c147780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f377400c730 tx=0x7f377400cbf0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 shutdown_connections 2024-01-31T05:01:06.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f375406aed0 0x7f375406d390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c15b400 0x7f377c147780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0cb810 0x7f377c0999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 --2- 172.21.15.97:0/3503946056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c06db70 0x7f377c0ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 >> 172.21.15.97:0/3503946056 conn(0x7f377c0b0a10 msgr2=0x7f377c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:06.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 shutdown_connections 2024-01-31T05:01:06.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.355+0000 7f3783800640 1 -- 172.21.15.97:0/3503946056 wait complete. 2024-01-31T05:01:06.373 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-31T05:01:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.590+0000 7f5ab8800640 1 Processor -- start 2024-01-31T05:01:06.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.590+0000 7f5ab8800640 1 -- start start 2024-01-31T05:01:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.591+0000 7f5ab8800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00cfca0 0x7f5ab00b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.591+0000 7f5ab8800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ab00d3bb0 0x7f5ab00b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.591+0000 7f5ab8800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab00d1b20 con 0x7f5ab00b3e90 2024-01-31T05:01:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.591+0000 7f5ab8800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab00d1e00 con 0x7f5ab00d3bb0 2024-01-31T05:01:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.591+0000 7f5ab8800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00d20e0 con 0x7f5ab00cfca0 2024-01-31T05:01:06.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ab00d3bb0 0x7f5ab00b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ab00d3bb0 0x7f5ab00b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42968/0 (socket says 172.21.15.97:42968) 2024-01-31T05:01:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 -- 172.21.15.97:0/556169174 learned_addr learned my addr 172.21.15.97:0/556169174 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5e00640 1 --2- 172.21.15.97:0/556169174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00cfca0 0x7f5ab00b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 -- 172.21.15.97:0/556169174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00cfca0 msgr2=0x7f5ab00b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 --2- 172.21.15.97:0/556169174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00cfca0 0x7f5ab00b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 -- 172.21.15.97:0/556169174 >> v1:172.21.15.97:6789/0 conn(0x7f5ab00b3e90 legacy=0x7f5ab00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.592+0000 7f5ab5400640 1 -- 172.21.15.97:0/556169174 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab00d4120 con 0x7f5ab00d3bb0 2024-01-31T05:01:06.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.593+0000 7f5ab5400640 1 --2- 172.21.15.97:0/556169174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00d3bb0 0x7f5ab00b3690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009e50 tx=0x7f5a9802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07879a1e914f88 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:06.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab4a00640 1 -- 172.21.15.97:0/556169174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9802c8d0 con 0x7f5ab00d3bb0 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab4a00640 1 -- 172.21.15.97:0/556169174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9802ca30 con 0x7f5ab00d3bb0 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab4a00640 1 -- 172.21.15.97:0/556169174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9802cdc0 con 0x7f5ab00d3bb0 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 -- 172.21.15.97:0/556169174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00d3bb0 msgr2=0x7f5ab00b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 --2- 172.21.15.97:0/556169174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00d3bb0 0x7f5ab00b3690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a98009e50 tx=0x7f5a9802bdb0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 -- 172.21.15.97:0/556169174 shutdown_connections 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 --2- 172.21.15.97:0/556169174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00d3bb0 0x7f5ab00b3690 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 --2- 172.21.15.97:0/556169174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00cfca0 0x7f5ab00b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 -- 172.21.15.97:0/556169174 >> 172.21.15.97:0/556169174 conn(0x7f5ab009c570 msgr2=0x7f5ab00d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.594+0000 7f5ab8800640 1 -- 172.21.15.97:0/556169174 shutdown_connections 2024-01-31T05:01:06.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.595+0000 7f5ab8800640 1 -- 172.21.15.97:0/556169174 wait complete. 2024-01-31T05:01:06.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.596+0000 7f5ab8800640 1 Processor -- start 2024-01-31T05:01:06.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.596+0000 7f5ab8800640 1 -- start start 2024-01-31T05:01:06.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab8800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 0x7f5ab00b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab8800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab00cfca0 0x7f5ab00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab8800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00d3bb0 0x7f5ab00cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab8800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00c8d40 con 0x7f5ab00b3e90 2024-01-31T05:01:06.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab8800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00c6dc0 con 0x7f5ab00cfca0 2024-01-31T05:01:06.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab8800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab00c7c20 con 0x7f5ab00d3bb0 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 0x7f5ab00b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.597+0000 7f5ab5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 0x7f5ab00b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42970/0 (socket says 172.21.15.97:42970) 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 -- 172.21.15.97:0/1344278038 learned_addr learned my addr 172.21.15.97:0/1344278038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5400640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab00cfca0 0x7f5ab00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab6800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00d3bb0 0x7f5ab00cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 -- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00d3bb0 msgr2=0x7f5ab00cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00d3bb0 0x7f5ab00cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 -- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab00cfca0 msgr2=0x7f5ab00c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab00cfca0 0x7f5ab00c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 -- 172.21.15.97:0/1344278038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009b30 con 0x7f5ab00b3e90 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5400640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab00cfca0 0x7f5ab00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab6800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00d3bb0 0x7f5ab00cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.598+0000 7f5ab5e00640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 0x7f5ab00b82d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000d800 tx=0x7f5aa000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:06.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.599+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000b6c0 con 0x7f5ab00b3e90 2024-01-31T05:01:06.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.599+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa000b820 con 0x7f5ab00b3e90 2024-01-31T05:01:06.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.599+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000bbb0 con 0x7f5ab00b3e90 2024-01-31T05:01:06.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.599+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab00c7010 con 0x7f5ab00b3e90 2024-01-31T05:01:06.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.599+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab00c75d0 con 0x7f5ab00b3e90 2024-01-31T05:01:06.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.600+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab0067830 con 0x7f5ab00b3e90 2024-01-31T05:01:06.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.601+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5aa000bd10 con 0x7f5ab00b3e90 2024-01-31T05:01:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.602+0000 7f5aae000640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a8806aed0 0x7f5a8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.602+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5aa0089ec0 con 0x7f5ab00b3e90 2024-01-31T05:01:06.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.602+0000 7f5ab5400640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a8806aed0 0x7f5a8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:06.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.603+0000 7f5ab5400640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a8806aed0 0x7f5a8806d390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5a98035000 tx=0x7f5a98030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:06.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.606+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5aa0054aa0 con 0x7f5ab00b3e90 2024-01-31T05:01:06.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:06.845+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5ab00d10f0 con 0x7f5ab00b3e90 2024-01-31T05:01:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.812+0000 7f5aae000640 1 -- 172.21.15.97:0/1344278038 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5aa00580c0 con 0x7f5ab00b3e90 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a8806aed0 msgr2=0x7f5a8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a8806aed0 0x7f5a8806d390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5a98035000 tx=0x7f5a98030000 comp rx=0 tx=0).stop 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 msgr2=0x7f5ab00b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 0x7f5ab00b82d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000d800 tx=0x7f5aa000dcc0 comp rx=0 tx=0).stop 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 shutdown_connections 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ab00d3bb0 0x7f5ab00cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ab00cfca0 0x7f5ab00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a8806aed0 0x7f5a8806d390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 --2- 172.21.15.97:0/1344278038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ab00b3e90 0x7f5ab00b82d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 >> 172.21.15.97:0/1344278038 conn(0x7f5ab009c570 msgr2=0x7f5ab00d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.816+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 shutdown_connections 2024-01-31T05:01:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:07.817+0000 7f5ab8800640 1 -- 172.21.15.97:0/1344278038 wait complete. 2024-01-31T05:01:07.837 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-01-31T05:01:07.838 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-31T05:01:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.053+0000 7fde6ca00640 1 Processor -- start 2024-01-31T05:01:08.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.053+0000 7fde6ca00640 1 -- start start 2024-01-31T05:01:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.054+0000 7fde6ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 0x7fde640722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.054+0000 7fde6ca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde640cbf20 0x7fde640ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.054+0000 7fde6ca00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6406cd20 con 0x7fde640abcf0 2024-01-31T05:01:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.054+0000 7fde6ca00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640ce970 con 0x7fde640cbf20 2024-01-31T05:01:08.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.054+0000 7fde6ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415b850 con 0x7fde6415b470 2024-01-31T05:01:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.055+0000 7fde6a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 0x7fde640722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.055+0000 7fde6a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 0x7fde640722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59568/0 (socket says 172.21.15.97:59568) 2024-01-31T05:01:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.055+0000 7fde6a000640 1 -- 172.21.15.97:0/1215755844 learned_addr learned my addr 172.21.15.97:0/1215755844 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.055+0000 7fde6aa00640 1 --2- 172.21.15.97:0/1215755844 >> v2:172.21.15.188:3300/0 conn(0x7fde640cbf20 0x7fde640ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.056+0000 7fde68c00640 1 -- 172.21.15.97:0/1215755844 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060218711 0 0) 0x7fde6406cd20 con 0x7fde640abcf0 2024-01-31T05:01:08.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.056+0000 7fde68c00640 1 -- 172.21.15.97:0/1215755844 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde44003520 con 0x7fde640abcf0 2024-01-31T05:01:08.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.056+0000 7fde6a000640 1 -- 172.21.15.97:0/1215755844 >> v2:172.21.15.188:3300/0 conn(0x7fde640cbf20 msgr2=0x7fde640ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:08.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.056+0000 7fde6a000640 1 --2- 172.21.15.97:0/1215755844 >> v2:172.21.15.188:3300/0 conn(0x7fde640cbf20 0x7fde640ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.056+0000 7fde6a000640 1 -- 172.21.15.97:0/1215755844 >> v1:172.21.15.188:6789/0 conn(0x7fde640abcf0 legacy=0x7fde640cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:08.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.056+0000 7fde6a000640 1 -- 172.21.15.97:0/1215755844 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640ceb60 con 0x7fde6415b470 2024-01-31T05:01:08.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.057+0000 7fde6a000640 1 --2- 172.21.15.97:0/1215755844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 0x7fde640722c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde54009880 tx=0x7fde5402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a8073e2046fdad1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.058+0000 7fde68c00640 1 -- 172.21.15.97:0/1215755844 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54004230 con 0x7fde6415b470 2024-01-31T05:01:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.058+0000 7fde68c00640 1 -- 172.21.15.97:0/1215755844 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54004390 con 0x7fde6415b470 2024-01-31T05:01:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.058+0000 7fde68c00640 1 -- 172.21.15.97:0/1215755844 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54003810 con 0x7fde6415b470 2024-01-31T05:01:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.058+0000 7fde6ca00640 1 -- 172.21.15.97:0/1215755844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 msgr2=0x7fde640722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.058+0000 7fde6ca00640 1 --2- 172.21.15.97:0/1215755844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 0x7fde640722c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde54009880 tx=0x7fde5402b960 comp rx=0 tx=0).stop 2024-01-31T05:01:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.059+0000 7fde6ca00640 1 -- 172.21.15.97:0/1215755844 shutdown_connections 2024-01-31T05:01:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.059+0000 7fde6ca00640 1 --2- 172.21.15.97:0/1215755844 >> v2:172.21.15.188:3300/0 conn(0x7fde640cbf20 0x7fde640ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.059+0000 7fde6ca00640 1 --2- 172.21.15.97:0/1215755844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6415b470 0x7fde640722c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.059+0000 7fde6ca00640 1 -- 172.21.15.97:0/1215755844 >> 172.21.15.97:0/1215755844 conn(0x7fde640b0a10 msgr2=0x7fde640b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.059+0000 7fde6ca00640 1 -- 172.21.15.97:0/1215755844 shutdown_connections 2024-01-31T05:01:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.059+0000 7fde6ca00640 1 -- 172.21.15.97:0/1215755844 wait complete. 2024-01-31T05:01:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.060+0000 7fde6ca00640 1 Processor -- start 2024-01-31T05:01:08.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.061+0000 7fde6ca00640 1 -- start start 2024-01-31T05:01:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde640abcf0 0x7fde6415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:08.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6ca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde640cbf20 0x7fde6409a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 0x7fde64156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64099dd0 con 0x7fde6415b470 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6ca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415c9a0 con 0x7fde640cbf20 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415c3e0 con 0x7fde640abcf0 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 0x7fde64156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 0x7fde64156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42984/0 (socket says 172.21.15.97:42984) 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 -- 172.21.15.97:0/2250364851 learned_addr learned my addr 172.21.15.97:0/2250364851 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6a000640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde640abcf0 0x7fde6415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde69600640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde640cbf20 0x7fde6409a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 -- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde640abcf0 msgr2=0x7fde6415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde640abcf0 0x7fde6415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 -- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde640cbf20 msgr2=0x7fde6409a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:08.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde640cbf20 0x7fde6409a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6aa00640 1 -- 172.21.15.97:0/2250364851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde54009460 con 0x7fde6415b470 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde6a000640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde640abcf0 0x7fde6415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.062+0000 7fde69600640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde640cbf20 0x7fde6409a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.063+0000 7fde6aa00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 0x7fde64156840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00efc0 tx=0x7fde5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.063+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c009ca0 con 0x7fde6415b470 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.063+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c00f040 con 0x7fde6415b470 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.063+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0146c0 con 0x7fde6415b470 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.063+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6415c6c0 con 0x7fde6415b470 2024-01-31T05:01:08.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.063+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde641516a0 con 0x7fde6415b470 2024-01-31T05:01:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.064+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6406d190 con 0x7fde6415b470 2024-01-31T05:01:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.065+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fde5c014820 con 0x7fde6415b470 2024-01-31T05:01:08.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.066+0000 7fde62000640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fde3c06aed0 0x7fde3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.066+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde5c08a960 con 0x7fde6415b470 2024-01-31T05:01:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.066+0000 7fde6a000640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fde3c06aed0 0x7fde3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:08.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.067+0000 7fde6a000640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fde3c06aed0 0x7fde3c06d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fde54009880 tx=0x7fde54008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:08.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.071+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fde6406d190 con 0x7fde6415b470 2024-01-31T05:01:08.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.301+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde6406d190 con 0x7fde6415b470 2024-01-31T05:01:08.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.859+0000 7fde62000640 1 -- 172.21.15.97:0/2250364851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde6406d190 con 0x7fde6415b470 2024-01-31T05:01:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.863+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fde3c06aed0 msgr2=0x7fde3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.863+0000 7fde6ca00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fde3c06aed0 0x7fde3c06d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fde54009880 tx=0x7fde54008030 comp rx=0 tx=0).stop 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.863+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 msgr2=0x7fde64156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.863+0000 7fde6ca00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 0x7fde64156840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00efc0 tx=0x7fde5c00c490 comp rx=0 tx=0).stop 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 shutdown_connections 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fde3c06aed0 0x7fde3c06d390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde6415b470 0x7fde64156840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde640cbf20 0x7fde6409a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 --2- 172.21.15.97:0/2250364851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde640abcf0 0x7fde6415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 >> 172.21.15.97:0/2250364851 conn(0x7fde640b0a10 msgr2=0x7fde640cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 shutdown_connections 2024-01-31T05:01:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:08.864+0000 7fde6ca00640 1 -- 172.21.15.97:0/2250364851 wait complete. 2024-01-31T05:01:08.884 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-31T05:01:08.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:01:09.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.093+0000 7f9562800640 1 Processor -- start 2024-01-31T05:01:09.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.093+0000 7f9562800640 1 -- start start 2024-01-31T05:01:09.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.095+0000 7f9562800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f955c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.095+0000 7f9562800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c15ea70 con 0x7f955c06e0a0 2024-01-31T05:01:09.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.095+0000 7f9562800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0b2630 con 0x7f955c15a600 2024-01-31T05:01:09.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.095+0000 7f9562800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06c8b0 con 0x7f955c0b1d10 2024-01-31T05:01:09.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.095+0000 7f955be00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f955c06e0a0 0x7f955c15e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41686/0 (socket says 172.21.15.97:41686) 2024-01-31T05:01:09.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.095+0000 7f955be00640 1 -- 172.21.15.97:0/1012008865 learned_addr learned my addr 172.21.15.97:0/1012008865 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:09.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955b400640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f955c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738299677 0 0) 0x7f955c15ea70 con 0x7f955c06e0a0 2024-01-31T05:01:09.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953c005730 con 0x7f955c06e0a0 2024-01-31T05:01:09.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942682221 0 0) 0x7f953c005730 con 0x7f955c06e0a0 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 msgr2=0x7f955c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f955c0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 >> v1:172.21.15.188:6789/0 conn(0x7f955c15a600 legacy=0x7f955c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c15b120 con 0x7f955c06e0a0 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f95540019f0 con 0x7f955c06e0a0 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 >> v1:172.21.15.97:6789/0 conn(0x7f955c06e0a0 legacy=0x7f955c15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c005bb0 0x7f953c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c0069d0 0x7f953c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f953c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c06c8b0 con 0x7f953c005bb0 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c00dc40 con 0x7f953c0069d0 2024-01-31T05:01:09.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.096+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c00de00 con 0x7f955c0b1d10 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955b400640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c005bb0 0x7f953c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955be00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f953c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955b400640 1 -- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 msgr2=0x7f953c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955b400640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f953c007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955b400640 1 -- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c0069d0 msgr2=0x7f953c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f9553e00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c0069d0 0x7f953c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955b400640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c0069d0 0x7f953c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955b400640 1 -- 172.21.15.97:0/1012008865 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c005730 con 0x7f953c005bb0 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f955be00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c0b1d10 0x7f953c007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:09.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.097+0000 7f9553e00640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c0069d0 0x7f953c006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.098+0000 7f955b400640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c005bb0 0x7f953c005fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f954c009a90 tx=0x7f953c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17e41f64a42a5f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.098+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c02c680 con 0x7f953c005bb0 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.098+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c02c7e0 con 0x7f953c005bb0 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.098+0000 7f955aa00640 1 -- 172.21.15.97:0/1012008865 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c02cb70 con 0x7f953c005bb0 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f9562800640 1 -- 172.21.15.97:0/1012008865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c005bb0 msgr2=0x7f953c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f9562800640 1 --2- 172.21.15.97:0/1012008865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f953c005bb0 0x7f953c005fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f954c009a90 tx=0x7f953c007a00 comp rx=0 tx=0).stop 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f955be00640 1 -- 172.21.15.97:0/1012008865 reap_dead start 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f9562800640 1 -- 172.21.15.97:0/1012008865 shutdown_connections 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f9562800640 1 -- 172.21.15.97:0/1012008865 >> 172.21.15.97:0/1012008865 conn(0x7f955c0b0a10 msgr2=0x7f955c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f9562800640 1 -- 172.21.15.97:0/1012008865 shutdown_connections 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.099+0000 7f9562800640 1 -- 172.21.15.97:0/1012008865 wait complete. 2024-01-31T05:01:09.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.101+0000 7f9562800640 1 Processor -- start 2024-01-31T05:01:09.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.101+0000 7f9562800640 1 -- start start 2024-01-31T05:01:09.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9562800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c005bb0 0x7f955c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9562800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 0x7f955c09a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9562800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c15db50 0x7f955c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9562800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c15e510 con 0x7f955c09a3d0 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9562800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c149910 con 0x7f953c005bb0 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9562800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c149510 con 0x7f955c15db50 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 0x7f955c09a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 0x7f955c09a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43006/0 (socket says 172.21.15.97:43006) 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 -- 172.21.15.97:0/3751545621 learned_addr learned my addr 172.21.15.97:0/3751545621 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f955be00640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c15db50 0x7f955c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 -- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c15db50 msgr2=0x7f955c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c15db50 0x7f955c15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 -- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c005bb0 msgr2=0x7f955c148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.102+0000 7f9553e00640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c005bb0 0x7f955c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9553e00640 1 -- 172.21.15.97:0/3751545621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0adb80 con 0x7f955c09a3d0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f955be00640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c15db50 0x7f955c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9553e00640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 0x7f955c09a850 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f954400d770 tx=0x7f954400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544005270 con 0x7f955c09a3d0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440053d0 con 0x7f955c09a3d0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400b890 con 0x7f955c09a3d0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c09b730 con 0x7f955c09a3d0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.103+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c09b170 con 0x7f955c09a3d0 2024-01-31T05:01:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.104+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c0677c0 con 0x7f955c09a3d0 2024-01-31T05:01:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.105+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f954400b9f0 con 0x7f955c09a3d0 2024-01-31T05:01:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.106+0000 7f9553400640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f953406af00 0x7f953406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.106+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9544089890 con 0x7f955c09a3d0 2024-01-31T05:01:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.106+0000 7f955b400640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f953406af00 0x7f953406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.107+0000 7f955b400640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f953406af00 0x7f953406d3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:09.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.110+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f95440543e0 con 0x7f955c09a3d0 2024-01-31T05:01:09.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.341+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f955c149a50 con 0x7f955c09a3d0 2024-01-31T05:01:09.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.342+0000 7f9553400640 1 -- 172.21.15.97:0/3751545621 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f9544019310 con 0x7f955c09a3d0 2024-01-31T05:01:09.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:09.341 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:01:09.342 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:01:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.344+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f953406af00 msgr2=0x7f953406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.344+0000 7f9562800640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f953406af00 0x7f953406d3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c02e000 comp rx=0 tx=0).stop 2024-01-31T05:01:09.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 msgr2=0x7f955c09a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 0x7f955c09a850 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f954400d770 tx=0x7f954400dc30 comp rx=0 tx=0).stop 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 shutdown_connections 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f955c15db50 0x7f955c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f953406af00 0x7f953406d3c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c09a3d0 0x7f955c09a850 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 --2- 172.21.15.97:0/3751545621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f953c005bb0 0x7f955c148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 >> 172.21.15.97:0/3751545621 conn(0x7f955c0b0a10 msgr2=0x7f955c0cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 shutdown_connections 2024-01-31T05:01:09.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.345+0000 7f9562800640 1 -- 172.21.15.97:0/3751545621 wait complete. 2024-01-31T05:01:09.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:09.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.570+0000 7ff953e00640 1 Processor -- start 2024-01-31T05:01:09.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.571+0000 7ff953e00640 1 -- start start 2024-01-31T05:01:09.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.572+0000 7ff953e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff94c15d620 0x7ff94c06b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff953e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c09a0d0 0x7ff94c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff953e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c0cf430 con 0x7ff94c06bd60 2024-01-31T05:01:09.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff953e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c1612a0 con 0x7ff94c15d620 2024-01-31T05:01:09.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff953e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c0cf660 con 0x7ff94c09a0d0 2024-01-31T05:01:09.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff950a00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff94c06bd60 0x7ff94c0999c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:54858/0 (socket says 172.21.15.97:54858) 2024-01-31T05:01:09.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff950a00640 1 -- 172.21.15.97:0/3920332178 learned_addr learned my addr 172.21.15.97:0/3920332178 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:09.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff951400640 1 --2- 172.21.15.97:0/3920332178 >> v2:172.21.15.188:3300/0 conn(0x7ff94c15d620 0x7ff94c06b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.573+0000 7ff951e00640 1 --2- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c09a0d0 0x7ff94c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.574+0000 7ff951400640 1 -- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c09a0d0 msgr2=0x7ff94c0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:09.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.574+0000 7ff951400640 1 --2- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c09a0d0 0x7ff94c0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.574+0000 7ff951400640 1 -- 172.21.15.97:0/3920332178 >> v1:172.21.15.188:6789/0 conn(0x7ff94c06bd60 legacy=0x7ff94c0999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.574+0000 7ff951400640 1 -- 172.21.15.97:0/3920332178 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c06c390 con 0x7ff94c15d620 2024-01-31T05:01:09.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.576+0000 7ff951400640 1 --2- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c15d620 0x7ff94c06b820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009d90 tx=0x7ff93c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa703a80b26366a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:09.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.576+0000 7ff94be00640 1 -- 172.21.15.97:0/3920332178 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c02cc50 con 0x7ff94c15d620 2024-01-31T05:01:09.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.576+0000 7ff94be00640 1 -- 172.21.15.97:0/3920332178 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c02cdb0 con 0x7ff94c15d620 2024-01-31T05:01:09.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.576+0000 7ff94be00640 1 -- 172.21.15.97:0/3920332178 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c034660 con 0x7ff94c15d620 2024-01-31T05:01:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 -- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c15d620 msgr2=0x7ff94c06b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 --2- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c15d620 0x7ff94c06b820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009d90 tx=0x7ff93c02bea0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 -- 172.21.15.97:0/3920332178 shutdown_connections 2024-01-31T05:01:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 --2- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c09a0d0 0x7ff94c0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 --2- 172.21.15.97:0/3920332178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c15d620 0x7ff94c06b820 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 -- 172.21.15.97:0/3920332178 >> 172.21.15.97:0/3920332178 conn(0x7ff94c0b2bb0 msgr2=0x7ff94c0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 -- 172.21.15.97:0/3920332178 shutdown_connections 2024-01-31T05:01:09.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.577+0000 7ff953e00640 1 -- 172.21.15.97:0/3920332178 wait complete. 2024-01-31T05:01:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.579+0000 7ff953e00640 1 Processor -- start 2024-01-31T05:01:09.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.579+0000 7ff953e00640 1 -- start start 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff953e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 0x7ff94c1432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff953e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c09a0d0 0x7ff94c144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff953e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c15d620 0x7ff94c146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff953e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c144490 con 0x7ff94c06bd60 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff953e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c143880 con 0x7ff94c09a0d0 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff953e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c143b60 con 0x7ff94c15d620 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff951400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 0x7ff94c1432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff951400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 0x7ff94c1432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43018/0 (socket says 172.21.15.97:43018) 2024-01-31T05:01:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.580+0000 7ff951400640 1 -- 172.21.15.97:0/1315958090 learned_addr learned my addr 172.21.15.97:0/1315958090 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff950a00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c09a0d0 0x7ff94c144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c15d620 0x7ff94c146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951400640 1 -- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c15d620 msgr2=0x7ff94c146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951400640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c15d620 0x7ff94c146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951400640 1 -- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c09a0d0 msgr2=0x7ff94c144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951400640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c09a0d0 0x7ff94c144920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951400640 1 -- 172.21.15.97:0/1315958090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c009a70 con 0x7ff94c06bd60 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff950a00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c09a0d0 0x7ff94c144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff951400640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 0x7ff94c1432a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009d90 tx=0x7ff93c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.581+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c040650 con 0x7ff94c06bd60 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.582+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c0407b0 con 0x7ff94c06bd60 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.582+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c040b60 con 0x7ff94c06bd60 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.582+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c143e40 con 0x7ff94c06bd60 2024-01-31T05:01:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.582+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c144e60 con 0x7ff94c06bd60 2024-01-31T05:01:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.583+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c06cc30 con 0x7ff94c06bd60 2024-01-31T05:01:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.584+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff93c040cc0 con 0x7ff94c06bd60 2024-01-31T05:01:09.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.585+0000 7ff949600640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff92406aed0 0x7ff92406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:09.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.586+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff93c0ac790 con 0x7ff94c06bd60 2024-01-31T05:01:09.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.586+0000 7ff950a00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff92406aed0 0x7ff92406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:09.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.586+0000 7ff950a00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff92406aed0 0x7ff92406d390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1f6c20 tx=0x7ff934009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:09.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.590+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff93c077480 con 0x7ff94c06bd60 2024-01-31T05:01:09.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.784+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff94c1450b0 con 0x7ff94c06bd60 2024-01-31T05:01:09.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.787+0000 7ff949600640 1 -- 172.21.15.97:0/1315958090 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7ff93c07aaa0 con 0x7ff94c06bd60 2024-01-31T05:01:09.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:09.787 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff92406aed0 msgr2=0x7ff92406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff92406aed0 0x7ff92406d390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff94c1f6c20 tx=0x7ff934009040 comp rx=0 tx=0).stop 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 msgr2=0x7ff94c1432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 0x7ff94c1432a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009d90 tx=0x7ff93c004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 shutdown_connections 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff94c15d620 0x7ff94c146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff94c09a0d0 0x7ff94c144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff92406aed0 0x7ff92406d390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 --2- 172.21.15.97:0/1315958090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff94c06bd60 0x7ff94c1432a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 >> 172.21.15.97:0/1315958090 conn(0x7ff94c0b2bb0 msgr2=0x7ff94c15f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 shutdown_connections 2024-01-31T05:01:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:09.790+0000 7ff953e00640 1 -- 172.21.15.97:0/1315958090 wait complete. 2024-01-31T05:01:09.809 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-31T05:01:09.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:01:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.014+0000 7fd50a200640 1 Processor -- start 2024-01-31T05:01:10.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.014+0000 7fd50a200640 1 -- start start 2024-01-31T05:01:10.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.015+0000 7fd50a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5040b1d10 0x7fd5040b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.015+0000 7fd50a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 0x7fd50415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.015+0000 7fd50a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5040b2630 con 0x7fd50415dc50 2024-01-31T05:01:10.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.015+0000 7fd50a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50406c8b0 con 0x7fd5040b1d10 2024-01-31T05:01:10.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.015+0000 7fd50a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50415b120 con 0x7fd50415a600 2024-01-31T05:01:10.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.016+0000 7fd4fa800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 0x7fd50415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.016+0000 7fd4fa800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 0x7fd50415aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59624/0 (socket says 172.21.15.97:59624) 2024-01-31T05:01:10.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.016+0000 7fd4fa800640 1 -- 172.21.15.97:0/1129988104 learned_addr learned my addr 172.21.15.97:0/1129988104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:10.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.016+0000 7fd503400640 1 --2- 172.21.15.97:0/1129988104 >> v2:172.21.15.188:3300/0 conn(0x7fd5040b1d10 0x7fd5040b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd502a00640 1 -- 172.21.15.97:0/1129988104 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584727473 0 0) 0x7fd5040b2630 con 0x7fd50415dc50 2024-01-31T05:01:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd502a00640 1 -- 172.21.15.97:0/1129988104 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e4003520 con 0x7fd50415dc50 2024-01-31T05:01:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd503400640 1 -- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 msgr2=0x7fd50415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd503400640 1 --2- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 0x7fd50415aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd503400640 1 -- 172.21.15.97:0/1129988104 >> v1:172.21.15.188:6789/0 conn(0x7fd50415dc50 legacy=0x7fd50415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd503400640 1 -- 172.21.15.97:0/1129988104 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50415b400 con 0x7fd5040b1d10 2024-01-31T05:01:10.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.017+0000 7fd4fa800640 1 --2- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 0x7fd50415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.018+0000 7fd503400640 1 --2- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5040b1d10 0x7fd5040b20f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40097a0 tx=0x7fd4f402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c1afdc34fc257f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.018+0000 7fd502a00640 1 -- 172.21.15.97:0/1129988104 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f402d040 con 0x7fd5040b1d10 2024-01-31T05:01:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.018+0000 7fd502a00640 1 -- 172.21.15.97:0/1129988104 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4031810 con 0x7fd5040b1d10 2024-01-31T05:01:10.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.018+0000 7fd502a00640 1 -- 172.21.15.97:0/1129988104 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4031ba0 con 0x7fd5040b1d10 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 -- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5040b1d10 msgr2=0x7fd5040b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 --2- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5040b1d10 0x7fd5040b20f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40097a0 tx=0x7fd4f402b8b0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 -- 172.21.15.97:0/1129988104 shutdown_connections 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 --2- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415a600 0x7fd50415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 --2- 172.21.15.97:0/1129988104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5040b1d10 0x7fd5040b20f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 -- 172.21.15.97:0/1129988104 >> 172.21.15.97:0/1129988104 conn(0x7fd5040b0a10 msgr2=0x7fd5040b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 -- 172.21.15.97:0/1129988104 shutdown_connections 2024-01-31T05:01:10.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.019+0000 7fd50a200640 1 -- 172.21.15.97:0/1129988104 wait complete. 2024-01-31T05:01:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.021+0000 7fd50a200640 1 Processor -- start 2024-01-31T05:01:10.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.021+0000 7fd50a200640 1 -- start start 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd50a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 0x7fd504155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd50a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50415a600 0x7fd5040999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd50a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415dc50 0x7fd50415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd50a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50409b520 con 0x7fd5040b1d10 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd50a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50409a480 con 0x7fd50415a600 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd50a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50409a760 con 0x7fd50415dc50 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd503400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 0x7fd504155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd503400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 0x7fd504155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43032/0 (socket says 172.21.15.97:43032) 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.022+0000 7fd503400640 1 -- 172.21.15.97:0/2432366154 learned_addr learned my addr 172.21.15.97:0/2432366154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd4fa800640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50415a600 0x7fd5040999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503e00640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415dc50 0x7fd50415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503400640 1 -- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415dc50 msgr2=0x7fd50415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503400640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415dc50 0x7fd50415c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503400640 1 -- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50415a600 msgr2=0x7fd5040999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503400640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50415a600 0x7fd5040999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503400640 1 -- 172.21.15.97:0/2432366154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec009b30 con 0x7fd5040b1d10 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd503400640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 0x7fd504155f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40098c0 tx=0x7fd4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f403ad60 con 0x7fd5040b1d10 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f40443f0 con 0x7fd5040b1d10 2024-01-31T05:01:10.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40445a0 con 0x7fd5040b1d10 2024-01-31T05:01:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.023+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f4009400 con 0x7fd5040b1d10 2024-01-31T05:01:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.024+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5040adb80 con 0x7fd5040b1d10 2024-01-31T05:01:10.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.025+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5040677c0 con 0x7fd5040b1d10 2024-01-31T05:01:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.026+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd4f404d450 con 0x7fd5040b1d10 2024-01-31T05:01:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.026+0000 7fd4fbe00640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd4dc06aed0 0x7fd4dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.026+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4f40abb10 con 0x7fd5040b1d10 2024-01-31T05:01:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.027+0000 7fd4fa800640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd4dc06aed0 0x7fd4dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.027+0000 7fd4fa800640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd4dc06aed0 0x7fd4dc06d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5040728a0 tx=0x7fd4ec00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:10.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.031+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd4f4076810 con 0x7fd5040b1d10 2024-01-31T05:01:10.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.221+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5040ada00 con 0x7fd4dc06aed0 2024-01-31T05:01:10.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.235+0000 7fd4fbe00640 1 -- 172.21.15.97:0/2432366154 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345107 (secure 0 0 0) 0x7fd4ec00f040 con 0x7fd4dc06aed0 2024-01-31T05:01:10.239 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:10.241 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:01:10.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.240+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd4dc06aed0 msgr2=0x7fd4dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.240+0000 7fd50a200640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd4dc06aed0 0x7fd4dc06d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5040728a0 tx=0x7fd4ec00b040 comp rx=0 tx=0).stop 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 msgr2=0x7fd504155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 0x7fd504155f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40098c0 tx=0x7fd4f40046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 shutdown_connections 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50415dc50 0x7fd50415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50415a600 0x7fd5040999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd4dc06aed0 0x7fd4dc06d390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 --2- 172.21.15.97:0/2432366154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5040b1d10 0x7fd504155f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 >> 172.21.15.97:0/2432366154 conn(0x7fd5040b0a10 msgr2=0x7fd5040ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 shutdown_connections 2024-01-31T05:01:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.241+0000 7fd50a200640 1 -- 172.21.15.97:0/2432366154 wait complete. 2024-01-31T05:01:10.258 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-01-31T05:01:09.557111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2038292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679907+0000","last_change":"2024-01-31T05:00:45.571880+0000","last_active":"2024-01-31T05:00:51.679907+0000","last_peered":"2024-01-31T05:00:51.679907+0000","last_clean":"2024-01-31T05:00:51.679907+0000","last_became_active":"2024-01-31T05:00:45.571633+0000","last_became_peered":"2024-01-31T05:00:45.571633+0000","last_unstale":"2024-01-31T05:00:51.679907+0000","last_undegraded":"2024-01-31T05:00:51.679907+0000","last_fullsized":"2024-01-31T05:00:51.679907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681114+0000","last_change":"2024-01-31T05:00:46.593889+0000","last_active":"2024-01-31T05:00:51.681114+0000","last_peered":"2024-01-31T05:00:51.681114+0000","last_clean":"2024-01-31T05:00:51.681114+0000","last_became_active":"2024-01-31T05:00:46.593583+0000","last_became_peered":"2024-01-31T05:00:46.593583+0000","last_unstale":"2024-01-31T05:00:51.681114+0000","last_undegraded":"2024-01-31T05:00:51.681114+0000","last_fullsized":"2024-01-31T05:00:51.681114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914795+0000","last_change":"2024-01-31T05:00:50.653143+0000","last_active":"2024-01-31T05:00:51.914795+0000","last_peered":"2024-01-31T05:00:51.914795+0000","last_clean":"2024-01-31T05:00:51.914795+0000","last_became_active":"2024-01-31T05:00:50.652831+0000","last_became_peered":"2024-01-31T05:00:50.652831+0000","last_unstale":"2024-01-31T05:00:51.914795+0000","last_undegraded":"2024-01-31T05:00:51.914795+0000","last_fullsized":"2024-01-31T05:00:51.914795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:44.981765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681288+0000","last_change":"2024-01-31T05:00:45.578379+0000","last_active":"2024-01-31T05:00:51.681288+0000","last_peered":"2024-01-31T05:00:51.681288+0000","last_clean":"2024-01-31T05:00:51.681288+0000","last_became_active":"2024-01-31T05:00:45.574393+0000","last_became_peered":"2024-01-31T05:00:45.574393+0000","last_unstale":"2024-01-31T05:00:51.681288+0000","last_undegraded":"2024-01-31T05:00:51.681288+0000","last_fullsized":"2024-01-31T05:00:51.681288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681650+0000","last_change":"2024-01-31T05:00:46.573630+0000","last_active":"2024-01-31T05:00:51.681650+0000","last_peered":"2024-01-31T05:00:51.681650+0000","last_clean":"2024-01-31T05:00:51.681650+0000","last_became_active":"2024-01-31T05:00:46.572199+0000","last_became_peered":"2024-01-31T05:00:46.572199+0000","last_unstale":"2024-01-31T05:00:51.681650+0000","last_undegraded":"2024-01-31T05:00:51.681650+0000","last_fullsized":"2024-01-31T05:00:51.681650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912863+0000","last_change":"2024-01-31T05:00:50.644804+0000","last_active":"2024-01-31T05:00:51.912863+0000","last_peered":"2024-01-31T05:00:51.912863+0000","last_clean":"2024-01-31T05:00:51.912863+0000","last_became_active":"2024-01-31T05:00:50.640815+0000","last_became_peered":"2024-01-31T05:00:50.640815+0000","last_unstale":"2024-01-31T05:00:51.912863+0000","last_undegraded":"2024-01-31T05:00:51.912863+0000","last_fullsized":"2024-01-31T05:00:51.912863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:03:57.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915593+0000","last_change":"2024-01-31T05:00:45.582233+0000","last_active":"2024-01-31T05:00:51.915593+0000","last_peered":"2024-01-31T05:00:51.915593+0000","last_clean":"2024-01-31T05:00:51.915593+0000","last_became_active":"2024-01-31T05:00:45.578094+0000","last_became_peered":"2024-01-31T05:00:45.578094+0000","last_unstale":"2024-01-31T05:00:51.915593+0000","last_undegraded":"2024-01-31T05:00:51.915593+0000","last_fullsized":"2024-01-31T05:00:51.915593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:21:00.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913671+0000","last_change":"2024-01-31T05:00:46.589171+0000","last_active":"2024-01-31T05:00:51.913671+0000","last_peered":"2024-01-31T05:00:51.913671+0000","last_clean":"2024-01-31T05:00:51.913671+0000","last_became_active":"2024-01-31T05:00:46.588749+0000","last_became_peered":"2024-01-31T05:00:46.588749+0000","last_unstale":"2024-01-31T05:00:51.913671+0000","last_undegraded":"2024-01-31T05:00:51.913671+0000","last_fullsized":"2024-01-31T05:00:51.913671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914061+0000","last_change":"2024-01-31T05:00:50.635774+0000","last_active":"2024-01-31T05:00:51.914061+0000","last_peered":"2024-01-31T05:00:51.914061+0000","last_clean":"2024-01-31T05:00:51.914061+0000","last_became_active":"2024-01-31T05:00:50.635404+0000","last_became_peered":"2024-01-31T05:00:50.635404+0000","last_unstale":"2024-01-31T05:00:51.914061+0000","last_undegraded":"2024-01-31T05:00:51.914061+0000","last_fullsized":"2024-01-31T05:00:51.914061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:01:05.575230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915820+0000","last_change":"2024-01-31T05:00:45.583051+0000","last_active":"2024-01-31T05:00:51.915820+0000","last_peered":"2024-01-31T05:00:51.915820+0000","last_clean":"2024-01-31T05:00:51.915820+0000","last_became_active":"2024-01-31T05:00:45.582464+0000","last_became_peered":"2024-01-31T05:00:45.582464+0000","last_unstale":"2024-01-31T05:00:51.915820+0000","last_undegraded":"2024-01-31T05:00:51.915820+0000","last_fullsized":"2024-01-31T05:00:51.915820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:46:05.159804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674284+0000","last_change":"2024-01-31T05:00:46.593796+0000","last_active":"2024-01-31T05:00:51.674284+0000","last_peered":"2024-01-31T05:00:51.674284+0000","last_clean":"2024-01-31T05:00:51.674284+0000","last_became_active":"2024-01-31T05:00:46.593485+0000","last_became_peered":"2024-01-31T05:00:46.593485+0000","last_unstale":"2024-01-31T05:00:51.674284+0000","last_undegraded":"2024-01-31T05:00:51.674284+0000","last_fullsized":"2024-01-31T05:00:51.674284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:51:45.113263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912620+0000","last_change":"2024-01-31T05:00:50.645582+0000","last_active":"2024-01-31T05:00:51.912620+0000","last_peered":"2024-01-31T05:00:51.912620+0000","last_clean":"2024-01-31T05:00:51.912620+0000","last_became_active":"2024-01-31T05:00:50.645198+0000","last_became_peered":"2024-01-31T05:00:50.645198+0000","last_unstale":"2024-01-31T05:00:51.912620+0000","last_undegraded":"2024-01-31T05:00:51.912620+0000","last_fullsized":"2024-01-31T05:00:51.912620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:07:24.039515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673371+0000","last_change":"2024-01-31T05:00:46.581785+0000","last_active":"2024-01-31T05:00:51.673371+0000","last_peered":"2024-01-31T05:00:51.673371+0000","last_clean":"2024-01-31T05:00:51.673371+0000","last_became_active":"2024-01-31T05:00:46.581463+0000","last_became_peered":"2024-01-31T05:00:46.581463+0000","last_unstale":"2024-01-31T05:00:51.673371+0000","last_undegraded":"2024-01-31T05:00:51.673371+0000","last_fullsized":"2024-01-31T05:00:51.673371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:24:26.658435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673762+0000","last_change":"2024-01-31T05:00:45.566782+0000","last_active":"2024-01-31T05:00:51.673762+0000","last_peered":"2024-01-31T05:00:51.673762+0000","last_clean":"2024-01-31T05:00:51.673762+0000","last_became_active":"2024-01-31T05:00:45.565360+0000","last_became_peered":"2024-01-31T05:00:45.565360+0000","last_unstale":"2024-01-31T05:00:51.673762+0000","last_undegraded":"2024-01-31T05:00:51.673762+0000","last_fullsized":"2024-01-31T05:00:51.673762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:45.457514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674294+0000","last_change":"2024-01-31T05:00:50.639468+0000","last_active":"2024-01-31T05:00:51.674294+0000","last_peered":"2024-01-31T05:00:51.674294+0000","last_clean":"2024-01-31T05:00:51.674294+0000","last_became_active":"2024-01-31T05:00:50.639035+0000","last_became_peered":"2024-01-31T05:00:50.639035+0000","last_unstale":"2024-01-31T05:00:51.674294+0000","last_undegraded":"2024-01-31T05:00:51.674294+0000","last_fullsized":"2024-01-31T05:00:51.674294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:43:03.844139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915694+0000","last_change":"2024-01-31T05:00:46.583107+0000","last_active":"2024-01-31T05:00:51.915694+0000","last_peered":"2024-01-31T05:00:51.915694+0000","last_clean":"2024-01-31T05:00:51.915694+0000","last_became_active":"2024-01-31T05:00:46.582506+0000","last_became_peered":"2024-01-31T05:00:46.582506+0000","last_unstale":"2024-01-31T05:00:51.915694+0000","last_undegraded":"2024-01-31T05:00:51.915694+0000","last_fullsized":"2024-01-31T05:00:51.915694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917300+0000","last_change":"2024-01-31T05:00:45.585334+0000","last_active":"2024-01-31T05:00:51.917300+0000","last_peered":"2024-01-31T05:00:51.917300+0000","last_clean":"2024-01-31T05:00:51.917300+0000","last_became_active":"2024-01-31T05:00:45.585034+0000","last_became_peered":"2024-01-31T05:00:45.585034+0000","last_unstale":"2024-01-31T05:00:51.917300+0000","last_undegraded":"2024-01-31T05:00:51.917300+0000","last_fullsized":"2024-01-31T05:00:51.917300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915319+0000","last_change":"2024-01-31T05:00:50.642526+0000","last_active":"2024-01-31T05:00:51.915319+0000","last_peered":"2024-01-31T05:00:51.915319+0000","last_clean":"2024-01-31T05:00:51.915319+0000","last_became_active":"2024-01-31T05:00:50.642120+0000","last_became_peered":"2024-01-31T05:00:50.642120+0000","last_unstale":"2024-01-31T05:00:51.915319+0000","last_undegraded":"2024-01-31T05:00:51.915319+0000","last_fullsized":"2024-01-31T05:00:51.915319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674674+0000","last_change":"2024-01-31T05:00:46.585812+0000","last_active":"2024-01-31T05:00:51.674674+0000","last_peered":"2024-01-31T05:00:51.674674+0000","last_clean":"2024-01-31T05:00:51.674674+0000","last_became_active":"2024-01-31T05:00:46.585556+0000","last_became_peered":"2024-01-31T05:00:46.585556+0000","last_unstale":"2024-01-31T05:00:51.674674+0000","last_undegraded":"2024-01-31T05:00:51.674674+0000","last_fullsized":"2024-01-31T05:00:51.674674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:37.158256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676683+0000","last_change":"2024-01-31T05:00:45.580944+0000","last_active":"2024-01-31T05:00:51.676683+0000","last_peered":"2024-01-31T05:00:51.676683+0000","last_clean":"2024-01-31T05:00:51.676683+0000","last_became_active":"2024-01-31T05:00:45.580698+0000","last_became_peered":"2024-01-31T05:00:45.580698+0000","last_unstale":"2024-01-31T05:00:51.676683+0000","last_undegraded":"2024-01-31T05:00:51.676683+0000","last_fullsized":"2024-01-31T05:00:51.676683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:57.421278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916224+0000","last_change":"2024-01-31T05:00:50.659576+0000","last_active":"2024-01-31T05:00:51.916224+0000","last_peered":"2024-01-31T05:00:51.916224+0000","last_clean":"2024-01-31T05:00:51.916224+0000","last_became_active":"2024-01-31T05:00:50.659067+0000","last_became_peered":"2024-01-31T05:00:50.659067+0000","last_unstale":"2024-01-31T05:00:51.916224+0000","last_undegraded":"2024-01-31T05:00:51.916224+0000","last_fullsized":"2024-01-31T05:00:51.916224+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:11.777909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673016+0000","last_change":"2024-01-31T05:00:46.586716+0000","last_active":"2024-01-31T05:00:51.673016+0000","last_peered":"2024-01-31T05:00:51.673016+0000","last_clean":"2024-01-31T05:00:51.673016+0000","last_became_active":"2024-01-31T05:00:46.582889+0000","last_became_peered":"2024-01-31T05:00:46.582889+0000","last_unstale":"2024-01-31T05:00:51.673016+0000","last_undegraded":"2024-01-31T05:00:51.673016+0000","last_fullsized":"2024-01-31T05:00:51.673016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:07.424800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.572569+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.571841+0000","last_became_peered":"2024-01-31T05:00:45.571841+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914775+0000","last_change":"2024-01-31T05:00:50.637862+0000","last_active":"2024-01-31T05:00:51.914775+0000","last_peered":"2024-01-31T05:00:51.914775+0000","last_clean":"2024-01-31T05:00:51.914775+0000","last_became_active":"2024-01-31T05:00:50.637439+0000","last_became_peered":"2024-01-31T05:00:50.637439+0000","last_unstale":"2024-01-31T05:00:51.914775+0000","last_undegraded":"2024-01-31T05:00:51.914775+0000","last_fullsized":"2024-01-31T05:00:51.914775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676808+0000","last_change":"2024-01-31T05:00:46.588199+0000","last_active":"2024-01-31T05:00:51.676808+0000","last_peered":"2024-01-31T05:00:51.676808+0000","last_clean":"2024-01-31T05:00:51.676808+0000","last_became_active":"2024-01-31T05:00:46.585236+0000","last_became_peered":"2024-01-31T05:00:46.585236+0000","last_unstale":"2024-01-31T05:00:51.676808+0000","last_undegraded":"2024-01-31T05:00:51.676808+0000","last_fullsized":"2024-01-31T05:00:51.676808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:16.863565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675061+0000","last_change":"2024-01-31T05:00:45.577787+0000","last_active":"2024-01-31T05:00:51.675061+0000","last_peered":"2024-01-31T05:00:51.675061+0000","last_clean":"2024-01-31T05:00:51.675061+0000","last_became_active":"2024-01-31T05:00:45.577538+0000","last_became_peered":"2024-01-31T05:00:45.577538+0000","last_unstale":"2024-01-31T05:00:51.675061+0000","last_undegraded":"2024-01-31T05:00:51.675061+0000","last_fullsized":"2024-01-31T05:00:51.675061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:15.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912746+0000","last_change":"2024-01-31T05:00:50.630730+0000","last_active":"2024-01-31T05:00:51.912746+0000","last_peered":"2024-01-31T05:00:51.912746+0000","last_clean":"2024-01-31T05:00:51.912746+0000","last_became_active":"2024-01-31T05:00:50.629780+0000","last_became_peered":"2024-01-31T05:00:50.629780+0000","last_unstale":"2024-01-31T05:00:51.912746+0000","last_undegraded":"2024-01-31T05:00:51.912746+0000","last_fullsized":"2024-01-31T05:00:51.912746+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:07:58.674398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679833+0000","last_change":"2024-01-31T05:00:46.583111+0000","last_active":"2024-01-31T05:00:51.679833+0000","last_peered":"2024-01-31T05:00:51.679833+0000","last_clean":"2024-01-31T05:00:51.679833+0000","last_became_active":"2024-01-31T05:00:46.582827+0000","last_became_peered":"2024-01-31T05:00:46.582827+0000","last_unstale":"2024-01-31T05:00:51.679833+0000","last_undegraded":"2024-01-31T05:00:51.679833+0000","last_fullsized":"2024-01-31T05:00:51.679833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:40:42.367021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679389+0000","last_change":"2024-01-31T05:00:45.580550+0000","last_active":"2024-01-31T05:00:51.679389+0000","last_peered":"2024-01-31T05:00:51.679389+0000","last_clean":"2024-01-31T05:00:51.679389+0000","last_became_active":"2024-01-31T05:00:45.579549+0000","last_became_peered":"2024-01-31T05:00:45.579549+0000","last_unstale":"2024-01-31T05:00:51.679389+0000","last_undegraded":"2024-01-31T05:00:51.679389+0000","last_fullsized":"2024-01-31T05:00:51.679389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:43.683478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674795+0000","last_change":"2024-01-31T05:00:50.649333+0000","last_active":"2024-01-31T05:00:51.674795+0000","last_peered":"2024-01-31T05:00:51.674795+0000","last_clean":"2024-01-31T05:00:51.674795+0000","last_became_active":"2024-01-31T05:00:50.648927+0000","last_became_peered":"2024-01-31T05:00:50.648927+0000","last_unstale":"2024-01-31T05:00:51.674795+0000","last_undegraded":"2024-01-31T05:00:51.674795+0000","last_fullsized":"2024-01-31T05:00:51.674795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918351+0000","last_change":"2024-01-31T05:00:46.581800+0000","last_active":"2024-01-31T05:00:51.918351+0000","last_peered":"2024-01-31T05:00:51.918351+0000","last_clean":"2024-01-31T05:00:51.918351+0000","last_became_active":"2024-01-31T05:00:46.581435+0000","last_became_peered":"2024-01-31T05:00:46.581435+0000","last_unstale":"2024-01-31T05:00:51.918351+0000","last_undegraded":"2024-01-31T05:00:51.918351+0000","last_fullsized":"2024-01-31T05:00:51.918351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:55:29.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674998+0000","last_change":"2024-01-31T05:00:45.570866+0000","last_active":"2024-01-31T05:00:51.674998+0000","last_peered":"2024-01-31T05:00:51.674998+0000","last_clean":"2024-01-31T05:00:51.674998+0000","last_became_active":"2024-01-31T05:00:45.570644+0000","last_became_peered":"2024-01-31T05:00:45.570644+0000","last_unstale":"2024-01-31T05:00:51.674998+0000","last_undegraded":"2024-01-31T05:00:51.674998+0000","last_fullsized":"2024-01-31T05:00:51.674998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:54.101266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917676+0000","last_change":"2024-01-31T05:00:50.641145+0000","last_active":"2024-01-31T05:00:51.917676+0000","last_peered":"2024-01-31T05:00:51.917676+0000","last_clean":"2024-01-31T05:00:51.917676+0000","last_became_active":"2024-01-31T05:00:50.638626+0000","last_became_peered":"2024-01-31T05:00:50.638626+0000","last_unstale":"2024-01-31T05:00:51.917676+0000","last_undegraded":"2024-01-31T05:00:51.917676+0000","last_fullsized":"2024-01-31T05:00:51.917676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:46:01.447285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914572+0000","last_change":"2024-01-31T05:00:46.589437+0000","last_active":"2024-01-31T05:00:51.914572+0000","last_peered":"2024-01-31T05:00:51.914572+0000","last_clean":"2024-01-31T05:00:51.914572+0000","last_became_active":"2024-01-31T05:00:46.589134+0000","last_became_peered":"2024-01-31T05:00:46.589134+0000","last_unstale":"2024-01-31T05:00:51.914572+0000","last_undegraded":"2024-01-31T05:00:51.914572+0000","last_fullsized":"2024-01-31T05:00:51.914572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680322+0000","last_change":"2024-01-31T05:00:45.579033+0000","last_active":"2024-01-31T05:00:51.680322+0000","last_peered":"2024-01-31T05:00:51.680322+0000","last_clean":"2024-01-31T05:00:51.680322+0000","last_became_active":"2024-01-31T05:00:45.578730+0000","last_became_peered":"2024-01-31T05:00:45.578730+0000","last_unstale":"2024-01-31T05:00:51.680322+0000","last_undegraded":"2024-01-31T05:00:51.680322+0000","last_fullsized":"2024-01-31T05:00:51.680322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916714+0000","last_change":"2024-01-31T05:00:50.639581+0000","last_active":"2024-01-31T05:00:51.916714+0000","last_peered":"2024-01-31T05:00:51.916714+0000","last_clean":"2024-01-31T05:00:51.916714+0000","last_became_active":"2024-01-31T05:00:50.639192+0000","last_became_peered":"2024-01-31T05:00:50.639192+0000","last_unstale":"2024-01-31T05:00:51.916714+0000","last_undegraded":"2024-01-31T05:00:51.916714+0000","last_fullsized":"2024-01-31T05:00:51.916714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:28.385448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915081+0000","last_change":"2024-01-31T05:00:46.590496+0000","last_active":"2024-01-31T05:00:51.915081+0000","last_peered":"2024-01-31T05:00:51.915081+0000","last_clean":"2024-01-31T05:00:51.915081+0000","last_became_active":"2024-01-31T05:00:46.587386+0000","last_became_peered":"2024-01-31T05:00:46.587386+0000","last_unstale":"2024-01-31T05:00:51.915081+0000","last_undegraded":"2024-01-31T05:00:51.915081+0000","last_fullsized":"2024-01-31T05:00:51.915081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674054+0000","last_change":"2024-01-31T05:00:45.580484+0000","last_active":"2024-01-31T05:00:51.674054+0000","last_peered":"2024-01-31T05:00:51.674054+0000","last_clean":"2024-01-31T05:00:51.674054+0000","last_became_active":"2024-01-31T05:00:45.580179+0000","last_became_peered":"2024-01-31T05:00:45.580179+0000","last_unstale":"2024-01-31T05:00:51.674054+0000","last_undegraded":"2024-01-31T05:00:51.674054+0000","last_fullsized":"2024-01-31T05:00:51.674054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:54.780283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918220+0000","last_change":"2024-01-31T05:00:50.633651+0000","last_active":"2024-01-31T05:00:51.918220+0000","last_peered":"2024-01-31T05:00:51.918220+0000","last_clean":"2024-01-31T05:00:51.918220+0000","last_became_active":"2024-01-31T05:00:50.633176+0000","last_became_peered":"2024-01-31T05:00:50.633176+0000","last_unstale":"2024-01-31T05:00:51.918220+0000","last_undegraded":"2024-01-31T05:00:51.918220+0000","last_fullsized":"2024-01-31T05:00:51.918220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:41.984643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675265+0000","last_change":"2024-01-31T05:00:46.578636+0000","last_active":"2024-01-31T05:00:51.675265+0000","last_peered":"2024-01-31T05:00:51.675265+0000","last_clean":"2024-01-31T05:00:51.675265+0000","last_became_active":"2024-01-31T05:00:46.577232+0000","last_became_peered":"2024-01-31T05:00:46.577232+0000","last_unstale":"2024-01-31T05:00:51.675265+0000","last_undegraded":"2024-01-31T05:00:51.675265+0000","last_fullsized":"2024-01-31T05:00:51.675265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:37:22.658839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915084+0000","last_change":"2024-01-31T05:00:45.574367+0000","last_active":"2024-01-31T05:00:51.915084+0000","last_peered":"2024-01-31T05:00:51.915084+0000","last_clean":"2024-01-31T05:00:51.915084+0000","last_became_active":"2024-01-31T05:00:45.574010+0000","last_became_peered":"2024-01-31T05:00:45.574010+0000","last_unstale":"2024-01-31T05:00:51.915084+0000","last_undegraded":"2024-01-31T05:00:51.915084+0000","last_fullsized":"2024-01-31T05:00:51.915084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:12:03.585718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918910+0000","last_change":"2024-01-31T05:00:50.642311+0000","last_active":"2024-01-31T05:00:51.918910+0000","last_peered":"2024-01-31T05:00:51.918910+0000","last_clean":"2024-01-31T05:00:51.918910+0000","last_became_active":"2024-01-31T05:00:50.641937+0000","last_became_peered":"2024-01-31T05:00:50.641937+0000","last_unstale":"2024-01-31T05:00:51.918910+0000","last_undegraded":"2024-01-31T05:00:51.918910+0000","last_fullsized":"2024-01-31T05:00:51.918910+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:57.841272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916940+0000","last_change":"2024-01-31T05:00:46.589298+0000","last_active":"2024-01-31T05:00:51.916940+0000","last_peered":"2024-01-31T05:00:51.916940+0000","last_clean":"2024-01-31T05:00:51.916940+0000","last_became_active":"2024-01-31T05:00:46.589007+0000","last_became_peered":"2024-01-31T05:00:46.589007+0000","last_unstale":"2024-01-31T05:00:51.916940+0000","last_undegraded":"2024-01-31T05:00:51.916940+0000","last_fullsized":"2024-01-31T05:00:51.916940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:53:18.216173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673208+0000","last_change":"2024-01-31T05:00:45.573415+0000","last_active":"2024-01-31T05:00:51.673208+0000","last_peered":"2024-01-31T05:00:51.673208+0000","last_clean":"2024-01-31T05:00:51.673208+0000","last_became_active":"2024-01-31T05:00:45.573080+0000","last_became_peered":"2024-01-31T05:00:45.573080+0000","last_unstale":"2024-01-31T05:00:51.673208+0000","last_undegraded":"2024-01-31T05:00:51.673208+0000","last_fullsized":"2024-01-31T05:00:51.673208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:22.222326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680907+0000","last_change":"2024-01-31T05:00:50.642960+0000","last_active":"2024-01-31T05:00:51.680907+0000","last_peered":"2024-01-31T05:00:51.680907+0000","last_clean":"2024-01-31T05:00:51.680907+0000","last_became_active":"2024-01-31T05:00:50.640025+0000","last_became_peered":"2024-01-31T05:00:50.640025+0000","last_unstale":"2024-01-31T05:00:51.680907+0000","last_undegraded":"2024-01-31T05:00:51.680907+0000","last_fullsized":"2024-01-31T05:00:51.680907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:28:26.315820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677292+0000","last_change":"2024-01-31T05:00:46.592032+0000","last_active":"2024-01-31T05:00:51.677292+0000","last_peered":"2024-01-31T05:00:51.677292+0000","last_clean":"2024-01-31T05:00:51.677292+0000","last_became_active":"2024-01-31T05:00:46.588029+0000","last_became_peered":"2024-01-31T05:00:46.588029+0000","last_unstale":"2024-01-31T05:00:51.677292+0000","last_undegraded":"2024-01-31T05:00:51.677292+0000","last_fullsized":"2024-01-31T05:00:51.677292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:42.890080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673909+0000","last_change":"2024-01-31T05:00:45.562238+0000","last_active":"2024-01-31T05:00:51.673909+0000","last_peered":"2024-01-31T05:00:51.673909+0000","last_clean":"2024-01-31T05:00:51.673909+0000","last_became_active":"2024-01-31T05:00:45.561726+0000","last_became_peered":"2024-01-31T05:00:45.561726+0000","last_unstale":"2024-01-31T05:00:51.673909+0000","last_undegraded":"2024-01-31T05:00:51.673909+0000","last_fullsized":"2024-01-31T05:00:51.673909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:13.468671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912656+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:00:51.912656+0000","last_peered":"2024-01-31T05:00:51.912656+0000","last_clean":"2024-01-31T05:00:51.912656+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:00:51.912656+0000","last_undegraded":"2024-01-31T05:00:51.912656+0000","last_fullsized":"2024-01-31T05:00:51.912656+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674754+0000","last_change":"2024-01-31T05:00:50.629234+0000","last_active":"2024-01-31T05:00:51.674754+0000","last_peered":"2024-01-31T05:00:51.674754+0000","last_clean":"2024-01-31T05:00:51.674754+0000","last_became_active":"2024-01-31T05:00:50.628308+0000","last_became_peered":"2024-01-31T05:00:50.628308+0000","last_unstale":"2024-01-31T05:00:51.674754+0000","last_undegraded":"2024-01-31T05:00:51.674754+0000","last_fullsized":"2024-01-31T05:00:51.674754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:21:35.300095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674590+0000","last_change":"2024-01-31T05:00:46.583227+0000","last_active":"2024-01-31T05:00:51.674590+0000","last_peered":"2024-01-31T05:00:51.674590+0000","last_clean":"2024-01-31T05:00:51.674590+0000","last_became_active":"2024-01-31T05:00:46.582532+0000","last_became_peered":"2024-01-31T05:00:46.582532+0000","last_unstale":"2024-01-31T05:00:51.674590+0000","last_undegraded":"2024-01-31T05:00:51.674590+0000","last_fullsized":"2024-01-31T05:00:51.674590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:53:51.592799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918513+0000","last_change":"2024-01-31T05:00:45.583848+0000","last_active":"2024-01-31T05:00:51.918513+0000","last_peered":"2024-01-31T05:00:51.918513+0000","last_clean":"2024-01-31T05:00:51.918513+0000","last_became_active":"2024-01-31T05:00:45.583492+0000","last_became_peered":"2024-01-31T05:00:45.583492+0000","last_unstale":"2024-01-31T05:00:51.918513+0000","last_undegraded":"2024-01-31T05:00:51.918513+0000","last_fullsized":"2024-01-31T05:00:51.918513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912922+0000","last_change":"2024-01-31T05:00:50.642193+0000","last_active":"2024-01-31T05:00:51.912922+0000","last_peered":"2024-01-31T05:00:51.912922+0000","last_clean":"2024-01-31T05:00:51.912922+0000","last_became_active":"2024-01-31T05:00:50.639911+0000","last_became_peered":"2024-01-31T05:00:50.639911+0000","last_unstale":"2024-01-31T05:00:51.912922+0000","last_undegraded":"2024-01-31T05:00:51.912922+0000","last_fullsized":"2024-01-31T05:00:51.912922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:50:44.744734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915799+0000","last_change":"2024-01-31T05:00:46.580861+0000","last_active":"2024-01-31T05:00:51.915799+0000","last_peered":"2024-01-31T05:00:51.915799+0000","last_clean":"2024-01-31T05:00:51.915799+0000","last_became_active":"2024-01-31T05:00:46.580588+0000","last_became_peered":"2024-01-31T05:00:46.580588+0000","last_unstale":"2024-01-31T05:00:51.915799+0000","last_undegraded":"2024-01-31T05:00:51.915799+0000","last_fullsized":"2024-01-31T05:00:51.915799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681122+0000","last_change":"2024-01-31T05:00:45.567443+0000","last_active":"2024-01-31T05:00:51.681122+0000","last_peered":"2024-01-31T05:00:51.681122+0000","last_clean":"2024-01-31T05:00:51.681122+0000","last_became_active":"2024-01-31T05:00:45.567160+0000","last_became_peered":"2024-01-31T05:00:45.567160+0000","last_unstale":"2024-01-31T05:00:51.681122+0000","last_undegraded":"2024-01-31T05:00:51.681122+0000","last_fullsized":"2024-01-31T05:00:51.681122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679437+0000","last_change":"2024-01-31T05:00:50.647210+0000","last_active":"2024-01-31T05:00:51.679437+0000","last_peered":"2024-01-31T05:00:51.679437+0000","last_clean":"2024-01-31T05:00:51.679437+0000","last_became_active":"2024-01-31T05:00:50.646792+0000","last_became_peered":"2024-01-31T05:00:50.646792+0000","last_unstale":"2024-01-31T05:00:51.679437+0000","last_undegraded":"2024-01-31T05:00:51.679437+0000","last_fullsized":"2024-01-31T05:00:51.679437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915671+0000","last_change":"2024-01-31T05:00:46.593737+0000","last_active":"2024-01-31T05:00:51.915671+0000","last_peered":"2024-01-31T05:00:51.915671+0000","last_clean":"2024-01-31T05:00:51.915671+0000","last_became_active":"2024-01-31T05:00:46.593484+0000","last_became_peered":"2024-01-31T05:00:46.593484+0000","last_unstale":"2024-01-31T05:00:51.915671+0000","last_undegraded":"2024-01-31T05:00:51.915671+0000","last_fullsized":"2024-01-31T05:00:51.915671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:38:14.536828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679315+0000","last_change":"2024-01-31T05:00:45.582469+0000","last_active":"2024-01-31T05:00:51.679315+0000","last_peered":"2024-01-31T05:00:51.679315+0000","last_clean":"2024-01-31T05:00:51.679315+0000","last_became_active":"2024-01-31T05:00:45.582151+0000","last_became_peered":"2024-01-31T05:00:45.582151+0000","last_unstale":"2024-01-31T05:00:51.679315+0000","last_undegraded":"2024-01-31T05:00:51.679315+0000","last_fullsized":"2024-01-31T05:00:51.679315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:36:17.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672556+0000","last_change":"2024-01-31T05:00:50.642993+0000","last_active":"2024-01-31T05:00:51.672556+0000","last_peered":"2024-01-31T05:00:51.672556+0000","last_clean":"2024-01-31T05:00:51.672556+0000","last_became_active":"2024-01-31T05:00:50.642558+0000","last_became_peered":"2024-01-31T05:00:50.642558+0000","last_unstale":"2024-01-31T05:00:51.672556+0000","last_undegraded":"2024-01-31T05:00:51.672556+0000","last_fullsized":"2024-01-31T05:00:51.672556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:07:54.060532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914870+0000","last_change":"2024-01-31T05:00:46.597235+0000","last_active":"2024-01-31T05:00:51.914870+0000","last_peered":"2024-01-31T05:00:51.914870+0000","last_clean":"2024-01-31T05:00:51.914870+0000","last_became_active":"2024-01-31T05:00:46.592597+0000","last_became_peered":"2024-01-31T05:00:46.592597+0000","last_unstale":"2024-01-31T05:00:51.914870+0000","last_undegraded":"2024-01-31T05:00:51.914870+0000","last_fullsized":"2024-01-31T05:00:51.914870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:55.281235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915788+0000","last_change":"2024-01-31T05:00:45.583433+0000","last_active":"2024-01-31T05:00:51.915788+0000","last_peered":"2024-01-31T05:00:51.915788+0000","last_clean":"2024-01-31T05:00:51.915788+0000","last_became_active":"2024-01-31T05:00:45.582873+0000","last_became_peered":"2024-01-31T05:00:45.582873+0000","last_unstale":"2024-01-31T05:00:51.915788+0000","last_undegraded":"2024-01-31T05:00:51.915788+0000","last_fullsized":"2024-01-31T05:00:51.915788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:44:30.759721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913367+0000","last_change":"2024-01-31T05:00:50.651478+0000","last_active":"2024-01-31T05:00:51.913367+0000","last_peered":"2024-01-31T05:00:51.913367+0000","last_clean":"2024-01-31T05:00:51.913367+0000","last_became_active":"2024-01-31T05:00:50.650775+0000","last_became_peered":"2024-01-31T05:00:50.650775+0000","last_unstale":"2024-01-31T05:00:51.913367+0000","last_undegraded":"2024-01-31T05:00:51.913367+0000","last_fullsized":"2024-01-31T05:00:51.913367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672742+0000","last_change":"2024-01-31T05:00:46.588386+0000","last_active":"2024-01-31T05:00:51.672742+0000","last_peered":"2024-01-31T05:00:51.672742+0000","last_clean":"2024-01-31T05:00:51.672742+0000","last_became_active":"2024-01-31T05:00:46.588150+0000","last_became_peered":"2024-01-31T05:00:46.588150+0000","last_unstale":"2024-01-31T05:00:51.672742+0000","last_undegraded":"2024-01-31T05:00:51.672742+0000","last_fullsized":"2024-01-31T05:00:51.672742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:38.932253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674904+0000","last_change":"2024-01-31T05:00:45.572910+0000","last_active":"2024-01-31T05:00:51.674904+0000","last_peered":"2024-01-31T05:00:51.674904+0000","last_clean":"2024-01-31T05:00:51.674904+0000","last_became_active":"2024-01-31T05:00:45.570281+0000","last_became_peered":"2024-01-31T05:00:45.570281+0000","last_unstale":"2024-01-31T05:00:51.674904+0000","last_undegraded":"2024-01-31T05:00:51.674904+0000","last_fullsized":"2024-01-31T05:00:51.674904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681309+0000","last_change":"2024-01-31T05:00:50.646215+0000","last_active":"2024-01-31T05:00:51.681309+0000","last_peered":"2024-01-31T05:00:51.681309+0000","last_clean":"2024-01-31T05:00:51.681309+0000","last_became_active":"2024-01-31T05:00:50.640453+0000","last_became_peered":"2024-01-31T05:00:50.640453+0000","last_unstale":"2024-01-31T05:00:51.681309+0000","last_undegraded":"2024-01-31T05:00:51.681309+0000","last_fullsized":"2024-01-31T05:00:51.681309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:20.595164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912715+0000","last_change":"2024-01-31T05:00:46.582233+0000","last_active":"2024-01-31T05:00:51.912715+0000","last_peered":"2024-01-31T05:00:51.912715+0000","last_clean":"2024-01-31T05:00:51.912715+0000","last_became_active":"2024-01-31T05:00:46.581941+0000","last_became_peered":"2024-01-31T05:00:46.581941+0000","last_unstale":"2024-01-31T05:00:51.912715+0000","last_undegraded":"2024-01-31T05:00:51.912715+0000","last_fullsized":"2024-01-31T05:00:51.912715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914727+0000","last_change":"2024-01-31T05:00:45.586099+0000","last_active":"2024-01-31T05:00:51.914727+0000","last_peered":"2024-01-31T05:00:51.914727+0000","last_clean":"2024-01-31T05:00:51.914727+0000","last_became_active":"2024-01-31T05:00:45.583149+0000","last_became_peered":"2024-01-31T05:00:45.583149+0000","last_unstale":"2024-01-31T05:00:51.914727+0000","last_undegraded":"2024-01-31T05:00:51.914727+0000","last_fullsized":"2024-01-31T05:00:51.914727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681581+0000","last_change":"2024-01-31T05:00:50.644931+0000","last_active":"2024-01-31T05:00:51.681581+0000","last_peered":"2024-01-31T05:00:51.681581+0000","last_clean":"2024-01-31T05:00:51.681581+0000","last_became_active":"2024-01-31T05:00:50.644554+0000","last_became_peered":"2024-01-31T05:00:50.644554+0000","last_unstale":"2024-01-31T05:00:51.681581+0000","last_undegraded":"2024-01-31T05:00:51.681581+0000","last_fullsized":"2024-01-31T05:00:51.681581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:23:08.930803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673424+0000","last_change":"2024-01-31T05:00:46.582885+0000","last_active":"2024-01-31T05:00:51.673424+0000","last_peered":"2024-01-31T05:00:51.673424+0000","last_clean":"2024-01-31T05:00:51.673424+0000","last_became_active":"2024-01-31T05:00:46.582585+0000","last_became_peered":"2024-01-31T05:00:46.582585+0000","last_unstale":"2024-01-31T05:00:51.673424+0000","last_undegraded":"2024-01-31T05:00:51.673424+0000","last_fullsized":"2024-01-31T05:00:51.673424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:15:39.727609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673905+0000","last_change":"2024-01-31T05:00:45.570263+0000","last_active":"2024-01-31T05:00:51.673905+0000","last_peered":"2024-01-31T05:00:51.673905+0000","last_clean":"2024-01-31T05:00:51.673905+0000","last_became_active":"2024-01-31T05:00:45.568414+0000","last_became_peered":"2024-01-31T05:00:45.568414+0000","last_unstale":"2024-01-31T05:00:51.673905+0000","last_undegraded":"2024-01-31T05:00:51.673905+0000","last_fullsized":"2024-01-31T05:00:51.673905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:02.658111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918156+0000","last_change":"2024-01-31T05:00:50.640753+0000","last_active":"2024-01-31T05:00:51.918156+0000","last_peered":"2024-01-31T05:00:51.918156+0000","last_clean":"2024-01-31T05:00:51.918156+0000","last_became_active":"2024-01-31T05:00:50.640179+0000","last_became_peered":"2024-01-31T05:00:50.640179+0000","last_unstale":"2024-01-31T05:00:51.918156+0000","last_undegraded":"2024-01-31T05:00:51.918156+0000","last_fullsized":"2024-01-31T05:00:51.918156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:39.555756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916848+0000","last_change":"2024-01-31T05:00:46.596210+0000","last_active":"2024-01-31T05:00:51.916848+0000","last_peered":"2024-01-31T05:00:51.916848+0000","last_clean":"2024-01-31T05:00:51.916848+0000","last_became_active":"2024-01-31T05:00:46.595941+0000","last_became_peered":"2024-01-31T05:00:46.595941+0000","last_unstale":"2024-01-31T05:00:51.916848+0000","last_undegraded":"2024-01-31T05:00:51.916848+0000","last_fullsized":"2024-01-31T05:00:51.916848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:09.854108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916254+0000","last_change":"2024-01-31T05:00:45.577952+0000","last_active":"2024-01-31T05:00:51.916254+0000","last_peered":"2024-01-31T05:00:51.916254+0000","last_clean":"2024-01-31T05:00:51.916254+0000","last_became_active":"2024-01-31T05:00:45.577632+0000","last_became_peered":"2024-01-31T05:00:45.577632+0000","last_unstale":"2024-01-31T05:00:51.916254+0000","last_undegraded":"2024-01-31T05:00:51.916254+0000","last_fullsized":"2024-01-31T05:00:51.916254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673947+0000","last_change":"2024-01-31T05:00:50.639630+0000","last_active":"2024-01-31T05:00:51.673947+0000","last_peered":"2024-01-31T05:00:51.673947+0000","last_clean":"2024-01-31T05:00:51.673947+0000","last_became_active":"2024-01-31T05:00:50.639187+0000","last_became_peered":"2024-01-31T05:00:50.639187+0000","last_unstale":"2024-01-31T05:00:51.673947+0000","last_undegraded":"2024-01-31T05:00:51.673947+0000","last_fullsized":"2024-01-31T05:00:51.673947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:41:38.465126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915506+0000","last_change":"2024-01-31T05:00:46.595225+0000","last_active":"2024-01-31T05:00:51.915506+0000","last_peered":"2024-01-31T05:00:51.915506+0000","last_clean":"2024-01-31T05:00:51.915506+0000","last_became_active":"2024-01-31T05:00:46.594892+0000","last_became_peered":"2024-01-31T05:00:46.594892+0000","last_unstale":"2024-01-31T05:00:51.915506+0000","last_undegraded":"2024-01-31T05:00:51.915506+0000","last_fullsized":"2024-01-31T05:00:51.915506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:57.797805+0000","last_change":"2024-01-31T05:00:45.575073+0000","last_active":"2024-01-31T05:00:57.797805+0000","last_peered":"2024-01-31T05:00:57.797805+0000","last_clean":"2024-01-31T05:00:57.797805+0000","last_became_active":"2024-01-31T05:00:45.574725+0000","last_became_peered":"2024-01-31T05:00:45.574725+0000","last_unstale":"2024-01-31T05:00:57.797805+0000","last_undegraded":"2024-01-31T05:00:57.797805+0000","last_fullsized":"2024-01-31T05:00:57.797805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674916+0000","last_change":"2024-01-31T05:00:50.640744+0000","last_active":"2024-01-31T05:00:51.674916+0000","last_peered":"2024-01-31T05:00:51.674916+0000","last_clean":"2024-01-31T05:00:51.674916+0000","last_became_active":"2024-01-31T05:00:50.640181+0000","last_became_peered":"2024-01-31T05:00:50.640181+0000","last_unstale":"2024-01-31T05:00:51.674916+0000","last_undegraded":"2024-01-31T05:00:51.674916+0000","last_fullsized":"2024-01-31T05:00:51.674916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673600+0000","last_change":"2024-01-31T05:00:46.587774+0000","last_active":"2024-01-31T05:00:51.673600+0000","last_peered":"2024-01-31T05:00:51.673600+0000","last_clean":"2024-01-31T05:00:51.673600+0000","last_became_active":"2024-01-31T05:00:46.587495+0000","last_became_peered":"2024-01-31T05:00:46.587495+0000","last_unstale":"2024-01-31T05:00:51.673600+0000","last_undegraded":"2024-01-31T05:00:51.673600+0000","last_fullsized":"2024-01-31T05:00:51.673600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:19:31.923714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917086+0000","last_change":"2024-01-31T05:00:45.577387+0000","last_active":"2024-01-31T05:00:51.917086+0000","last_peered":"2024-01-31T05:00:51.917086+0000","last_clean":"2024-01-31T05:00:51.917086+0000","last_became_active":"2024-01-31T05:00:45.577035+0000","last_became_peered":"2024-01-31T05:00:45.577035+0000","last_unstale":"2024-01-31T05:00:51.917086+0000","last_undegraded":"2024-01-31T05:00:51.917086+0000","last_fullsized":"2024-01-31T05:00:51.917086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914130+0000","last_change":"2024-01-31T05:00:50.641531+0000","last_active":"2024-01-31T05:00:51.914130+0000","last_peered":"2024-01-31T05:00:51.914130+0000","last_clean":"2024-01-31T05:00:51.914130+0000","last_became_active":"2024-01-31T05:00:50.641164+0000","last_became_peered":"2024-01-31T05:00:50.641164+0000","last_unstale":"2024-01-31T05:00:51.914130+0000","last_undegraded":"2024-01-31T05:00:51.914130+0000","last_fullsized":"2024-01-31T05:00:51.914130+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681764+0000","last_change":"2024-01-31T05:00:46.584219+0000","last_active":"2024-01-31T05:00:51.681764+0000","last_peered":"2024-01-31T05:00:51.681764+0000","last_clean":"2024-01-31T05:00:51.681764+0000","last_became_active":"2024-01-31T05:00:46.583942+0000","last_became_peered":"2024-01-31T05:00:46.583942+0000","last_unstale":"2024-01-31T05:00:51.681764+0000","last_undegraded":"2024-01-31T05:00:51.681764+0000","last_fullsized":"2024-01-31T05:00:51.681764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915433+0000","last_change":"2024-01-31T05:00:45.576533+0000","last_active":"2024-01-31T05:00:51.915433+0000","last_peered":"2024-01-31T05:00:51.915433+0000","last_clean":"2024-01-31T05:00:51.915433+0000","last_became_active":"2024-01-31T05:00:45.576327+0000","last_became_peered":"2024-01-31T05:00:45.576327+0000","last_unstale":"2024-01-31T05:00:51.915433+0000","last_undegraded":"2024-01-31T05:00:51.915433+0000","last_fullsized":"2024-01-31T05:00:51.915433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:09:58.811691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914596+0000","last_change":"2024-01-31T05:00:50.654987+0000","last_active":"2024-01-31T05:00:51.914596+0000","last_peered":"2024-01-31T05:00:51.914596+0000","last_clean":"2024-01-31T05:00:51.914596+0000","last_became_active":"2024-01-31T05:00:50.654461+0000","last_became_peered":"2024-01-31T05:00:50.654461+0000","last_unstale":"2024-01-31T05:00:51.914596+0000","last_undegraded":"2024-01-31T05:00:51.914596+0000","last_fullsized":"2024-01-31T05:00:51.914596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914829+0000","last_change":"2024-01-31T05:00:46.597875+0000","last_active":"2024-01-31T05:00:51.914829+0000","last_peered":"2024-01-31T05:00:51.914829+0000","last_clean":"2024-01-31T05:00:51.914829+0000","last_became_active":"2024-01-31T05:00:46.597494+0000","last_became_peered":"2024-01-31T05:00:46.597494+0000","last_unstale":"2024-01-31T05:00:51.914829+0000","last_undegraded":"2024-01-31T05:00:51.914829+0000","last_fullsized":"2024-01-31T05:00:51.914829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:14.648640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.574121+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.573715+0000","last_became_peered":"2024-01-31T05:00:45.573715+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676164+0000","last_change":"2024-01-31T05:00:50.642176+0000","last_active":"2024-01-31T05:00:51.676164+0000","last_peered":"2024-01-31T05:00:51.676164+0000","last_clean":"2024-01-31T05:00:51.676164+0000","last_became_active":"2024-01-31T05:00:50.639134+0000","last_became_peered":"2024-01-31T05:00:50.639134+0000","last_unstale":"2024-01-31T05:00:51.676164+0000","last_undegraded":"2024-01-31T05:00:51.676164+0000","last_fullsized":"2024-01-31T05:00:51.676164+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:11.485002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676564+0000","last_change":"2024-01-31T05:00:50.649633+0000","last_active":"2024-01-31T05:00:51.676564+0000","last_peered":"2024-01-31T05:00:51.676564+0000","last_clean":"2024-01-31T05:00:51.676564+0000","last_became_active":"2024-01-31T05:00:50.647001+0000","last_became_peered":"2024-01-31T05:00:50.647001+0000","last_unstale":"2024-01-31T05:00:51.676564+0000","last_undegraded":"2024-01-31T05:00:51.676564+0000","last_fullsized":"2024-01-31T05:00:51.676564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:42.992455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676393+0000","last_change":"2024-01-31T05:00:46.567858+0000","last_active":"2024-01-31T05:00:51.676393+0000","last_peered":"2024-01-31T05:00:51.676393+0000","last_clean":"2024-01-31T05:00:51.676393+0000","last_became_active":"2024-01-31T05:00:46.567351+0000","last_became_peered":"2024-01-31T05:00:46.567351+0000","last_unstale":"2024-01-31T05:00:51.676393+0000","last_undegraded":"2024-01-31T05:00:51.676393+0000","last_fullsized":"2024-01-31T05:00:51.676393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:07.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673099+0000","last_change":"2024-01-31T05:00:45.562694+0000","last_active":"2024-01-31T05:00:51.673099+0000","last_peered":"2024-01-31T05:00:51.673099+0000","last_clean":"2024-01-31T05:00:51.673099+0000","last_became_active":"2024-01-31T05:00:45.560991+0000","last_became_peered":"2024-01-31T05:00:45.560991+0000","last_unstale":"2024-01-31T05:00:51.673099+0000","last_undegraded":"2024-01-31T05:00:51.673099+0000","last_fullsized":"2024-01-31T05:00:51.673099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674525+0000","last_change":"2024-01-31T05:00:50.638454+0000","last_active":"2024-01-31T05:00:51.674525+0000","last_peered":"2024-01-31T05:00:51.674525+0000","last_clean":"2024-01-31T05:00:51.674525+0000","last_became_active":"2024-01-31T05:00:50.637600+0000","last_became_peered":"2024-01-31T05:00:50.637600+0000","last_unstale":"2024-01-31T05:00:51.674525+0000","last_undegraded":"2024-01-31T05:00:51.674525+0000","last_fullsized":"2024-01-31T05:00:51.674525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:16.246206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676944+0000","last_change":"2024-01-31T05:00:46.574103+0000","last_active":"2024-01-31T05:00:51.676944+0000","last_peered":"2024-01-31T05:00:51.676944+0000","last_clean":"2024-01-31T05:00:51.676944+0000","last_became_active":"2024-01-31T05:00:46.572921+0000","last_became_peered":"2024-01-31T05:00:46.572921+0000","last_unstale":"2024-01-31T05:00:51.676944+0000","last_undegraded":"2024-01-31T05:00:51.676944+0000","last_fullsized":"2024-01-31T05:00:51.676944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:36.342238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916504+0000","last_change":"2024-01-31T05:00:45.587813+0000","last_active":"2024-01-31T05:00:51.916504+0000","last_peered":"2024-01-31T05:00:51.916504+0000","last_clean":"2024-01-31T05:00:51.916504+0000","last_became_active":"2024-01-31T05:00:45.587473+0000","last_became_peered":"2024-01-31T05:00:45.587473+0000","last_unstale":"2024-01-31T05:00:51.916504+0000","last_undegraded":"2024-01-31T05:00:51.916504+0000","last_fullsized":"2024-01-31T05:00:51.916504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:44:11.745313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673739+0000","last_change":"2024-01-31T05:00:45.574390+0000","last_active":"2024-01-31T05:00:51.673739+0000","last_peered":"2024-01-31T05:00:51.673739+0000","last_clean":"2024-01-31T05:00:51.673739+0000","last_became_active":"2024-01-31T05:00:45.574121+0000","last_became_peered":"2024-01-31T05:00:45.574121+0000","last_unstale":"2024-01-31T05:00:51.673739+0000","last_undegraded":"2024-01-31T05:00:51.673739+0000","last_fullsized":"2024-01-31T05:00:51.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:01.042942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677784+0000","last_change":"2024-01-31T05:00:46.582109+0000","last_active":"2024-01-31T05:00:51.677784+0000","last_peered":"2024-01-31T05:00:51.677784+0000","last_clean":"2024-01-31T05:00:51.677784+0000","last_became_active":"2024-01-31T05:00:46.581814+0000","last_became_peered":"2024-01-31T05:00:46.581814+0000","last_unstale":"2024-01-31T05:00:51.677784+0000","last_undegraded":"2024-01-31T05:00:51.677784+0000","last_fullsized":"2024-01-31T05:00:51.677784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:40.921562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.678543+0000","last_change":"2024-01-31T05:00:50.652235+0000","last_active":"2024-01-31T05:00:51.678543+0000","last_peered":"2024-01-31T05:00:51.678543+0000","last_clean":"2024-01-31T05:00:51.678543+0000","last_became_active":"2024-01-31T05:00:50.650199+0000","last_became_peered":"2024-01-31T05:00:50.650199+0000","last_unstale":"2024-01-31T05:00:51.678543+0000","last_undegraded":"2024-01-31T05:00:51.678543+0000","last_fullsized":"2024-01-31T05:00:51.678543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:15.584111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679821+0000","last_change":"2024-01-31T05:00:45.580589+0000","last_active":"2024-01-31T05:00:51.679821+0000","last_peered":"2024-01-31T05:00:51.679821+0000","last_clean":"2024-01-31T05:00:51.679821+0000","last_became_active":"2024-01-31T05:00:45.579288+0000","last_became_peered":"2024-01-31T05:00:45.579288+0000","last_unstale":"2024-01-31T05:00:51.679821+0000","last_undegraded":"2024-01-31T05:00:51.679821+0000","last_fullsized":"2024-01-31T05:00:51.679821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912730+0000","last_change":"2024-01-31T05:00:46.579010+0000","last_active":"2024-01-31T05:00:51.912730+0000","last_peered":"2024-01-31T05:00:51.912730+0000","last_clean":"2024-01-31T05:00:51.912730+0000","last_became_active":"2024-01-31T05:00:46.578792+0000","last_became_peered":"2024-01-31T05:00:46.578792+0000","last_unstale":"2024-01-31T05:00:51.912730+0000","last_undegraded":"2024-01-31T05:00:51.912730+0000","last_fullsized":"2024-01-31T05:00:51.912730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916113+0000","last_change":"2024-01-31T05:00:50.659322+0000","last_active":"2024-01-31T05:00:51.916113+0000","last_peered":"2024-01-31T05:00:51.916113+0000","last_clean":"2024-01-31T05:00:51.916113+0000","last_became_active":"2024-01-31T05:00:50.658743+0000","last_became_peered":"2024-01-31T05:00:50.658743+0000","last_unstale":"2024-01-31T05:00:51.916113+0000","last_undegraded":"2024-01-31T05:00:51.916113+0000","last_fullsized":"2024-01-31T05:00:51.916113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:20.923767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913064+0000","last_change":"2024-01-31T05:00:46.579418+0000","last_active":"2024-01-31T05:00:51.913064+0000","last_peered":"2024-01-31T05:00:51.913064+0000","last_clean":"2024-01-31T05:00:51.913064+0000","last_became_active":"2024-01-31T05:00:46.579076+0000","last_became_peered":"2024-01-31T05:00:46.579076+0000","last_unstale":"2024-01-31T05:00:51.913064+0000","last_undegraded":"2024-01-31T05:00:51.913064+0000","last_fullsized":"2024-01-31T05:00:51.913064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914823+0000","last_change":"2024-01-31T05:00:46.585461+0000","last_active":"2024-01-31T05:00:51.914823+0000","last_peered":"2024-01-31T05:00:51.914823+0000","last_clean":"2024-01-31T05:00:51.914823+0000","last_became_active":"2024-01-31T05:00:46.584818+0000","last_became_peered":"2024-01-31T05:00:46.584818+0000","last_unstale":"2024-01-31T05:00:51.914823+0000","last_undegraded":"2024-01-31T05:00:51.914823+0000","last_fullsized":"2024-01-31T05:00:51.914823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:52.011589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673116+0000","last_change":"2024-01-31T05:00:46.586264+0000","last_active":"2024-01-31T05:00:51.673116+0000","last_peered":"2024-01-31T05:00:51.673116+0000","last_clean":"2024-01-31T05:00:51.673116+0000","last_became_active":"2024-01-31T05:00:46.585995+0000","last_became_peered":"2024-01-31T05:00:46.585995+0000","last_unstale":"2024-01-31T05:00:51.673116+0000","last_undegraded":"2024-01-31T05:00:51.673116+0000","last_fullsized":"2024-01-31T05:00:51.673116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:11.523909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679572+0000","last_change":"2024-01-31T05:00:46.579199+0000","last_active":"2024-01-31T05:00:51.679572+0000","last_peered":"2024-01-31T05:00:51.679572+0000","last_clean":"2024-01-31T05:00:51.679572+0000","last_became_active":"2024-01-31T05:00:46.578257+0000","last_became_peered":"2024-01-31T05:00:46.578257+0000","last_unstale":"2024-01-31T05:00:51.679572+0000","last_undegraded":"2024-01-31T05:00:51.679572+0000","last_fullsized":"2024-01-31T05:00:51.679572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918235+0000","last_change":"2024-01-31T05:00:46.593320+0000","last_active":"2024-01-31T05:00:51.918235+0000","last_peered":"2024-01-31T05:00:51.918235+0000","last_clean":"2024-01-31T05:00:51.918235+0000","last_became_active":"2024-01-31T05:00:46.593053+0000","last_became_peered":"2024-01-31T05:00:46.593053+0000","last_unstale":"2024-01-31T05:00:51.918235+0000","last_undegraded":"2024-01-31T05:00:51.918235+0000","last_fullsized":"2024-01-31T05:00:51.918235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:21.326870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918207+0000","last_change":"2024-01-31T05:00:46.583943+0000","last_active":"2024-01-31T05:00:51.918207+0000","last_peered":"2024-01-31T05:00:51.918207+0000","last_clean":"2024-01-31T05:00:51.918207+0000","last_became_active":"2024-01-31T05:00:46.580249+0000","last_became_peered":"2024-01-31T05:00:46.580249+0000","last_unstale":"2024-01-31T05:00:51.918207+0000","last_undegraded":"2024-01-31T05:00:51.918207+0000","last_fullsized":"2024-01-31T05:00:51.918207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914742+0000","last_change":"2024-01-31T05:00:46.601110+0000","last_active":"2024-01-31T05:00:51.914742+0000","last_peered":"2024-01-31T05:00:51.914742+0000","last_clean":"2024-01-31T05:00:51.914742+0000","last_became_active":"2024-01-31T05:00:46.600844+0000","last_became_peered":"2024-01-31T05:00:46.600844+0000","last_unstale":"2024-01-31T05:00:51.914742+0000","last_undegraded":"2024-01-31T05:00:51.914742+0000","last_fullsized":"2024-01-31T05:00:51.914742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681192+0000","last_change":"2024-01-31T05:00:46.579792+0000","last_active":"2024-01-31T05:00:51.681192+0000","last_peered":"2024-01-31T05:00:51.681192+0000","last_clean":"2024-01-31T05:00:51.681192+0000","last_became_active":"2024-01-31T05:00:46.579357+0000","last_became_peered":"2024-01-31T05:00:46.579357+0000","last_unstale":"2024-01-31T05:00:51.681192+0000","last_undegraded":"2024-01-31T05:00:51.681192+0000","last_fullsized":"2024-01-31T05:00:51.681192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672904+0000","last_change":"2024-01-31T05:00:46.582507+0000","last_active":"2024-01-31T05:00:51.672904+0000","last_peered":"2024-01-31T05:00:51.672904+0000","last_clean":"2024-01-31T05:00:51.672904+0000","last_became_active":"2024-01-31T05:00:46.582243+0000","last_became_peered":"2024-01-31T05:00:46.582243+0000","last_unstale":"2024-01-31T05:00:51.672904+0000","last_undegraded":"2024-01-31T05:00:51.672904+0000","last_fullsized":"2024-01-31T05:00:51.672904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:07.689214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676563+0000","last_change":"2024-01-31T05:00:46.588900+0000","last_active":"2024-01-31T05:00:51.676563+0000","last_peered":"2024-01-31T05:00:51.676563+0000","last_clean":"2024-01-31T05:00:51.676563+0000","last_became_active":"2024-01-31T05:00:46.585994+0000","last_became_peered":"2024-01-31T05:00:46.585994+0000","last_unstale":"2024-01-31T05:00:51.676563+0000","last_undegraded":"2024-01-31T05:00:51.676563+0000","last_fullsized":"2024-01-31T05:00:51.676563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912637+0000","last_change":"2024-01-31T05:00:46.576223+0000","last_active":"2024-01-31T05:00:51.912637+0000","last_peered":"2024-01-31T05:00:51.912637+0000","last_clean":"2024-01-31T05:00:51.912637+0000","last_became_active":"2024-01-31T05:00:46.575962+0000","last_became_peered":"2024-01-31T05:00:46.575962+0000","last_unstale":"2024-01-31T05:00:51.912637+0000","last_undegraded":"2024-01-31T05:00:51.912637+0000","last_fullsized":"2024-01-31T05:00:51.912637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677638+0000","last_change":"2024-01-31T05:00:46.576001+0000","last_active":"2024-01-31T05:00:51.677638+0000","last_peered":"2024-01-31T05:00:51.677638+0000","last_clean":"2024-01-31T05:00:51.677638+0000","last_became_active":"2024-01-31T05:00:46.574721+0000","last_became_peered":"2024-01-31T05:00:46.574721+0000","last_unstale":"2024-01-31T05:00:51.677638+0000","last_undegraded":"2024-01-31T05:00:51.677638+0000","last_fullsized":"2024-01-31T05:00:51.677638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:31:53.337974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915731+0000","last_change":"2024-01-31T05:00:46.589503+0000","last_active":"2024-01-31T05:00:51.915731+0000","last_peered":"2024-01-31T05:00:51.915731+0000","last_clean":"2024-01-31T05:00:51.915731+0000","last_became_active":"2024-01-31T05:00:46.581750+0000","last_became_peered":"2024-01-31T05:00:46.581750+0000","last_unstale":"2024-01-31T05:00:51.915731+0000","last_undegraded":"2024-01-31T05:00:51.915731+0000","last_fullsized":"2024-01-31T05:00:51.915731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674812+0000","last_change":"2024-01-31T05:00:46.582051+0000","last_active":"2024-01-31T05:00:51.674812+0000","last_peered":"2024-01-31T05:00:51.674812+0000","last_clean":"2024-01-31T05:00:51.674812+0000","last_became_active":"2024-01-31T05:00:46.578134+0000","last_became_peered":"2024-01-31T05:00:46.578134+0000","last_unstale":"2024-01-31T05:00:51.674812+0000","last_undegraded":"2024-01-31T05:00:51.674812+0000","last_fullsized":"2024-01-31T05:00:51.674812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:31.255109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917009+0000","last_change":"2024-01-31T05:00:46.587070+0000","last_active":"2024-01-31T05:00:51.917009+0000","last_peered":"2024-01-31T05:00:51.917009+0000","last_clean":"2024-01-31T05:00:51.917009+0000","last_became_active":"2024-01-31T05:00:46.586707+0000","last_became_peered":"2024-01-31T05:00:46.586707+0000","last_unstale":"2024-01-31T05:00:51.917009+0000","last_undegraded":"2024-01-31T05:00:51.917009+0000","last_fullsized":"2024-01-31T05:00:51.917009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674901+0000","last_change":"2024-01-31T05:00:46.587042+0000","last_active":"2024-01-31T05:00:51.674901+0000","last_peered":"2024-01-31T05:00:51.674901+0000","last_clean":"2024-01-31T05:00:51.674901+0000","last_became_active":"2024-01-31T05:00:46.586518+0000","last_became_peered":"2024-01-31T05:00:46.586518+0000","last_unstale":"2024-01-31T05:00:51.674901+0000","last_undegraded":"2024-01-31T05:00:51.674901+0000","last_fullsized":"2024-01-31T05:00:51.674901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:54:17.597851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672833+0000","last_change":"2024-01-31T05:00:46.593111+0000","last_active":"2024-01-31T05:00:51.672833+0000","last_peered":"2024-01-31T05:00:51.672833+0000","last_clean":"2024-01-31T05:00:51.672833+0000","last_became_active":"2024-01-31T05:00:46.592828+0000","last_became_peered":"2024-01-31T05:00:46.592828+0000","last_unstale":"2024-01-31T05:00:51.672833+0000","last_undegraded":"2024-01-31T05:00:51.672833+0000","last_fullsized":"2024-01-31T05:00:51.672833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:59.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676004+0000","last_change":"2024-01-31T05:00:46.589807+0000","last_active":"2024-01-31T05:00:51.676004+0000","last_peered":"2024-01-31T05:00:51.676004+0000","last_clean":"2024-01-31T05:00:51.676004+0000","last_became_active":"2024-01-31T05:00:46.589518+0000","last_became_peered":"2024-01-31T05:00:46.589518+0000","last_unstale":"2024-01-31T05:00:51.676004+0000","last_undegraded":"2024-01-31T05:00:51.676004+0000","last_fullsized":"2024-01-31T05:00:51.676004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:06:03.428645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916672+0000","last_change":"2024-01-31T05:00:46.583014+0000","last_active":"2024-01-31T05:00:51.916672+0000","last_peered":"2024-01-31T05:00:51.916672+0000","last_clean":"2024-01-31T05:00:51.916672+0000","last_became_active":"2024-01-31T05:00:46.582607+0000","last_became_peered":"2024-01-31T05:00:46.582607+0000","last_unstale":"2024-01-31T05:00:51.916672+0000","last_undegraded":"2024-01-31T05:00:51.916672+0000","last_fullsized":"2024-01-31T05:00:51.916672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:21:09.066019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917869+0000","last_change":"2024-01-31T05:00:46.594160+0000","last_active":"2024-01-31T05:00:51.917869+0000","last_peered":"2024-01-31T05:00:51.917869+0000","last_clean":"2024-01-31T05:00:51.917869+0000","last_became_active":"2024-01-31T05:00:46.593838+0000","last_became_peered":"2024-01-31T05:00:46.593838+0000","last_unstale":"2024-01-31T05:00:51.917869+0000","last_undegraded":"2024-01-31T05:00:51.917869+0000","last_fullsized":"2024-01-31T05:00:51.917869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:58:27.273440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915637+0000","last_change":"2024-01-31T05:00:46.590079+0000","last_active":"2024-01-31T05:00:51.915637+0000","last_peered":"2024-01-31T05:00:51.915637+0000","last_clean":"2024-01-31T05:00:51.915637+0000","last_became_active":"2024-01-31T05:00:46.589828+0000","last_became_peered":"2024-01-31T05:00:46.589828+0000","last_unstale":"2024-01-31T05:00:51.915637+0000","last_undegraded":"2024-01-31T05:00:51.915637+0000","last_fullsized":"2024-01-31T05:00:51.915637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915410+0000","last_change":"2024-01-31T05:00:46.601762+0000","last_active":"2024-01-31T05:00:51.915410+0000","last_peered":"2024-01-31T05:00:51.915410+0000","last_clean":"2024-01-31T05:00:51.915410+0000","last_became_active":"2024-01-31T05:00:46.600161+0000","last_became_peered":"2024-01-31T05:00:46.600161+0000","last_unstale":"2024-01-31T05:00:51.915410+0000","last_undegraded":"2024-01-31T05:00:51.915410+0000","last_fullsized":"2024-01-31T05:00:51.915410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915285+0000","last_change":"2024-01-31T05:00:46.588560+0000","last_active":"2024-01-31T05:00:51.915285+0000","last_peered":"2024-01-31T05:00:51.915285+0000","last_clean":"2024-01-31T05:00:51.915285+0000","last_became_active":"2024-01-31T05:00:46.588291+0000","last_became_peered":"2024-01-31T05:00:46.588291+0000","last_unstale":"2024-01-31T05:00:51.915285+0000","last_undegraded":"2024-01-31T05:00:51.915285+0000","last_fullsized":"2024-01-31T05:00:51.915285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914247+0000","last_change":"2024-01-31T05:00:46.570493+0000","last_active":"2024-01-31T05:00:51.914247+0000","last_peered":"2024-01-31T05:00:51.914247+0000","last_clean":"2024-01-31T05:00:51.914247+0000","last_became_active":"2024-01-31T05:00:46.570196+0000","last_became_peered":"2024-01-31T05:00:46.570196+0000","last_unstale":"2024-01-31T05:00:51.914247+0000","last_undegraded":"2024-01-31T05:00:51.914247+0000","last_fullsized":"2024-01-31T05:00:51.914247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674510+0000","last_change":"2024-01-31T05:00:46.580076+0000","last_active":"2024-01-31T05:00:51.674510+0000","last_peered":"2024-01-31T05:00:51.674510+0000","last_clean":"2024-01-31T05:00:51.674510+0000","last_became_active":"2024-01-31T05:00:46.578312+0000","last_became_peered":"2024-01-31T05:00:46.578312+0000","last_unstale":"2024-01-31T05:00:51.674510+0000","last_undegraded":"2024-01-31T05:00:51.674510+0000","last_fullsized":"2024-01-31T05:00:51.674510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914636+0000","last_change":"2024-01-31T05:00:46.577353+0000","last_active":"2024-01-31T05:00:51.914636+0000","last_peered":"2024-01-31T05:00:51.914636+0000","last_clean":"2024-01-31T05:00:51.914636+0000","last_became_active":"2024-01-31T05:00:46.577019+0000","last_became_peered":"2024-01-31T05:00:46.577019+0000","last_unstale":"2024-01-31T05:00:51.914636+0000","last_undegraded":"2024-01-31T05:00:51.914636+0000","last_fullsized":"2024-01-31T05:00:51.914636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:09:13.929721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912937+0000","last_change":"2024-01-31T05:00:46.575406+0000","last_active":"2024-01-31T05:00:51.912937+0000","last_peered":"2024-01-31T05:00:51.912937+0000","last_clean":"2024-01-31T05:00:51.912937+0000","last_became_active":"2024-01-31T05:00:46.575056+0000","last_became_peered":"2024-01-31T05:00:46.575056+0000","last_unstale":"2024-01-31T05:00:51.912937+0000","last_undegraded":"2024-01-31T05:00:51.912937+0000","last_fullsized":"2024-01-31T05:00:51.912937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915174+0000","last_change":"2024-01-31T05:00:46.582321+0000","last_active":"2024-01-31T05:00:51.915174+0000","last_peered":"2024-01-31T05:00:51.915174+0000","last_clean":"2024-01-31T05:00:51.915174+0000","last_became_active":"2024-01-31T05:00:46.581547+0000","last_became_peered":"2024-01-31T05:00:46.581547+0000","last_unstale":"2024-01-31T05:00:51.915174+0000","last_undegraded":"2024-01-31T05:00:51.915174+0000","last_fullsized":"2024-01-31T05:00:51.915174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914144+0000","last_change":"2024-01-31T05:00:46.575296+0000","last_active":"2024-01-31T05:00:51.914144+0000","last_peered":"2024-01-31T05:00:51.914144+0000","last_clean":"2024-01-31T05:00:51.914144+0000","last_became_active":"2024-01-31T05:00:46.574983+0000","last_became_peered":"2024-01-31T05:00:46.574983+0000","last_unstale":"2024-01-31T05:00:51.914144+0000","last_undegraded":"2024-01-31T05:00:51.914144+0000","last_fullsized":"2024-01-31T05:00:51.914144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680670+0000","last_change":"2024-01-31T05:00:46.574733+0000","last_active":"2024-01-31T05:00:51.680670+0000","last_peered":"2024-01-31T05:00:51.680670+0000","last_clean":"2024-01-31T05:00:51.680670+0000","last_became_active":"2024-01-31T05:00:46.574335+0000","last_became_peered":"2024-01-31T05:00:46.574335+0000","last_unstale":"2024-01-31T05:00:51.680670+0000","last_undegraded":"2024-01-31T05:00:51.680670+0000","last_fullsized":"2024-01-31T05:00:51.680670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680984+0000","last_change":"2024-01-31T05:00:46.578081+0000","last_active":"2024-01-31T05:00:51.680984+0000","last_peered":"2024-01-31T05:00:51.680984+0000","last_clean":"2024-01-31T05:00:51.680984+0000","last_became_active":"2024-01-31T05:00:46.577576+0000","last_became_peered":"2024-01-31T05:00:46.577576+0000","last_unstale":"2024-01-31T05:00:51.680984+0000","last_undegraded":"2024-01-31T05:00:51.680984+0000","last_fullsized":"2024-01-31T05:00:51.680984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915608+0000","last_change":"2024-01-31T05:00:46.592072+0000","last_active":"2024-01-31T05:00:51.915608+0000","last_peered":"2024-01-31T05:00:51.915608+0000","last_clean":"2024-01-31T05:00:51.915608+0000","last_became_active":"2024-01-31T05:00:46.591682+0000","last_became_peered":"2024-01-31T05:00:46.591682+0000","last_unstale":"2024-01-31T05:00:51.915608+0000","last_undegraded":"2024-01-31T05:00:51.915608+0000","last_fullsized":"2024-01-31T05:00:51.915608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:13:21.174300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":106194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.665}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":106652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":696508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":106126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.71}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:01:10.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:01:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.506+0000 7fe6dde00640 1 Processor -- start 2024-01-31T05:01:10.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.506+0000 7fe6dde00640 1 -- start start 2024-01-31T05:01:10.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6dde00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 0x7fe6d80cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6dde00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6d80ce430 0x7fe6d8160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6dde00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d80abcf0 con 0x7fe6d8161500 2024-01-31T05:01:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6dde00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d8161990 con 0x7fe6d80ce430 2024-01-31T05:01:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6dde00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d815d900 con 0x7fe6d815d520 2024-01-31T05:01:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6d7400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6d8161500 0x7fe6d80b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41700/0 (socket says 172.21.15.97:41700) 2024-01-31T05:01:10.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.508+0000 7fe6d7400640 1 -- 172.21.15.97:0/1154757348 learned_addr learned my addr 172.21.15.97:0/1154757348 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 0x7fe6d80cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d6000640 1 --2- 172.21.15.97:0/1154757348 >> v2:172.21.15.188:3300/0 conn(0x7fe6d80ce430 0x7fe6d8160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723619241 0 0) 0x7fe6d80abcf0 con 0x7fe6d8161500 2024-01-31T05:01:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b8003520 con 0x7fe6d8161500 2024-01-31T05:01:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1864424636 0 0) 0x7fe6b8003520 con 0x7fe6d8161500 2024-01-31T05:01:10.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 msgr2=0x7fe6d80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 0x7fe6d80cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 >> v2:172.21.15.188:3300/0 conn(0x7fe6d80ce430 msgr2=0x7fe6d8160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 --2- 172.21.15.97:0/1154757348 >> v2:172.21.15.188:3300/0 conn(0x7fe6d80ce430 0x7fe6d8160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d806ca40 con 0x7fe6d8161500 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fe6d00019f0 con 0x7fe6d8161500 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 >> v1:172.21.15.97:6789/0 conn(0x7fe6d8161500 legacy=0x7fe6d80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 0x7fe6b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b8004d00 0x7fe6b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d8161990 con 0x7fe6b8003970 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d815d900 con 0x7fe6b8004d00 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.509+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b800b980 con 0x7fe6d815d520 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6000640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 0x7fe6b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d7400640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b8004d00 0x7fe6b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6a00640 1 -- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 msgr2=0x7fe6b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d815d520 0x7fe6b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6a00640 1 -- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b8004d00 msgr2=0x7fe6b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b8004d00 0x7fe6b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d6a00640 1 -- 172.21.15.97:0/1154757348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b8003520 con 0x7fe6b8003970 2024-01-31T05:01:10.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.510+0000 7fe6d7400640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6b8004d00 0x7fe6b8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.511+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009ac0 tx=0x7fe6b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94d0438996f80ce server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c802c6b0 con 0x7fe6b8003970 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c802c810 con 0x7fe6b8003970 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6d5600640 1 -- 172.21.15.97:0/1154757348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c802cba0 con 0x7fe6b8003970 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6dde00640 1 -- 172.21.15.97:0/1154757348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b8003970 msgr2=0x7fe6b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6dde00640 1 --2- 172.21.15.97:0/1154757348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6b8003970 0x7fe6b8003d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009ac0 tx=0x7fe6b80057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6d7400640 1 -- 172.21.15.97:0/1154757348 reap_dead start 2024-01-31T05:01:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6dde00640 1 -- 172.21.15.97:0/1154757348 shutdown_connections 2024-01-31T05:01:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.512+0000 7fe6dde00640 1 -- 172.21.15.97:0/1154757348 >> 172.21.15.97:0/1154757348 conn(0x7fe6d80b0a10 msgr2=0x7fe6d80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.513+0000 7fe6dde00640 1 -- 172.21.15.97:0/1154757348 shutdown_connections 2024-01-31T05:01:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.513+0000 7fe6dde00640 1 -- 172.21.15.97:0/1154757348 wait complete. 2024-01-31T05:01:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.514+0000 7fe6dde00640 1 Processor -- start 2024-01-31T05:01:10.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.514+0000 7fe6dde00640 1 -- start start 2024-01-31T05:01:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.515+0000 7fe6dde00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b8003970 0x7fe6d809b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.515+0000 7fe6dde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 0x7fe6d8148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.515+0000 7fe6dde00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d806d670 0x7fe6d806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.515+0000 7fe6dde00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d806e030 con 0x7fe6d81f7500 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.515+0000 7fe6dde00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d809be70 con 0x7fe6d806d670 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.515+0000 7fe6dde00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d809c150 con 0x7fe6b8003970 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 0x7fe6d8148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 0x7fe6d8148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43044/0 (socket says 172.21.15.97:43044) 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 -- 172.21.15.97:0/3181306903 learned_addr learned my addr 172.21.15.97:0/3181306903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b8003970 0x7fe6d809b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d7400640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d806d670 0x7fe6d806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 -- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b8003970 msgr2=0x7fe6d809b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b8003970 0x7fe6d809b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 -- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d806d670 msgr2=0x7fe6d806daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d806d670 0x7fe6d806daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 -- 172.21.15.97:0/3181306903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d809c430 con 0x7fe6d81f7500 2024-01-31T05:01:10.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.516+0000 7fe6d6000640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 0x7fe6d8148780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00116d0 tx=0x7fe6c0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.517+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000f9c0 con 0x7fe6d81f7500 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.517+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c000fb20 con 0x7fe6d81f7500 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.517+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d815dbc0 con 0x7fe6d81f7500 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.517+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000fe90 con 0x7fe6d81f7500 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.517+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d8148cc0 con 0x7fe6d81f7500 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.518+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d8160d50 con 0x7fe6d81f7500 2024-01-31T05:01:10.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.519+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe6c0015460 con 0x7fe6d81f7500 2024-01-31T05:01:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.520+0000 7fe6cf400640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe6b006af00 0x7fe6b006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.520+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe6c008a140 con 0x7fe6d81f7500 2024-01-31T05:01:10.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.520+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe6b006af00 0x7fe6b006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:10.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.521+0000 7fe6d6a00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe6b006af00 0x7fe6b006d3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8036000 tx=0x7fe6c8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.524+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe6c0054d20 con 0x7fe6d81f7500 2024-01-31T05:01:10.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.722+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6d815da40 con 0x7fe6b006af00 2024-01-31T05:01:10.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.733+0000 7fe6cf400640 1 -- 172.21.15.97:0/3181306903 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345107 (secure 0 0 0) 0x7fe6c8004140 con 0x7fe6b006af00 2024-01-31T05:01:10.738 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:10.740 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:01:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe6b006af00 msgr2=0x7fe6b006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe6b006af00 0x7fe6b006d3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8036000 tx=0x7fe6c8030000 comp rx=0 tx=0).stop 2024-01-31T05:01:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 msgr2=0x7fe6d8148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 0x7fe6d8148780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00116d0 tx=0x7fe6c0011b90 comp rx=0 tx=0).stop 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 shutdown_connections 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d806d670 0x7fe6d806daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe6b006af00 0x7fe6b006d3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6d81f7500 0x7fe6d8148780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 --2- 172.21.15.97:0/3181306903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6b8003970 0x7fe6d809b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.740+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 >> 172.21.15.97:0/3181306903 conn(0x7fe6d80b0a10 msgr2=0x7fe6d80b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.741+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 shutdown_connections 2024-01-31T05:01:10.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:10.741+0000 7fe6dde00640 1 -- 172.21.15.97:0/3181306903 wait complete. 2024-01-31T05:01:10.759 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-01-31T05:01:09.557111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2038292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679907+0000","last_change":"2024-01-31T05:00:45.571880+0000","last_active":"2024-01-31T05:00:51.679907+0000","last_peered":"2024-01-31T05:00:51.679907+0000","last_clean":"2024-01-31T05:00:51.679907+0000","last_became_active":"2024-01-31T05:00:45.571633+0000","last_became_peered":"2024-01-31T05:00:45.571633+0000","last_unstale":"2024-01-31T05:00:51.679907+0000","last_undegraded":"2024-01-31T05:00:51.679907+0000","last_fullsized":"2024-01-31T05:00:51.679907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681114+0000","last_change":"2024-01-31T05:00:46.593889+0000","last_active":"2024-01-31T05:00:51.681114+0000","last_peered":"2024-01-31T05:00:51.681114+0000","last_clean":"2024-01-31T05:00:51.681114+0000","last_became_active":"2024-01-31T05:00:46.593583+0000","last_became_peered":"2024-01-31T05:00:46.593583+0000","last_unstale":"2024-01-31T05:00:51.681114+0000","last_undegraded":"2024-01-31T05:00:51.681114+0000","last_fullsized":"2024-01-31T05:00:51.681114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914795+0000","last_change":"2024-01-31T05:00:50.653143+0000","last_active":"2024-01-31T05:00:51.914795+0000","last_peered":"2024-01-31T05:00:51.914795+0000","last_clean":"2024-01-31T05:00:51.914795+0000","last_became_active":"2024-01-31T05:00:50.652831+0000","last_became_peered":"2024-01-31T05:00:50.652831+0000","last_unstale":"2024-01-31T05:00:51.914795+0000","last_undegraded":"2024-01-31T05:00:51.914795+0000","last_fullsized":"2024-01-31T05:00:51.914795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:44.981765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681288+0000","last_change":"2024-01-31T05:00:45.578379+0000","last_active":"2024-01-31T05:00:51.681288+0000","last_peered":"2024-01-31T05:00:51.681288+0000","last_clean":"2024-01-31T05:00:51.681288+0000","last_became_active":"2024-01-31T05:00:45.574393+0000","last_became_peered":"2024-01-31T05:00:45.574393+0000","last_unstale":"2024-01-31T05:00:51.681288+0000","last_undegraded":"2024-01-31T05:00:51.681288+0000","last_fullsized":"2024-01-31T05:00:51.681288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681650+0000","last_change":"2024-01-31T05:00:46.573630+0000","last_active":"2024-01-31T05:00:51.681650+0000","last_peered":"2024-01-31T05:00:51.681650+0000","last_clean":"2024-01-31T05:00:51.681650+0000","last_became_active":"2024-01-31T05:00:46.572199+0000","last_became_peered":"2024-01-31T05:00:46.572199+0000","last_unstale":"2024-01-31T05:00:51.681650+0000","last_undegraded":"2024-01-31T05:00:51.681650+0000","last_fullsized":"2024-01-31T05:00:51.681650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912863+0000","last_change":"2024-01-31T05:00:50.644804+0000","last_active":"2024-01-31T05:00:51.912863+0000","last_peered":"2024-01-31T05:00:51.912863+0000","last_clean":"2024-01-31T05:00:51.912863+0000","last_became_active":"2024-01-31T05:00:50.640815+0000","last_became_peered":"2024-01-31T05:00:50.640815+0000","last_unstale":"2024-01-31T05:00:51.912863+0000","last_undegraded":"2024-01-31T05:00:51.912863+0000","last_fullsized":"2024-01-31T05:00:51.912863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:03:57.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915593+0000","last_change":"2024-01-31T05:00:45.582233+0000","last_active":"2024-01-31T05:00:51.915593+0000","last_peered":"2024-01-31T05:00:51.915593+0000","last_clean":"2024-01-31T05:00:51.915593+0000","last_became_active":"2024-01-31T05:00:45.578094+0000","last_became_peered":"2024-01-31T05:00:45.578094+0000","last_unstale":"2024-01-31T05:00:51.915593+0000","last_undegraded":"2024-01-31T05:00:51.915593+0000","last_fullsized":"2024-01-31T05:00:51.915593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:21:00.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913671+0000","last_change":"2024-01-31T05:00:46.589171+0000","last_active":"2024-01-31T05:00:51.913671+0000","last_peered":"2024-01-31T05:00:51.913671+0000","last_clean":"2024-01-31T05:00:51.913671+0000","last_became_active":"2024-01-31T05:00:46.588749+0000","last_became_peered":"2024-01-31T05:00:46.588749+0000","last_unstale":"2024-01-31T05:00:51.913671+0000","last_undegraded":"2024-01-31T05:00:51.913671+0000","last_fullsized":"2024-01-31T05:00:51.913671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914061+0000","last_change":"2024-01-31T05:00:50.635774+0000","last_active":"2024-01-31T05:00:51.914061+0000","last_peered":"2024-01-31T05:00:51.914061+0000","last_clean":"2024-01-31T05:00:51.914061+0000","last_became_active":"2024-01-31T05:00:50.635404+0000","last_became_peered":"2024-01-31T05:00:50.635404+0000","last_unstale":"2024-01-31T05:00:51.914061+0000","last_undegraded":"2024-01-31T05:00:51.914061+0000","last_fullsized":"2024-01-31T05:00:51.914061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:01:05.575230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915820+0000","last_change":"2024-01-31T05:00:45.583051+0000","last_active":"2024-01-31T05:00:51.915820+0000","last_peered":"2024-01-31T05:00:51.915820+0000","last_clean":"2024-01-31T05:00:51.915820+0000","last_became_active":"2024-01-31T05:00:45.582464+0000","last_became_peered":"2024-01-31T05:00:45.582464+0000","last_unstale":"2024-01-31T05:00:51.915820+0000","last_undegraded":"2024-01-31T05:00:51.915820+0000","last_fullsized":"2024-01-31T05:00:51.915820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:46:05.159804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674284+0000","last_change":"2024-01-31T05:00:46.593796+0000","last_active":"2024-01-31T05:00:51.674284+0000","last_peered":"2024-01-31T05:00:51.674284+0000","last_clean":"2024-01-31T05:00:51.674284+0000","last_became_active":"2024-01-31T05:00:46.593485+0000","last_became_peered":"2024-01-31T05:00:46.593485+0000","last_unstale":"2024-01-31T05:00:51.674284+0000","last_undegraded":"2024-01-31T05:00:51.674284+0000","last_fullsized":"2024-01-31T05:00:51.674284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:51:45.113263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912620+0000","last_change":"2024-01-31T05:00:50.645582+0000","last_active":"2024-01-31T05:00:51.912620+0000","last_peered":"2024-01-31T05:00:51.912620+0000","last_clean":"2024-01-31T05:00:51.912620+0000","last_became_active":"2024-01-31T05:00:50.645198+0000","last_became_peered":"2024-01-31T05:00:50.645198+0000","last_unstale":"2024-01-31T05:00:51.912620+0000","last_undegraded":"2024-01-31T05:00:51.912620+0000","last_fullsized":"2024-01-31T05:00:51.912620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:07:24.039515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673371+0000","last_change":"2024-01-31T05:00:46.581785+0000","last_active":"2024-01-31T05:00:51.673371+0000","last_peered":"2024-01-31T05:00:51.673371+0000","last_clean":"2024-01-31T05:00:51.673371+0000","last_became_active":"2024-01-31T05:00:46.581463+0000","last_became_peered":"2024-01-31T05:00:46.581463+0000","last_unstale":"2024-01-31T05:00:51.673371+0000","last_undegraded":"2024-01-31T05:00:51.673371+0000","last_fullsized":"2024-01-31T05:00:51.673371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:24:26.658435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673762+0000","last_change":"2024-01-31T05:00:45.566782+0000","last_active":"2024-01-31T05:00:51.673762+0000","last_peered":"2024-01-31T05:00:51.673762+0000","last_clean":"2024-01-31T05:00:51.673762+0000","last_became_active":"2024-01-31T05:00:45.565360+0000","last_became_peered":"2024-01-31T05:00:45.565360+0000","last_unstale":"2024-01-31T05:00:51.673762+0000","last_undegraded":"2024-01-31T05:00:51.673762+0000","last_fullsized":"2024-01-31T05:00:51.673762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:45.457514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674294+0000","last_change":"2024-01-31T05:00:50.639468+0000","last_active":"2024-01-31T05:00:51.674294+0000","last_peered":"2024-01-31T05:00:51.674294+0000","last_clean":"2024-01-31T05:00:51.674294+0000","last_became_active":"2024-01-31T05:00:50.639035+0000","last_became_peered":"2024-01-31T05:00:50.639035+0000","last_unstale":"2024-01-31T05:00:51.674294+0000","last_undegraded":"2024-01-31T05:00:51.674294+0000","last_fullsized":"2024-01-31T05:00:51.674294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:43:03.844139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915694+0000","last_change":"2024-01-31T05:00:46.583107+0000","last_active":"2024-01-31T05:00:51.915694+0000","last_peered":"2024-01-31T05:00:51.915694+0000","last_clean":"2024-01-31T05:00:51.915694+0000","last_became_active":"2024-01-31T05:00:46.582506+0000","last_became_peered":"2024-01-31T05:00:46.582506+0000","last_unstale":"2024-01-31T05:00:51.915694+0000","last_undegraded":"2024-01-31T05:00:51.915694+0000","last_fullsized":"2024-01-31T05:00:51.915694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917300+0000","last_change":"2024-01-31T05:00:45.585334+0000","last_active":"2024-01-31T05:00:51.917300+0000","last_peered":"2024-01-31T05:00:51.917300+0000","last_clean":"2024-01-31T05:00:51.917300+0000","last_became_active":"2024-01-31T05:00:45.585034+0000","last_became_peered":"2024-01-31T05:00:45.585034+0000","last_unstale":"2024-01-31T05:00:51.917300+0000","last_undegraded":"2024-01-31T05:00:51.917300+0000","last_fullsized":"2024-01-31T05:00:51.917300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915319+0000","last_change":"2024-01-31T05:00:50.642526+0000","last_active":"2024-01-31T05:00:51.915319+0000","last_peered":"2024-01-31T05:00:51.915319+0000","last_clean":"2024-01-31T05:00:51.915319+0000","last_became_active":"2024-01-31T05:00:50.642120+0000","last_became_peered":"2024-01-31T05:00:50.642120+0000","last_unstale":"2024-01-31T05:00:51.915319+0000","last_undegraded":"2024-01-31T05:00:51.915319+0000","last_fullsized":"2024-01-31T05:00:51.915319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674674+0000","last_change":"2024-01-31T05:00:46.585812+0000","last_active":"2024-01-31T05:00:51.674674+0000","last_peered":"2024-01-31T05:00:51.674674+0000","last_clean":"2024-01-31T05:00:51.674674+0000","last_became_active":"2024-01-31T05:00:46.585556+0000","last_became_peered":"2024-01-31T05:00:46.585556+0000","last_unstale":"2024-01-31T05:00:51.674674+0000","last_undegraded":"2024-01-31T05:00:51.674674+0000","last_fullsized":"2024-01-31T05:00:51.674674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:37.158256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676683+0000","last_change":"2024-01-31T05:00:45.580944+0000","last_active":"2024-01-31T05:00:51.676683+0000","last_peered":"2024-01-31T05:00:51.676683+0000","last_clean":"2024-01-31T05:00:51.676683+0000","last_became_active":"2024-01-31T05:00:45.580698+0000","last_became_peered":"2024-01-31T05:00:45.580698+0000","last_unstale":"2024-01-31T05:00:51.676683+0000","last_undegraded":"2024-01-31T05:00:51.676683+0000","last_fullsized":"2024-01-31T05:00:51.676683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:57.421278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916224+0000","last_change":"2024-01-31T05:00:50.659576+0000","last_active":"2024-01-31T05:00:51.916224+0000","last_peered":"2024-01-31T05:00:51.916224+0000","last_clean":"2024-01-31T05:00:51.916224+0000","last_became_active":"2024-01-31T05:00:50.659067+0000","last_became_peered":"2024-01-31T05:00:50.659067+0000","last_unstale":"2024-01-31T05:00:51.916224+0000","last_undegraded":"2024-01-31T05:00:51.916224+0000","last_fullsized":"2024-01-31T05:00:51.916224+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:11.777909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673016+0000","last_change":"2024-01-31T05:00:46.586716+0000","last_active":"2024-01-31T05:00:51.673016+0000","last_peered":"2024-01-31T05:00:51.673016+0000","last_clean":"2024-01-31T05:00:51.673016+0000","last_became_active":"2024-01-31T05:00:46.582889+0000","last_became_peered":"2024-01-31T05:00:46.582889+0000","last_unstale":"2024-01-31T05:00:51.673016+0000","last_undegraded":"2024-01-31T05:00:51.673016+0000","last_fullsized":"2024-01-31T05:00:51.673016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:07.424800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.572569+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.571841+0000","last_became_peered":"2024-01-31T05:00:45.571841+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914775+0000","last_change":"2024-01-31T05:00:50.637862+0000","last_active":"2024-01-31T05:00:51.914775+0000","last_peered":"2024-01-31T05:00:51.914775+0000","last_clean":"2024-01-31T05:00:51.914775+0000","last_became_active":"2024-01-31T05:00:50.637439+0000","last_became_peered":"2024-01-31T05:00:50.637439+0000","last_unstale":"2024-01-31T05:00:51.914775+0000","last_undegraded":"2024-01-31T05:00:51.914775+0000","last_fullsized":"2024-01-31T05:00:51.914775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676808+0000","last_change":"2024-01-31T05:00:46.588199+0000","last_active":"2024-01-31T05:00:51.676808+0000","last_peered":"2024-01-31T05:00:51.676808+0000","last_clean":"2024-01-31T05:00:51.676808+0000","last_became_active":"2024-01-31T05:00:46.585236+0000","last_became_peered":"2024-01-31T05:00:46.585236+0000","last_unstale":"2024-01-31T05:00:51.676808+0000","last_undegraded":"2024-01-31T05:00:51.676808+0000","last_fullsized":"2024-01-31T05:00:51.676808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:16.863565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675061+0000","last_change":"2024-01-31T05:00:45.577787+0000","last_active":"2024-01-31T05:00:51.675061+0000","last_peered":"2024-01-31T05:00:51.675061+0000","last_clean":"2024-01-31T05:00:51.675061+0000","last_became_active":"2024-01-31T05:00:45.577538+0000","last_became_peered":"2024-01-31T05:00:45.577538+0000","last_unstale":"2024-01-31T05:00:51.675061+0000","last_undegraded":"2024-01-31T05:00:51.675061+0000","last_fullsized":"2024-01-31T05:00:51.675061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:15.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912746+0000","last_change":"2024-01-31T05:00:50.630730+0000","last_active":"2024-01-31T05:00:51.912746+0000","last_peered":"2024-01-31T05:00:51.912746+0000","last_clean":"2024-01-31T05:00:51.912746+0000","last_became_active":"2024-01-31T05:00:50.629780+0000","last_became_peered":"2024-01-31T05:00:50.629780+0000","last_unstale":"2024-01-31T05:00:51.912746+0000","last_undegraded":"2024-01-31T05:00:51.912746+0000","last_fullsized":"2024-01-31T05:00:51.912746+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:07:58.674398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679833+0000","last_change":"2024-01-31T05:00:46.583111+0000","last_active":"2024-01-31T05:00:51.679833+0000","last_peered":"2024-01-31T05:00:51.679833+0000","last_clean":"2024-01-31T05:00:51.679833+0000","last_became_active":"2024-01-31T05:00:46.582827+0000","last_became_peered":"2024-01-31T05:00:46.582827+0000","last_unstale":"2024-01-31T05:00:51.679833+0000","last_undegraded":"2024-01-31T05:00:51.679833+0000","last_fullsized":"2024-01-31T05:00:51.679833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:40:42.367021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679389+0000","last_change":"2024-01-31T05:00:45.580550+0000","last_active":"2024-01-31T05:00:51.679389+0000","last_peered":"2024-01-31T05:00:51.679389+0000","last_clean":"2024-01-31T05:00:51.679389+0000","last_became_active":"2024-01-31T05:00:45.579549+0000","last_became_peered":"2024-01-31T05:00:45.579549+0000","last_unstale":"2024-01-31T05:00:51.679389+0000","last_undegraded":"2024-01-31T05:00:51.679389+0000","last_fullsized":"2024-01-31T05:00:51.679389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:43.683478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674795+0000","last_change":"2024-01-31T05:00:50.649333+0000","last_active":"2024-01-31T05:00:51.674795+0000","last_peered":"2024-01-31T05:00:51.674795+0000","last_clean":"2024-01-31T05:00:51.674795+0000","last_became_active":"2024-01-31T05:00:50.648927+0000","last_became_peered":"2024-01-31T05:00:50.648927+0000","last_unstale":"2024-01-31T05:00:51.674795+0000","last_undegraded":"2024-01-31T05:00:51.674795+0000","last_fullsized":"2024-01-31T05:00:51.674795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918351+0000","last_change":"2024-01-31T05:00:46.581800+0000","last_active":"2024-01-31T05:00:51.918351+0000","last_peered":"2024-01-31T05:00:51.918351+0000","last_clean":"2024-01-31T05:00:51.918351+0000","last_became_active":"2024-01-31T05:00:46.581435+0000","last_became_peered":"2024-01-31T05:00:46.581435+0000","last_unstale":"2024-01-31T05:00:51.918351+0000","last_undegraded":"2024-01-31T05:00:51.918351+0000","last_fullsized":"2024-01-31T05:00:51.918351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:55:29.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674998+0000","last_change":"2024-01-31T05:00:45.570866+0000","last_active":"2024-01-31T05:00:51.674998+0000","last_peered":"2024-01-31T05:00:51.674998+0000","last_clean":"2024-01-31T05:00:51.674998+0000","last_became_active":"2024-01-31T05:00:45.570644+0000","last_became_peered":"2024-01-31T05:00:45.570644+0000","last_unstale":"2024-01-31T05:00:51.674998+0000","last_undegraded":"2024-01-31T05:00:51.674998+0000","last_fullsized":"2024-01-31T05:00:51.674998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:54.101266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917676+0000","last_change":"2024-01-31T05:00:50.641145+0000","last_active":"2024-01-31T05:00:51.917676+0000","last_peered":"2024-01-31T05:00:51.917676+0000","last_clean":"2024-01-31T05:00:51.917676+0000","last_became_active":"2024-01-31T05:00:50.638626+0000","last_became_peered":"2024-01-31T05:00:50.638626+0000","last_unstale":"2024-01-31T05:00:51.917676+0000","last_undegraded":"2024-01-31T05:00:51.917676+0000","last_fullsized":"2024-01-31T05:00:51.917676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:46:01.447285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914572+0000","last_change":"2024-01-31T05:00:46.589437+0000","last_active":"2024-01-31T05:00:51.914572+0000","last_peered":"2024-01-31T05:00:51.914572+0000","last_clean":"2024-01-31T05:00:51.914572+0000","last_became_active":"2024-01-31T05:00:46.589134+0000","last_became_peered":"2024-01-31T05:00:46.589134+0000","last_unstale":"2024-01-31T05:00:51.914572+0000","last_undegraded":"2024-01-31T05:00:51.914572+0000","last_fullsized":"2024-01-31T05:00:51.914572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680322+0000","last_change":"2024-01-31T05:00:45.579033+0000","last_active":"2024-01-31T05:00:51.680322+0000","last_peered":"2024-01-31T05:00:51.680322+0000","last_clean":"2024-01-31T05:00:51.680322+0000","last_became_active":"2024-01-31T05:00:45.578730+0000","last_became_peered":"2024-01-31T05:00:45.578730+0000","last_unstale":"2024-01-31T05:00:51.680322+0000","last_undegraded":"2024-01-31T05:00:51.680322+0000","last_fullsized":"2024-01-31T05:00:51.680322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916714+0000","last_change":"2024-01-31T05:00:50.639581+0000","last_active":"2024-01-31T05:00:51.916714+0000","last_peered":"2024-01-31T05:00:51.916714+0000","last_clean":"2024-01-31T05:00:51.916714+0000","last_became_active":"2024-01-31T05:00:50.639192+0000","last_became_peered":"2024-01-31T05:00:50.639192+0000","last_unstale":"2024-01-31T05:00:51.916714+0000","last_undegraded":"2024-01-31T05:00:51.916714+0000","last_fullsized":"2024-01-31T05:00:51.916714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:28.385448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915081+0000","last_change":"2024-01-31T05:00:46.590496+0000","last_active":"2024-01-31T05:00:51.915081+0000","last_peered":"2024-01-31T05:00:51.915081+0000","last_clean":"2024-01-31T05:00:51.915081+0000","last_became_active":"2024-01-31T05:00:46.587386+0000","last_became_peered":"2024-01-31T05:00:46.587386+0000","last_unstale":"2024-01-31T05:00:51.915081+0000","last_undegraded":"2024-01-31T05:00:51.915081+0000","last_fullsized":"2024-01-31T05:00:51.915081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674054+0000","last_change":"2024-01-31T05:00:45.580484+0000","last_active":"2024-01-31T05:00:51.674054+0000","last_peered":"2024-01-31T05:00:51.674054+0000","last_clean":"2024-01-31T05:00:51.674054+0000","last_became_active":"2024-01-31T05:00:45.580179+0000","last_became_peered":"2024-01-31T05:00:45.580179+0000","last_unstale":"2024-01-31T05:00:51.674054+0000","last_undegraded":"2024-01-31T05:00:51.674054+0000","last_fullsized":"2024-01-31T05:00:51.674054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:54.780283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918220+0000","last_change":"2024-01-31T05:00:50.633651+0000","last_active":"2024-01-31T05:00:51.918220+0000","last_peered":"2024-01-31T05:00:51.918220+0000","last_clean":"2024-01-31T05:00:51.918220+0000","last_became_active":"2024-01-31T05:00:50.633176+0000","last_became_peered":"2024-01-31T05:00:50.633176+0000","last_unstale":"2024-01-31T05:00:51.918220+0000","last_undegraded":"2024-01-31T05:00:51.918220+0000","last_fullsized":"2024-01-31T05:00:51.918220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:41.984643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675265+0000","last_change":"2024-01-31T05:00:46.578636+0000","last_active":"2024-01-31T05:00:51.675265+0000","last_peered":"2024-01-31T05:00:51.675265+0000","last_clean":"2024-01-31T05:00:51.675265+0000","last_became_active":"2024-01-31T05:00:46.577232+0000","last_became_peered":"2024-01-31T05:00:46.577232+0000","last_unstale":"2024-01-31T05:00:51.675265+0000","last_undegraded":"2024-01-31T05:00:51.675265+0000","last_fullsized":"2024-01-31T05:00:51.675265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:37:22.658839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915084+0000","last_change":"2024-01-31T05:00:45.574367+0000","last_active":"2024-01-31T05:00:51.915084+0000","last_peered":"2024-01-31T05:00:51.915084+0000","last_clean":"2024-01-31T05:00:51.915084+0000","last_became_active":"2024-01-31T05:00:45.574010+0000","last_became_peered":"2024-01-31T05:00:45.574010+0000","last_unstale":"2024-01-31T05:00:51.915084+0000","last_undegraded":"2024-01-31T05:00:51.915084+0000","last_fullsized":"2024-01-31T05:00:51.915084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:12:03.585718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918910+0000","last_change":"2024-01-31T05:00:50.642311+0000","last_active":"2024-01-31T05:00:51.918910+0000","last_peered":"2024-01-31T05:00:51.918910+0000","last_clean":"2024-01-31T05:00:51.918910+0000","last_became_active":"2024-01-31T05:00:50.641937+0000","last_became_peered":"2024-01-31T05:00:50.641937+0000","last_unstale":"2024-01-31T05:00:51.918910+0000","last_undegraded":"2024-01-31T05:00:51.918910+0000","last_fullsized":"2024-01-31T05:00:51.918910+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:57.841272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916940+0000","last_change":"2024-01-31T05:00:46.589298+0000","last_active":"2024-01-31T05:00:51.916940+0000","last_peered":"2024-01-31T05:00:51.916940+0000","last_clean":"2024-01-31T05:00:51.916940+0000","last_became_active":"2024-01-31T05:00:46.589007+0000","last_became_peered":"2024-01-31T05:00:46.589007+0000","last_unstale":"2024-01-31T05:00:51.916940+0000","last_undegraded":"2024-01-31T05:00:51.916940+0000","last_fullsized":"2024-01-31T05:00:51.916940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:53:18.216173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673208+0000","last_change":"2024-01-31T05:00:45.573415+0000","last_active":"2024-01-31T05:00:51.673208+0000","last_peered":"2024-01-31T05:00:51.673208+0000","last_clean":"2024-01-31T05:00:51.673208+0000","last_became_active":"2024-01-31T05:00:45.573080+0000","last_became_peered":"2024-01-31T05:00:45.573080+0000","last_unstale":"2024-01-31T05:00:51.673208+0000","last_undegraded":"2024-01-31T05:00:51.673208+0000","last_fullsized":"2024-01-31T05:00:51.673208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:22.222326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680907+0000","last_change":"2024-01-31T05:00:50.642960+0000","last_active":"2024-01-31T05:00:51.680907+0000","last_peered":"2024-01-31T05:00:51.680907+0000","last_clean":"2024-01-31T05:00:51.680907+0000","last_became_active":"2024-01-31T05:00:50.640025+0000","last_became_peered":"2024-01-31T05:00:50.640025+0000","last_unstale":"2024-01-31T05:00:51.680907+0000","last_undegraded":"2024-01-31T05:00:51.680907+0000","last_fullsized":"2024-01-31T05:00:51.680907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:28:26.315820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677292+0000","last_change":"2024-01-31T05:00:46.592032+0000","last_active":"2024-01-31T05:00:51.677292+0000","last_peered":"2024-01-31T05:00:51.677292+0000","last_clean":"2024-01-31T05:00:51.677292+0000","last_became_active":"2024-01-31T05:00:46.588029+0000","last_became_peered":"2024-01-31T05:00:46.588029+0000","last_unstale":"2024-01-31T05:00:51.677292+0000","last_undegraded":"2024-01-31T05:00:51.677292+0000","last_fullsized":"2024-01-31T05:00:51.677292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:42.890080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673909+0000","last_change":"2024-01-31T05:00:45.562238+0000","last_active":"2024-01-31T05:00:51.673909+0000","last_peered":"2024-01-31T05:00:51.673909+0000","last_clean":"2024-01-31T05:00:51.673909+0000","last_became_active":"2024-01-31T05:00:45.561726+0000","last_became_peered":"2024-01-31T05:00:45.561726+0000","last_unstale":"2024-01-31T05:00:51.673909+0000","last_undegraded":"2024-01-31T05:00:51.673909+0000","last_fullsized":"2024-01-31T05:00:51.673909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:13.468671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912656+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:00:51.912656+0000","last_peered":"2024-01-31T05:00:51.912656+0000","last_clean":"2024-01-31T05:00:51.912656+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:00:51.912656+0000","last_undegraded":"2024-01-31T05:00:51.912656+0000","last_fullsized":"2024-01-31T05:00:51.912656+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674754+0000","last_change":"2024-01-31T05:00:50.629234+0000","last_active":"2024-01-31T05:00:51.674754+0000","last_peered":"2024-01-31T05:00:51.674754+0000","last_clean":"2024-01-31T05:00:51.674754+0000","last_became_active":"2024-01-31T05:00:50.628308+0000","last_became_peered":"2024-01-31T05:00:50.628308+0000","last_unstale":"2024-01-31T05:00:51.674754+0000","last_undegraded":"2024-01-31T05:00:51.674754+0000","last_fullsized":"2024-01-31T05:00:51.674754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:21:35.300095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674590+0000","last_change":"2024-01-31T05:00:46.583227+0000","last_active":"2024-01-31T05:00:51.674590+0000","last_peered":"2024-01-31T05:00:51.674590+0000","last_clean":"2024-01-31T05:00:51.674590+0000","last_became_active":"2024-01-31T05:00:46.582532+0000","last_became_peered":"2024-01-31T05:00:46.582532+0000","last_unstale":"2024-01-31T05:00:51.674590+0000","last_undegraded":"2024-01-31T05:00:51.674590+0000","last_fullsized":"2024-01-31T05:00:51.674590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:53:51.592799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918513+0000","last_change":"2024-01-31T05:00:45.583848+0000","last_active":"2024-01-31T05:00:51.918513+0000","last_peered":"2024-01-31T05:00:51.918513+0000","last_clean":"2024-01-31T05:00:51.918513+0000","last_became_active":"2024-01-31T05:00:45.583492+0000","last_became_peered":"2024-01-31T05:00:45.583492+0000","last_unstale":"2024-01-31T05:00:51.918513+0000","last_undegraded":"2024-01-31T05:00:51.918513+0000","last_fullsized":"2024-01-31T05:00:51.918513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912922+0000","last_change":"2024-01-31T05:00:50.642193+0000","last_active":"2024-01-31T05:00:51.912922+0000","last_peered":"2024-01-31T05:00:51.912922+0000","last_clean":"2024-01-31T05:00:51.912922+0000","last_became_active":"2024-01-31T05:00:50.639911+0000","last_became_peered":"2024-01-31T05:00:50.639911+0000","last_unstale":"2024-01-31T05:00:51.912922+0000","last_undegraded":"2024-01-31T05:00:51.912922+0000","last_fullsized":"2024-01-31T05:00:51.912922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:50:44.744734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915799+0000","last_change":"2024-01-31T05:00:46.580861+0000","last_active":"2024-01-31T05:00:51.915799+0000","last_peered":"2024-01-31T05:00:51.915799+0000","last_clean":"2024-01-31T05:00:51.915799+0000","last_became_active":"2024-01-31T05:00:46.580588+0000","last_became_peered":"2024-01-31T05:00:46.580588+0000","last_unstale":"2024-01-31T05:00:51.915799+0000","last_undegraded":"2024-01-31T05:00:51.915799+0000","last_fullsized":"2024-01-31T05:00:51.915799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681122+0000","last_change":"2024-01-31T05:00:45.567443+0000","last_active":"2024-01-31T05:00:51.681122+0000","last_peered":"2024-01-31T05:00:51.681122+0000","last_clean":"2024-01-31T05:00:51.681122+0000","last_became_active":"2024-01-31T05:00:45.567160+0000","last_became_peered":"2024-01-31T05:00:45.567160+0000","last_unstale":"2024-01-31T05:00:51.681122+0000","last_undegraded":"2024-01-31T05:00:51.681122+0000","last_fullsized":"2024-01-31T05:00:51.681122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679437+0000","last_change":"2024-01-31T05:00:50.647210+0000","last_active":"2024-01-31T05:00:51.679437+0000","last_peered":"2024-01-31T05:00:51.679437+0000","last_clean":"2024-01-31T05:00:51.679437+0000","last_became_active":"2024-01-31T05:00:50.646792+0000","last_became_peered":"2024-01-31T05:00:50.646792+0000","last_unstale":"2024-01-31T05:00:51.679437+0000","last_undegraded":"2024-01-31T05:00:51.679437+0000","last_fullsized":"2024-01-31T05:00:51.679437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915671+0000","last_change":"2024-01-31T05:00:46.593737+0000","last_active":"2024-01-31T05:00:51.915671+0000","last_peered":"2024-01-31T05:00:51.915671+0000","last_clean":"2024-01-31T05:00:51.915671+0000","last_became_active":"2024-01-31T05:00:46.593484+0000","last_became_peered":"2024-01-31T05:00:46.593484+0000","last_unstale":"2024-01-31T05:00:51.915671+0000","last_undegraded":"2024-01-31T05:00:51.915671+0000","last_fullsized":"2024-01-31T05:00:51.915671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:38:14.536828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679315+0000","last_change":"2024-01-31T05:00:45.582469+0000","last_active":"2024-01-31T05:00:51.679315+0000","last_peered":"2024-01-31T05:00:51.679315+0000","last_clean":"2024-01-31T05:00:51.679315+0000","last_became_active":"2024-01-31T05:00:45.582151+0000","last_became_peered":"2024-01-31T05:00:45.582151+0000","last_unstale":"2024-01-31T05:00:51.679315+0000","last_undegraded":"2024-01-31T05:00:51.679315+0000","last_fullsized":"2024-01-31T05:00:51.679315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:36:17.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672556+0000","last_change":"2024-01-31T05:00:50.642993+0000","last_active":"2024-01-31T05:00:51.672556+0000","last_peered":"2024-01-31T05:00:51.672556+0000","last_clean":"2024-01-31T05:00:51.672556+0000","last_became_active":"2024-01-31T05:00:50.642558+0000","last_became_peered":"2024-01-31T05:00:50.642558+0000","last_unstale":"2024-01-31T05:00:51.672556+0000","last_undegraded":"2024-01-31T05:00:51.672556+0000","last_fullsized":"2024-01-31T05:00:51.672556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:07:54.060532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914870+0000","last_change":"2024-01-31T05:00:46.597235+0000","last_active":"2024-01-31T05:00:51.914870+0000","last_peered":"2024-01-31T05:00:51.914870+0000","last_clean":"2024-01-31T05:00:51.914870+0000","last_became_active":"2024-01-31T05:00:46.592597+0000","last_became_peered":"2024-01-31T05:00:46.592597+0000","last_unstale":"2024-01-31T05:00:51.914870+0000","last_undegraded":"2024-01-31T05:00:51.914870+0000","last_fullsized":"2024-01-31T05:00:51.914870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:55.281235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915788+0000","last_change":"2024-01-31T05:00:45.583433+0000","last_active":"2024-01-31T05:00:51.915788+0000","last_peered":"2024-01-31T05:00:51.915788+0000","last_clean":"2024-01-31T05:00:51.915788+0000","last_became_active":"2024-01-31T05:00:45.582873+0000","last_became_peered":"2024-01-31T05:00:45.582873+0000","last_unstale":"2024-01-31T05:00:51.915788+0000","last_undegraded":"2024-01-31T05:00:51.915788+0000","last_fullsized":"2024-01-31T05:00:51.915788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:44:30.759721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913367+0000","last_change":"2024-01-31T05:00:50.651478+0000","last_active":"2024-01-31T05:00:51.913367+0000","last_peered":"2024-01-31T05:00:51.913367+0000","last_clean":"2024-01-31T05:00:51.913367+0000","last_became_active":"2024-01-31T05:00:50.650775+0000","last_became_peered":"2024-01-31T05:00:50.650775+0000","last_unstale":"2024-01-31T05:00:51.913367+0000","last_undegraded":"2024-01-31T05:00:51.913367+0000","last_fullsized":"2024-01-31T05:00:51.913367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672742+0000","last_change":"2024-01-31T05:00:46.588386+0000","last_active":"2024-01-31T05:00:51.672742+0000","last_peered":"2024-01-31T05:00:51.672742+0000","last_clean":"2024-01-31T05:00:51.672742+0000","last_became_active":"2024-01-31T05:00:46.588150+0000","last_became_peered":"2024-01-31T05:00:46.588150+0000","last_unstale":"2024-01-31T05:00:51.672742+0000","last_undegraded":"2024-01-31T05:00:51.672742+0000","last_fullsized":"2024-01-31T05:00:51.672742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:38.932253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674904+0000","last_change":"2024-01-31T05:00:45.572910+0000","last_active":"2024-01-31T05:00:51.674904+0000","last_peered":"2024-01-31T05:00:51.674904+0000","last_clean":"2024-01-31T05:00:51.674904+0000","last_became_active":"2024-01-31T05:00:45.570281+0000","last_became_peered":"2024-01-31T05:00:45.570281+0000","last_unstale":"2024-01-31T05:00:51.674904+0000","last_undegraded":"2024-01-31T05:00:51.674904+0000","last_fullsized":"2024-01-31T05:00:51.674904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681309+0000","last_change":"2024-01-31T05:00:50.646215+0000","last_active":"2024-01-31T05:00:51.681309+0000","last_peered":"2024-01-31T05:00:51.681309+0000","last_clean":"2024-01-31T05:00:51.681309+0000","last_became_active":"2024-01-31T05:00:50.640453+0000","last_became_peered":"2024-01-31T05:00:50.640453+0000","last_unstale":"2024-01-31T05:00:51.681309+0000","last_undegraded":"2024-01-31T05:00:51.681309+0000","last_fullsized":"2024-01-31T05:00:51.681309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:20.595164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912715+0000","last_change":"2024-01-31T05:00:46.582233+0000","last_active":"2024-01-31T05:00:51.912715+0000","last_peered":"2024-01-31T05:00:51.912715+0000","last_clean":"2024-01-31T05:00:51.912715+0000","last_became_active":"2024-01-31T05:00:46.581941+0000","last_became_peered":"2024-01-31T05:00:46.581941+0000","last_unstale":"2024-01-31T05:00:51.912715+0000","last_undegraded":"2024-01-31T05:00:51.912715+0000","last_fullsized":"2024-01-31T05:00:51.912715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914727+0000","last_change":"2024-01-31T05:00:45.586099+0000","last_active":"2024-01-31T05:00:51.914727+0000","last_peered":"2024-01-31T05:00:51.914727+0000","last_clean":"2024-01-31T05:00:51.914727+0000","last_became_active":"2024-01-31T05:00:45.583149+0000","last_became_peered":"2024-01-31T05:00:45.583149+0000","last_unstale":"2024-01-31T05:00:51.914727+0000","last_undegraded":"2024-01-31T05:00:51.914727+0000","last_fullsized":"2024-01-31T05:00:51.914727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681581+0000","last_change":"2024-01-31T05:00:50.644931+0000","last_active":"2024-01-31T05:00:51.681581+0000","last_peered":"2024-01-31T05:00:51.681581+0000","last_clean":"2024-01-31T05:00:51.681581+0000","last_became_active":"2024-01-31T05:00:50.644554+0000","last_became_peered":"2024-01-31T05:00:50.644554+0000","last_unstale":"2024-01-31T05:00:51.681581+0000","last_undegraded":"2024-01-31T05:00:51.681581+0000","last_fullsized":"2024-01-31T05:00:51.681581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:23:08.930803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673424+0000","last_change":"2024-01-31T05:00:46.582885+0000","last_active":"2024-01-31T05:00:51.673424+0000","last_peered":"2024-01-31T05:00:51.673424+0000","last_clean":"2024-01-31T05:00:51.673424+0000","last_became_active":"2024-01-31T05:00:46.582585+0000","last_became_peered":"2024-01-31T05:00:46.582585+0000","last_unstale":"2024-01-31T05:00:51.673424+0000","last_undegraded":"2024-01-31T05:00:51.673424+0000","last_fullsized":"2024-01-31T05:00:51.673424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:15:39.727609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673905+0000","last_change":"2024-01-31T05:00:45.570263+0000","last_active":"2024-01-31T05:00:51.673905+0000","last_peered":"2024-01-31T05:00:51.673905+0000","last_clean":"2024-01-31T05:00:51.673905+0000","last_became_active":"2024-01-31T05:00:45.568414+0000","last_became_peered":"2024-01-31T05:00:45.568414+0000","last_unstale":"2024-01-31T05:00:51.673905+0000","last_undegraded":"2024-01-31T05:00:51.673905+0000","last_fullsized":"2024-01-31T05:00:51.673905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:02.658111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918156+0000","last_change":"2024-01-31T05:00:50.640753+0000","last_active":"2024-01-31T05:00:51.918156+0000","last_peered":"2024-01-31T05:00:51.918156+0000","last_clean":"2024-01-31T05:00:51.918156+0000","last_became_active":"2024-01-31T05:00:50.640179+0000","last_became_peered":"2024-01-31T05:00:50.640179+0000","last_unstale":"2024-01-31T05:00:51.918156+0000","last_undegraded":"2024-01-31T05:00:51.918156+0000","last_fullsized":"2024-01-31T05:00:51.918156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:39.555756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916848+0000","last_change":"2024-01-31T05:00:46.596210+0000","last_active":"2024-01-31T05:00:51.916848+0000","last_peered":"2024-01-31T05:00:51.916848+0000","last_clean":"2024-01-31T05:00:51.916848+0000","last_became_active":"2024-01-31T05:00:46.595941+0000","last_became_peered":"2024-01-31T05:00:46.595941+0000","last_unstale":"2024-01-31T05:00:51.916848+0000","last_undegraded":"2024-01-31T05:00:51.916848+0000","last_fullsized":"2024-01-31T05:00:51.916848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:09.854108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916254+0000","last_change":"2024-01-31T05:00:45.577952+0000","last_active":"2024-01-31T05:00:51.916254+0000","last_peered":"2024-01-31T05:00:51.916254+0000","last_clean":"2024-01-31T05:00:51.916254+0000","last_became_active":"2024-01-31T05:00:45.577632+0000","last_became_peered":"2024-01-31T05:00:45.577632+0000","last_unstale":"2024-01-31T05:00:51.916254+0000","last_undegraded":"2024-01-31T05:00:51.916254+0000","last_fullsized":"2024-01-31T05:00:51.916254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673947+0000","last_change":"2024-01-31T05:00:50.639630+0000","last_active":"2024-01-31T05:00:51.673947+0000","last_peered":"2024-01-31T05:00:51.673947+0000","last_clean":"2024-01-31T05:00:51.673947+0000","last_became_active":"2024-01-31T05:00:50.639187+0000","last_became_peered":"2024-01-31T05:00:50.639187+0000","last_unstale":"2024-01-31T05:00:51.673947+0000","last_undegraded":"2024-01-31T05:00:51.673947+0000","last_fullsized":"2024-01-31T05:00:51.673947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:41:38.465126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915506+0000","last_change":"2024-01-31T05:00:46.595225+0000","last_active":"2024-01-31T05:00:51.915506+0000","last_peered":"2024-01-31T05:00:51.915506+0000","last_clean":"2024-01-31T05:00:51.915506+0000","last_became_active":"2024-01-31T05:00:46.594892+0000","last_became_peered":"2024-01-31T05:00:46.594892+0000","last_unstale":"2024-01-31T05:00:51.915506+0000","last_undegraded":"2024-01-31T05:00:51.915506+0000","last_fullsized":"2024-01-31T05:00:51.915506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:57.797805+0000","last_change":"2024-01-31T05:00:45.575073+0000","last_active":"2024-01-31T05:00:57.797805+0000","last_peered":"2024-01-31T05:00:57.797805+0000","last_clean":"2024-01-31T05:00:57.797805+0000","last_became_active":"2024-01-31T05:00:45.574725+0000","last_became_peered":"2024-01-31T05:00:45.574725+0000","last_unstale":"2024-01-31T05:00:57.797805+0000","last_undegraded":"2024-01-31T05:00:57.797805+0000","last_fullsized":"2024-01-31T05:00:57.797805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674916+0000","last_change":"2024-01-31T05:00:50.640744+0000","last_active":"2024-01-31T05:00:51.674916+0000","last_peered":"2024-01-31T05:00:51.674916+0000","last_clean":"2024-01-31T05:00:51.674916+0000","last_became_active":"2024-01-31T05:00:50.640181+0000","last_became_peered":"2024-01-31T05:00:50.640181+0000","last_unstale":"2024-01-31T05:00:51.674916+0000","last_undegraded":"2024-01-31T05:00:51.674916+0000","last_fullsized":"2024-01-31T05:00:51.674916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673600+0000","last_change":"2024-01-31T05:00:46.587774+0000","last_active":"2024-01-31T05:00:51.673600+0000","last_peered":"2024-01-31T05:00:51.673600+0000","last_clean":"2024-01-31T05:00:51.673600+0000","last_became_active":"2024-01-31T05:00:46.587495+0000","last_became_peered":"2024-01-31T05:00:46.587495+0000","last_unstale":"2024-01-31T05:00:51.673600+0000","last_undegraded":"2024-01-31T05:00:51.673600+0000","last_fullsized":"2024-01-31T05:00:51.673600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:19:31.923714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917086+0000","last_change":"2024-01-31T05:00:45.577387+0000","last_active":"2024-01-31T05:00:51.917086+0000","last_peered":"2024-01-31T05:00:51.917086+0000","last_clean":"2024-01-31T05:00:51.917086+0000","last_became_active":"2024-01-31T05:00:45.577035+0000","last_became_peered":"2024-01-31T05:00:45.577035+0000","last_unstale":"2024-01-31T05:00:51.917086+0000","last_undegraded":"2024-01-31T05:00:51.917086+0000","last_fullsized":"2024-01-31T05:00:51.917086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914130+0000","last_change":"2024-01-31T05:00:50.641531+0000","last_active":"2024-01-31T05:00:51.914130+0000","last_peered":"2024-01-31T05:00:51.914130+0000","last_clean":"2024-01-31T05:00:51.914130+0000","last_became_active":"2024-01-31T05:00:50.641164+0000","last_became_peered":"2024-01-31T05:00:50.641164+0000","last_unstale":"2024-01-31T05:00:51.914130+0000","last_undegraded":"2024-01-31T05:00:51.914130+0000","last_fullsized":"2024-01-31T05:00:51.914130+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681764+0000","last_change":"2024-01-31T05:00:46.584219+0000","last_active":"2024-01-31T05:00:51.681764+0000","last_peered":"2024-01-31T05:00:51.681764+0000","last_clean":"2024-01-31T05:00:51.681764+0000","last_became_active":"2024-01-31T05:00:46.583942+0000","last_became_peered":"2024-01-31T05:00:46.583942+0000","last_unstale":"2024-01-31T05:00:51.681764+0000","last_undegraded":"2024-01-31T05:00:51.681764+0000","last_fullsized":"2024-01-31T05:00:51.681764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915433+0000","last_change":"2024-01-31T05:00:45.576533+0000","last_active":"2024-01-31T05:00:51.915433+0000","last_peered":"2024-01-31T05:00:51.915433+0000","last_clean":"2024-01-31T05:00:51.915433+0000","last_became_active":"2024-01-31T05:00:45.576327+0000","last_became_peered":"2024-01-31T05:00:45.576327+0000","last_unstale":"2024-01-31T05:00:51.915433+0000","last_undegraded":"2024-01-31T05:00:51.915433+0000","last_fullsized":"2024-01-31T05:00:51.915433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:09:58.811691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914596+0000","last_change":"2024-01-31T05:00:50.654987+0000","last_active":"2024-01-31T05:00:51.914596+0000","last_peered":"2024-01-31T05:00:51.914596+0000","last_clean":"2024-01-31T05:00:51.914596+0000","last_became_active":"2024-01-31T05:00:50.654461+0000","last_became_peered":"2024-01-31T05:00:50.654461+0000","last_unstale":"2024-01-31T05:00:51.914596+0000","last_undegraded":"2024-01-31T05:00:51.914596+0000","last_fullsized":"2024-01-31T05:00:51.914596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914829+0000","last_change":"2024-01-31T05:00:46.597875+0000","last_active":"2024-01-31T05:00:51.914829+0000","last_peered":"2024-01-31T05:00:51.914829+0000","last_clean":"2024-01-31T05:00:51.914829+0000","last_became_active":"2024-01-31T05:00:46.597494+0000","last_became_peered":"2024-01-31T05:00:46.597494+0000","last_unstale":"2024-01-31T05:00:51.914829+0000","last_undegraded":"2024-01-31T05:00:51.914829+0000","last_fullsized":"2024-01-31T05:00:51.914829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:14.648640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.574121+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.573715+0000","last_became_peered":"2024-01-31T05:00:45.573715+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676164+0000","last_change":"2024-01-31T05:00:50.642176+0000","last_active":"2024-01-31T05:00:51.676164+0000","last_peered":"2024-01-31T05:00:51.676164+0000","last_clean":"2024-01-31T05:00:51.676164+0000","last_became_active":"2024-01-31T05:00:50.639134+0000","last_became_peered":"2024-01-31T05:00:50.639134+0000","last_unstale":"2024-01-31T05:00:51.676164+0000","last_undegraded":"2024-01-31T05:00:51.676164+0000","last_fullsized":"2024-01-31T05:00:51.676164+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:11.485002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676564+0000","last_change":"2024-01-31T05:00:50.649633+0000","last_active":"2024-01-31T05:00:51.676564+0000","last_peered":"2024-01-31T05:00:51.676564+0000","last_clean":"2024-01-31T05:00:51.676564+0000","last_became_active":"2024-01-31T05:00:50.647001+0000","last_became_peered":"2024-01-31T05:00:50.647001+0000","last_unstale":"2024-01-31T05:00:51.676564+0000","last_undegraded":"2024-01-31T05:00:51.676564+0000","last_fullsized":"2024-01-31T05:00:51.676564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:42.992455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676393+0000","last_change":"2024-01-31T05:00:46.567858+0000","last_active":"2024-01-31T05:00:51.676393+0000","last_peered":"2024-01-31T05:00:51.676393+0000","last_clean":"2024-01-31T05:00:51.676393+0000","last_became_active":"2024-01-31T05:00:46.567351+0000","last_became_peered":"2024-01-31T05:00:46.567351+0000","last_unstale":"2024-01-31T05:00:51.676393+0000","last_undegraded":"2024-01-31T05:00:51.676393+0000","last_fullsized":"2024-01-31T05:00:51.676393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:07.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673099+0000","last_change":"2024-01-31T05:00:45.562694+0000","last_active":"2024-01-31T05:00:51.673099+0000","last_peered":"2024-01-31T05:00:51.673099+0000","last_clean":"2024-01-31T05:00:51.673099+0000","last_became_active":"2024-01-31T05:00:45.560991+0000","last_became_peered":"2024-01-31T05:00:45.560991+0000","last_unstale":"2024-01-31T05:00:51.673099+0000","last_undegraded":"2024-01-31T05:00:51.673099+0000","last_fullsized":"2024-01-31T05:00:51.673099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674525+0000","last_change":"2024-01-31T05:00:50.638454+0000","last_active":"2024-01-31T05:00:51.674525+0000","last_peered":"2024-01-31T05:00:51.674525+0000","last_clean":"2024-01-31T05:00:51.674525+0000","last_became_active":"2024-01-31T05:00:50.637600+0000","last_became_peered":"2024-01-31T05:00:50.637600+0000","last_unstale":"2024-01-31T05:00:51.674525+0000","last_undegraded":"2024-01-31T05:00:51.674525+0000","last_fullsized":"2024-01-31T05:00:51.674525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:16.246206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676944+0000","last_change":"2024-01-31T05:00:46.574103+0000","last_active":"2024-01-31T05:00:51.676944+0000","last_peered":"2024-01-31T05:00:51.676944+0000","last_clean":"2024-01-31T05:00:51.676944+0000","last_became_active":"2024-01-31T05:00:46.572921+0000","last_became_peered":"2024-01-31T05:00:46.572921+0000","last_unstale":"2024-01-31T05:00:51.676944+0000","last_undegraded":"2024-01-31T05:00:51.676944+0000","last_fullsized":"2024-01-31T05:00:51.676944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:36.342238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916504+0000","last_change":"2024-01-31T05:00:45.587813+0000","last_active":"2024-01-31T05:00:51.916504+0000","last_peered":"2024-01-31T05:00:51.916504+0000","last_clean":"2024-01-31T05:00:51.916504+0000","last_became_active":"2024-01-31T05:00:45.587473+0000","last_became_peered":"2024-01-31T05:00:45.587473+0000","last_unstale":"2024-01-31T05:00:51.916504+0000","last_undegraded":"2024-01-31T05:00:51.916504+0000","last_fullsized":"2024-01-31T05:00:51.916504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:44:11.745313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673739+0000","last_change":"2024-01-31T05:00:45.574390+0000","last_active":"2024-01-31T05:00:51.673739+0000","last_peered":"2024-01-31T05:00:51.673739+0000","last_clean":"2024-01-31T05:00:51.673739+0000","last_became_active":"2024-01-31T05:00:45.574121+0000","last_became_peered":"2024-01-31T05:00:45.574121+0000","last_unstale":"2024-01-31T05:00:51.673739+0000","last_undegraded":"2024-01-31T05:00:51.673739+0000","last_fullsized":"2024-01-31T05:00:51.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:01.042942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677784+0000","last_change":"2024-01-31T05:00:46.582109+0000","last_active":"2024-01-31T05:00:51.677784+0000","last_peered":"2024-01-31T05:00:51.677784+0000","last_clean":"2024-01-31T05:00:51.677784+0000","last_became_active":"2024-01-31T05:00:46.581814+0000","last_became_peered":"2024-01-31T05:00:46.581814+0000","last_unstale":"2024-01-31T05:00:51.677784+0000","last_undegraded":"2024-01-31T05:00:51.677784+0000","last_fullsized":"2024-01-31T05:00:51.677784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:40.921562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.678543+0000","last_change":"2024-01-31T05:00:50.652235+0000","last_active":"2024-01-31T05:00:51.678543+0000","last_peered":"2024-01-31T05:00:51.678543+0000","last_clean":"2024-01-31T05:00:51.678543+0000","last_became_active":"2024-01-31T05:00:50.650199+0000","last_became_peered":"2024-01-31T05:00:50.650199+0000","last_unstale":"2024-01-31T05:00:51.678543+0000","last_undegraded":"2024-01-31T05:00:51.678543+0000","last_fullsized":"2024-01-31T05:00:51.678543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:15.584111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679821+0000","last_change":"2024-01-31T05:00:45.580589+0000","last_active":"2024-01-31T05:00:51.679821+0000","last_peered":"2024-01-31T05:00:51.679821+0000","last_clean":"2024-01-31T05:00:51.679821+0000","last_became_active":"2024-01-31T05:00:45.579288+0000","last_became_peered":"2024-01-31T05:00:45.579288+0000","last_unstale":"2024-01-31T05:00:51.679821+0000","last_undegraded":"2024-01-31T05:00:51.679821+0000","last_fullsized":"2024-01-31T05:00:51.679821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912730+0000","last_change":"2024-01-31T05:00:46.579010+0000","last_active":"2024-01-31T05:00:51.912730+0000","last_peered":"2024-01-31T05:00:51.912730+0000","last_clean":"2024-01-31T05:00:51.912730+0000","last_became_active":"2024-01-31T05:00:46.578792+0000","last_became_peered":"2024-01-31T05:00:46.578792+0000","last_unstale":"2024-01-31T05:00:51.912730+0000","last_undegraded":"2024-01-31T05:00:51.912730+0000","last_fullsized":"2024-01-31T05:00:51.912730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916113+0000","last_change":"2024-01-31T05:00:50.659322+0000","last_active":"2024-01-31T05:00:51.916113+0000","last_peered":"2024-01-31T05:00:51.916113+0000","last_clean":"2024-01-31T05:00:51.916113+0000","last_became_active":"2024-01-31T05:00:50.658743+0000","last_became_peered":"2024-01-31T05:00:50.658743+0000","last_unstale":"2024-01-31T05:00:51.916113+0000","last_undegraded":"2024-01-31T05:00:51.916113+0000","last_fullsized":"2024-01-31T05:00:51.916113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:20.923767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913064+0000","last_change":"2024-01-31T05:00:46.579418+0000","last_active":"2024-01-31T05:00:51.913064+0000","last_peered":"2024-01-31T05:00:51.913064+0000","last_clean":"2024-01-31T05:00:51.913064+0000","last_became_active":"2024-01-31T05:00:46.579076+0000","last_became_peered":"2024-01-31T05:00:46.579076+0000","last_unstale":"2024-01-31T05:00:51.913064+0000","last_undegraded":"2024-01-31T05:00:51.913064+0000","last_fullsized":"2024-01-31T05:00:51.913064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914823+0000","last_change":"2024-01-31T05:00:46.585461+0000","last_active":"2024-01-31T05:00:51.914823+0000","last_peered":"2024-01-31T05:00:51.914823+0000","last_clean":"2024-01-31T05:00:51.914823+0000","last_became_active":"2024-01-31T05:00:46.584818+0000","last_became_peered":"2024-01-31T05:00:46.584818+0000","last_unstale":"2024-01-31T05:00:51.914823+0000","last_undegraded":"2024-01-31T05:00:51.914823+0000","last_fullsized":"2024-01-31T05:00:51.914823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:52.011589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673116+0000","last_change":"2024-01-31T05:00:46.586264+0000","last_active":"2024-01-31T05:00:51.673116+0000","last_peered":"2024-01-31T05:00:51.673116+0000","last_clean":"2024-01-31T05:00:51.673116+0000","last_became_active":"2024-01-31T05:00:46.585995+0000","last_became_peered":"2024-01-31T05:00:46.585995+0000","last_unstale":"2024-01-31T05:00:51.673116+0000","last_undegraded":"2024-01-31T05:00:51.673116+0000","last_fullsized":"2024-01-31T05:00:51.673116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:11.523909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679572+0000","last_change":"2024-01-31T05:00:46.579199+0000","last_active":"2024-01-31T05:00:51.679572+0000","last_peered":"2024-01-31T05:00:51.679572+0000","last_clean":"2024-01-31T05:00:51.679572+0000","last_became_active":"2024-01-31T05:00:46.578257+0000","last_became_peered":"2024-01-31T05:00:46.578257+0000","last_unstale":"2024-01-31T05:00:51.679572+0000","last_undegraded":"2024-01-31T05:00:51.679572+0000","last_fullsized":"2024-01-31T05:00:51.679572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918235+0000","last_change":"2024-01-31T05:00:46.593320+0000","last_active":"2024-01-31T05:00:51.918235+0000","last_peered":"2024-01-31T05:00:51.918235+0000","last_clean":"2024-01-31T05:00:51.918235+0000","last_became_active":"2024-01-31T05:00:46.593053+0000","last_became_peered":"2024-01-31T05:00:46.593053+0000","last_unstale":"2024-01-31T05:00:51.918235+0000","last_undegraded":"2024-01-31T05:00:51.918235+0000","last_fullsized":"2024-01-31T05:00:51.918235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:21.326870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918207+0000","last_change":"2024-01-31T05:00:46.583943+0000","last_active":"2024-01-31T05:00:51.918207+0000","last_peered":"2024-01-31T05:00:51.918207+0000","last_clean":"2024-01-31T05:00:51.918207+0000","last_became_active":"2024-01-31T05:00:46.580249+0000","last_became_peered":"2024-01-31T05:00:46.580249+0000","last_unstale":"2024-01-31T05:00:51.918207+0000","last_undegraded":"2024-01-31T05:00:51.918207+0000","last_fullsized":"2024-01-31T05:00:51.918207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914742+0000","last_change":"2024-01-31T05:00:46.601110+0000","last_active":"2024-01-31T05:00:51.914742+0000","last_peered":"2024-01-31T05:00:51.914742+0000","last_clean":"2024-01-31T05:00:51.914742+0000","last_became_active":"2024-01-31T05:00:46.600844+0000","last_became_peered":"2024-01-31T05:00:46.600844+0000","last_unstale":"2024-01-31T05:00:51.914742+0000","last_undegraded":"2024-01-31T05:00:51.914742+0000","last_fullsized":"2024-01-31T05:00:51.914742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681192+0000","last_change":"2024-01-31T05:00:46.579792+0000","last_active":"2024-01-31T05:00:51.681192+0000","last_peered":"2024-01-31T05:00:51.681192+0000","last_clean":"2024-01-31T05:00:51.681192+0000","last_became_active":"2024-01-31T05:00:46.579357+0000","last_became_peered":"2024-01-31T05:00:46.579357+0000","last_unstale":"2024-01-31T05:00:51.681192+0000","last_undegraded":"2024-01-31T05:00:51.681192+0000","last_fullsized":"2024-01-31T05:00:51.681192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672904+0000","last_change":"2024-01-31T05:00:46.582507+0000","last_active":"2024-01-31T05:00:51.672904+0000","last_peered":"2024-01-31T05:00:51.672904+0000","last_clean":"2024-01-31T05:00:51.672904+0000","last_became_active":"2024-01-31T05:00:46.582243+0000","last_became_peered":"2024-01-31T05:00:46.582243+0000","last_unstale":"2024-01-31T05:00:51.672904+0000","last_undegraded":"2024-01-31T05:00:51.672904+0000","last_fullsized":"2024-01-31T05:00:51.672904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:07.689214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676563+0000","last_change":"2024-01-31T05:00:46.588900+0000","last_active":"2024-01-31T05:00:51.676563+0000","last_peered":"2024-01-31T05:00:51.676563+0000","last_clean":"2024-01-31T05:00:51.676563+0000","last_became_active":"2024-01-31T05:00:46.585994+0000","last_became_peered":"2024-01-31T05:00:46.585994+0000","last_unstale":"2024-01-31T05:00:51.676563+0000","last_undegraded":"2024-01-31T05:00:51.676563+0000","last_fullsized":"2024-01-31T05:00:51.676563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912637+0000","last_change":"2024-01-31T05:00:46.576223+0000","last_active":"2024-01-31T05:00:51.912637+0000","last_peered":"2024-01-31T05:00:51.912637+0000","last_clean":"2024-01-31T05:00:51.912637+0000","last_became_active":"2024-01-31T05:00:46.575962+0000","last_became_peered":"2024-01-31T05:00:46.575962+0000","last_unstale":"2024-01-31T05:00:51.912637+0000","last_undegraded":"2024-01-31T05:00:51.912637+0000","last_fullsized":"2024-01-31T05:00:51.912637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677638+0000","last_change":"2024-01-31T05:00:46.576001+0000","last_active":"2024-01-31T05:00:51.677638+0000","last_peered":"2024-01-31T05:00:51.677638+0000","last_clean":"2024-01-31T05:00:51.677638+0000","last_became_active":"2024-01-31T05:00:46.574721+0000","last_became_peered":"2024-01-31T05:00:46.574721+0000","last_unstale":"2024-01-31T05:00:51.677638+0000","last_undegraded":"2024-01-31T05:00:51.677638+0000","last_fullsized":"2024-01-31T05:00:51.677638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:31:53.337974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915731+0000","last_change":"2024-01-31T05:00:46.589503+0000","last_active":"2024-01-31T05:00:51.915731+0000","last_peered":"2024-01-31T05:00:51.915731+0000","last_clean":"2024-01-31T05:00:51.915731+0000","last_became_active":"2024-01-31T05:00:46.581750+0000","last_became_peered":"2024-01-31T05:00:46.581750+0000","last_unstale":"2024-01-31T05:00:51.915731+0000","last_undegraded":"2024-01-31T05:00:51.915731+0000","last_fullsized":"2024-01-31T05:00:51.915731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674812+0000","last_change":"2024-01-31T05:00:46.582051+0000","last_active":"2024-01-31T05:00:51.674812+0000","last_peered":"2024-01-31T05:00:51.674812+0000","last_clean":"2024-01-31T05:00:51.674812+0000","last_became_active":"2024-01-31T05:00:46.578134+0000","last_became_peered":"2024-01-31T05:00:46.578134+0000","last_unstale":"2024-01-31T05:00:51.674812+0000","last_undegraded":"2024-01-31T05:00:51.674812+0000","last_fullsized":"2024-01-31T05:00:51.674812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:31.255109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917009+0000","last_change":"2024-01-31T05:00:46.587070+0000","last_active":"2024-01-31T05:00:51.917009+0000","last_peered":"2024-01-31T05:00:51.917009+0000","last_clean":"2024-01-31T05:00:51.917009+0000","last_became_active":"2024-01-31T05:00:46.586707+0000","last_became_peered":"2024-01-31T05:00:46.586707+0000","last_unstale":"2024-01-31T05:00:51.917009+0000","last_undegraded":"2024-01-31T05:00:51.917009+0000","last_fullsized":"2024-01-31T05:00:51.917009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674901+0000","last_change":"2024-01-31T05:00:46.587042+0000","last_active":"2024-01-31T05:00:51.674901+0000","last_peered":"2024-01-31T05:00:51.674901+0000","last_clean":"2024-01-31T05:00:51.674901+0000","last_became_active":"2024-01-31T05:00:46.586518+0000","last_became_peered":"2024-01-31T05:00:46.586518+0000","last_unstale":"2024-01-31T05:00:51.674901+0000","last_undegraded":"2024-01-31T05:00:51.674901+0000","last_fullsized":"2024-01-31T05:00:51.674901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:54:17.597851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672833+0000","last_change":"2024-01-31T05:00:46.593111+0000","last_active":"2024-01-31T05:00:51.672833+0000","last_peered":"2024-01-31T05:00:51.672833+0000","last_clean":"2024-01-31T05:00:51.672833+0000","last_became_active":"2024-01-31T05:00:46.592828+0000","last_became_peered":"2024-01-31T05:00:46.592828+0000","last_unstale":"2024-01-31T05:00:51.672833+0000","last_undegraded":"2024-01-31T05:00:51.672833+0000","last_fullsized":"2024-01-31T05:00:51.672833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:59.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676004+0000","last_change":"2024-01-31T05:00:46.589807+0000","last_active":"2024-01-31T05:00:51.676004+0000","last_peered":"2024-01-31T05:00:51.676004+0000","last_clean":"2024-01-31T05:00:51.676004+0000","last_became_active":"2024-01-31T05:00:46.589518+0000","last_became_peered":"2024-01-31T05:00:46.589518+0000","last_unstale":"2024-01-31T05:00:51.676004+0000","last_undegraded":"2024-01-31T05:00:51.676004+0000","last_fullsized":"2024-01-31T05:00:51.676004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:06:03.428645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916672+0000","last_change":"2024-01-31T05:00:46.583014+0000","last_active":"2024-01-31T05:00:51.916672+0000","last_peered":"2024-01-31T05:00:51.916672+0000","last_clean":"2024-01-31T05:00:51.916672+0000","last_became_active":"2024-01-31T05:00:46.582607+0000","last_became_peered":"2024-01-31T05:00:46.582607+0000","last_unstale":"2024-01-31T05:00:51.916672+0000","last_undegraded":"2024-01-31T05:00:51.916672+0000","last_fullsized":"2024-01-31T05:00:51.916672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:21:09.066019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917869+0000","last_change":"2024-01-31T05:00:46.594160+0000","last_active":"2024-01-31T05:00:51.917869+0000","last_peered":"2024-01-31T05:00:51.917869+0000","last_clean":"2024-01-31T05:00:51.917869+0000","last_became_active":"2024-01-31T05:00:46.593838+0000","last_became_peered":"2024-01-31T05:00:46.593838+0000","last_unstale":"2024-01-31T05:00:51.917869+0000","last_undegraded":"2024-01-31T05:00:51.917869+0000","last_fullsized":"2024-01-31T05:00:51.917869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:58:27.273440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915637+0000","last_change":"2024-01-31T05:00:46.590079+0000","last_active":"2024-01-31T05:00:51.915637+0000","last_peered":"2024-01-31T05:00:51.915637+0000","last_clean":"2024-01-31T05:00:51.915637+0000","last_became_active":"2024-01-31T05:00:46.589828+0000","last_became_peered":"2024-01-31T05:00:46.589828+0000","last_unstale":"2024-01-31T05:00:51.915637+0000","last_undegraded":"2024-01-31T05:00:51.915637+0000","last_fullsized":"2024-01-31T05:00:51.915637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915410+0000","last_change":"2024-01-31T05:00:46.601762+0000","last_active":"2024-01-31T05:00:51.915410+0000","last_peered":"2024-01-31T05:00:51.915410+0000","last_clean":"2024-01-31T05:00:51.915410+0000","last_became_active":"2024-01-31T05:00:46.600161+0000","last_became_peered":"2024-01-31T05:00:46.600161+0000","last_unstale":"2024-01-31T05:00:51.915410+0000","last_undegraded":"2024-01-31T05:00:51.915410+0000","last_fullsized":"2024-01-31T05:00:51.915410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915285+0000","last_change":"2024-01-31T05:00:46.588560+0000","last_active":"2024-01-31T05:00:51.915285+0000","last_peered":"2024-01-31T05:00:51.915285+0000","last_clean":"2024-01-31T05:00:51.915285+0000","last_became_active":"2024-01-31T05:00:46.588291+0000","last_became_peered":"2024-01-31T05:00:46.588291+0000","last_unstale":"2024-01-31T05:00:51.915285+0000","last_undegraded":"2024-01-31T05:00:51.915285+0000","last_fullsized":"2024-01-31T05:00:51.915285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914247+0000","last_change":"2024-01-31T05:00:46.570493+0000","last_active":"2024-01-31T05:00:51.914247+0000","last_peered":"2024-01-31T05:00:51.914247+0000","last_clean":"2024-01-31T05:00:51.914247+0000","last_became_active":"2024-01-31T05:00:46.570196+0000","last_became_peered":"2024-01-31T05:00:46.570196+0000","last_unstale":"2024-01-31T05:00:51.914247+0000","last_undegraded":"2024-01-31T05:00:51.914247+0000","last_fullsized":"2024-01-31T05:00:51.914247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674510+0000","last_change":"2024-01-31T05:00:46.580076+0000","last_active":"2024-01-31T05:00:51.674510+0000","last_peered":"2024-01-31T05:00:51.674510+0000","last_clean":"2024-01-31T05:00:51.674510+0000","last_became_active":"2024-01-31T05:00:46.578312+0000","last_became_peered":"2024-01-31T05:00:46.578312+0000","last_unstale":"2024-01-31T05:00:51.674510+0000","last_undegraded":"2024-01-31T05:00:51.674510+0000","last_fullsized":"2024-01-31T05:00:51.674510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914636+0000","last_change":"2024-01-31T05:00:46.577353+0000","last_active":"2024-01-31T05:00:51.914636+0000","last_peered":"2024-01-31T05:00:51.914636+0000","last_clean":"2024-01-31T05:00:51.914636+0000","last_became_active":"2024-01-31T05:00:46.577019+0000","last_became_peered":"2024-01-31T05:00:46.577019+0000","last_unstale":"2024-01-31T05:00:51.914636+0000","last_undegraded":"2024-01-31T05:00:51.914636+0000","last_fullsized":"2024-01-31T05:00:51.914636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:09:13.929721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912937+0000","last_change":"2024-01-31T05:00:46.575406+0000","last_active":"2024-01-31T05:00:51.912937+0000","last_peered":"2024-01-31T05:00:51.912937+0000","last_clean":"2024-01-31T05:00:51.912937+0000","last_became_active":"2024-01-31T05:00:46.575056+0000","last_became_peered":"2024-01-31T05:00:46.575056+0000","last_unstale":"2024-01-31T05:00:51.912937+0000","last_undegraded":"2024-01-31T05:00:51.912937+0000","last_fullsized":"2024-01-31T05:00:51.912937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915174+0000","last_change":"2024-01-31T05:00:46.582321+0000","last_active":"2024-01-31T05:00:51.915174+0000","last_peered":"2024-01-31T05:00:51.915174+0000","last_clean":"2024-01-31T05:00:51.915174+0000","last_became_active":"2024-01-31T05:00:46.581547+0000","last_became_peered":"2024-01-31T05:00:46.581547+0000","last_unstale":"2024-01-31T05:00:51.915174+0000","last_undegraded":"2024-01-31T05:00:51.915174+0000","last_fullsized":"2024-01-31T05:00:51.915174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914144+0000","last_change":"2024-01-31T05:00:46.575296+0000","last_active":"2024-01-31T05:00:51.914144+0000","last_peered":"2024-01-31T05:00:51.914144+0000","last_clean":"2024-01-31T05:00:51.914144+0000","last_became_active":"2024-01-31T05:00:46.574983+0000","last_became_peered":"2024-01-31T05:00:46.574983+0000","last_unstale":"2024-01-31T05:00:51.914144+0000","last_undegraded":"2024-01-31T05:00:51.914144+0000","last_fullsized":"2024-01-31T05:00:51.914144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680670+0000","last_change":"2024-01-31T05:00:46.574733+0000","last_active":"2024-01-31T05:00:51.680670+0000","last_peered":"2024-01-31T05:00:51.680670+0000","last_clean":"2024-01-31T05:00:51.680670+0000","last_became_active":"2024-01-31T05:00:46.574335+0000","last_became_peered":"2024-01-31T05:00:46.574335+0000","last_unstale":"2024-01-31T05:00:51.680670+0000","last_undegraded":"2024-01-31T05:00:51.680670+0000","last_fullsized":"2024-01-31T05:00:51.680670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680984+0000","last_change":"2024-01-31T05:00:46.578081+0000","last_active":"2024-01-31T05:00:51.680984+0000","last_peered":"2024-01-31T05:00:51.680984+0000","last_clean":"2024-01-31T05:00:51.680984+0000","last_became_active":"2024-01-31T05:00:46.577576+0000","last_became_peered":"2024-01-31T05:00:46.577576+0000","last_unstale":"2024-01-31T05:00:51.680984+0000","last_undegraded":"2024-01-31T05:00:51.680984+0000","last_fullsized":"2024-01-31T05:00:51.680984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915608+0000","last_change":"2024-01-31T05:00:46.592072+0000","last_active":"2024-01-31T05:00:51.915608+0000","last_peered":"2024-01-31T05:00:51.915608+0000","last_clean":"2024-01-31T05:00:51.915608+0000","last_became_active":"2024-01-31T05:00:46.591682+0000","last_became_peered":"2024-01-31T05:00:46.591682+0000","last_unstale":"2024-01-31T05:00:51.915608+0000","last_undegraded":"2024-01-31T05:00:51.915608+0000","last_fullsized":"2024-01-31T05:00:51.915608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:13:21.174300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":106194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5189999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3879999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.601}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.665}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":106652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.631}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.323}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":696508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":106126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.603}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.71}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:01:10.764 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-31T05:01:10.764 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-31T05:01:10.765 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-31T05:01:10.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-31T05:01:10.765 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-31T05:01:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.015+0000 7f5a79e00640 1 Processor -- start 2024-01-31T05:01:11.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.015+0000 7f5a79e00640 1 -- start start 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.016+0000 7f5a79e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a741616d0 0x7f5a740cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a79e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a740abcf0 con 0x7f5a740ce430 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a79e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7406ca40 con 0x7f5a7415d520 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a79e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7415d900 con 0x7f5a741616d0 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a73400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5a740ce430 0x7f5a7406ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41716/0 (socket says 172.21.15.97:41716) 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a73400640 1 -- 172.21.15.97:0/3908434321 learned_addr learned my addr 172.21.15.97:0/3908434321 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533435622 0 0) 0x7f5a740abcf0 con 0x7f5a740ce430 2024-01-31T05:01:11.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a72000640 1 --2- 172.21.15.97:0/3908434321 >> v2:172.21.15.188:3300/0 conn(0x7f5a741616d0 0x7f5a740cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.017+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a54003520 con 0x7f5a740ce430 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2617486616 0 0) 0x7f5a54003520 con 0x7f5a740ce430 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 >> v2:172.21.15.188:3300/0 conn(0x7f5a741616d0 msgr2=0x7f5a740cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 --2- 172.21.15.97:0/3908434321 >> v2:172.21.15.188:3300/0 conn(0x7f5a741616d0 0x7f5a740cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 >> v1:172.21.15.188:6789/0 conn(0x7f5a7415d520 legacy=0x7f5a74160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a74161e20 con 0x7f5a740ce430 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f5a6c0019f0 con 0x7f5a740ce430 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 >> v1:172.21.15.97:6789/0 conn(0x7f5a740ce430 legacy=0x7f5a7406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a54003970 0x7f5a54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a540047c0 0x7f5a54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7415d900 con 0x7f5a54005180 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5400c020 con 0x7f5a540047c0 2024-01-31T05:01:11.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5400c1e0 con 0x7f5a54003970 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a73400640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a72a00640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a54003970 0x7f5a54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.018+0000 7f5a72000640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a540047c0 0x7f5a54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.019+0000 7f5a73400640 1 -- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a54003970 msgr2=0x7f5a54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.019+0000 7f5a73400640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a54003970 0x7f5a54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.019+0000 7f5a73400640 1 -- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a540047c0 msgr2=0x7f5a54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.019+0000 7f5a73400640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a540047c0 0x7f5a54004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.019+0000 7f5a73400640 1 -- 172.21.15.97:0/3908434321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54003520 con 0x7f5a54005180 2024-01-31T05:01:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a73400640 1 -- 172.21.15.97:0/3908434321 reap_dead start 2024-01-31T05:01:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a73400640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a5400b980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00a830 tx=0x7f5a540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3580c72c5d26128b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0072d0 con 0x7f5a54005180 2024-01-31T05:01:11.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c02f070 con 0x7f5a54005180 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a71600640 1 -- 172.21.15.97:0/3908434321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0076b0 con 0x7f5a54005180 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a79e00640 1 -- 172.21.15.97:0/3908434321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 msgr2=0x7f5a5400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.020+0000 7f5a79e00640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a5400b980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00a830 tx=0x7f5a540057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.021+0000 7f5a79e00640 1 -- 172.21.15.97:0/3908434321 shutdown_connections 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.021+0000 7f5a79e00640 1 --2- 172.21.15.97:0/3908434321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a5400b980 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.021+0000 7f5a79e00640 1 -- 172.21.15.97:0/3908434321 >> 172.21.15.97:0/3908434321 conn(0x7f5a740b0a10 msgr2=0x7f5a740b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.021+0000 7f5a79e00640 1 -- 172.21.15.97:0/3908434321 shutdown_connections 2024-01-31T05:01:11.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.021+0000 7f5a79e00640 1 -- 172.21.15.97:0/3908434321 wait complete. 2024-01-31T05:01:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.022+0000 7f5a79e00640 1 Processor -- start 2024-01-31T05:01:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 -- start start 2024-01-31T05:01:11.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a741f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a74071f30 0x7f5a7415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a7414b4b0 0x7f5a7414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7414be60 con 0x7f5a54005180 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741f3ff0 con 0x7f5a7414b4b0 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.023+0000 7f5a79e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741f4690 con 0x7f5a74071f30 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a741f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a741f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43062/0 (socket says 172.21.15.97:43062) 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 -- 172.21.15.97:0/2808147912 learned_addr learned my addr 172.21.15.97:0/2808147912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 -- 172.21.15.97:0/2808147912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a74071f30 msgr2=0x7f5a7415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a74071f30 0x7f5a7415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 -- 172.21.15.97:0/2808147912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a7414b4b0 msgr2=0x7f5a7414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a7414b4b0 0x7f5a7414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 -- 172.21.15.97:0/2808147912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a741f3a30 con 0x7f5a54005180 2024-01-31T05:01:11.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.024+0000 7f5a72a00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a741f30e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e4d0 tx=0x7f5a6400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.025+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640052e0 con 0x7f5a54005180 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.025+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64005440 con 0x7f5a54005180 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.025+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64003810 con 0x7f5a54005180 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.025+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a741f3d10 con 0x7f5a54005180 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.025+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a741f3620 con 0x7f5a54005180 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.026+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a741f37b0 con 0x7f5a54005180 2024-01-31T05:01:11.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.027+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5a64003970 con 0x7f5a54005180 2024-01-31T05:01:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.027+0000 7f5a6b400640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a4c06aed0 0x7f5a4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.028+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5a64089760 con 0x7f5a54005180 2024-01-31T05:01:11.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.028+0000 7f5a72000640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a4c06aed0 0x7f5a4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.028+0000 7f5a72000640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a4c06aed0 0x7f5a4c06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a74072830 tx=0x7f5a5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.032+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5a640542a0 con 0x7f5a54005180 2024-01-31T05:01:11.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.276+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5a74068060 con 0x7f5a54005180 2024-01-31T05:01:11.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.285+0000 7f5a6b400640 1 -- 172.21.15.97:0/2808147912 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170776 (secure 0 0 0) 0x7f5a640578c0 con 0x7f5a54005180 2024-01-31T05:01:11.287 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a4c06aed0 msgr2=0x7f5a4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a4c06aed0 0x7f5a4c06d390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a74072830 tx=0x7f5a5c008040 comp rx=0 tx=0).stop 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 msgr2=0x7f5a741f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a741f30e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400e4d0 tx=0x7f5a6400e990 comp rx=0 tx=0).stop 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 shutdown_connections 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a7414b4b0 0x7f5a7414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a74071f30 0x7f5a7415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5a4c06aed0 0x7f5a4c06d390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 --2- 172.21.15.97:0/2808147912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a54005180 0x7f5a741f30e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 >> 172.21.15.97:0/2808147912 conn(0x7f5a740b0a10 msgr2=0x7f5a740b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 shutdown_connections 2024-01-31T05:01:11.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.289+0000 7f5a79e00640 1 -- 172.21.15.97:0/2808147912 wait complete. 2024-01-31T05:01:11.308 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":631419576},{"type":"v1","addr":"172.21.15.97:6833","nonce":631419576}]},"active_addr":"172.21.15.97:6833/631419576","active_change":"2024-01-31T05:00:31.532819+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3813503915}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3483122121}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3015342106}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1335819249}]}]} 2024-01-31T05:01:11.311 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-31T05:01:11.311 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-31T05:01:11.311 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:11.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.508+0000 7fcbe3600640 1 Processor -- start 2024-01-31T05:01:11.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.508+0000 7fcbe3600640 1 -- start start 2024-01-31T05:01:11.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe3600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0c9100 0x7fcbdc0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe3600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbdc0cd4b0 0x7fcbdc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe3600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbdc0711a0 con 0x7fcbdc15d370 2024-01-31T05:01:11.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe3600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc071480 con 0x7fcbdc0cd4b0 2024-01-31T05:01:11.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe3600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc071760 con 0x7fcbdc0c9100 2024-01-31T05:01:11.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbdbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0c9100 0x7fcbdc0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe1600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbdc0cd4b0 0x7fcbdc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe1600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbdc0cd4b0 0x7fcbdc0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:58992/0 (socket says 172.21.15.97:58992) 2024-01-31T05:01:11.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe1600640 1 -- 172.21.15.97:0/4183055144 learned_addr learned my addr 172.21.15.97:0/4183055144 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.510+0000 7fcbe0c00640 1 --1- 172.21.15.97:0/4183055144 >> v1:172.21.15.188:6789/0 conn(0x7fcbdc15d370 0x7fcbdc0c89f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:54884/0 (socket says 172.21.15.97:54884) 2024-01-31T05:01:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.511+0000 7fcbe1600640 1 -- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0c9100 msgr2=0x7fcbdc0cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.511+0000 7fcbe1600640 1 --2- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0c9100 0x7fcbdc0cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.511+0000 7fcbe1600640 1 -- 172.21.15.97:0/4183055144 >> v1:172.21.15.188:6789/0 conn(0x7fcbdc15d370 legacy=0x7fcbdc0c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.511+0000 7fcbe1600640 1 -- 172.21.15.97:0/4183055144 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc071a40 con 0x7fcbdc0cd4b0 2024-01-31T05:01:11.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.512+0000 7fcbe1600640 1 --2- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0cd4b0 0x7fcbdc0cd920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009e50 tx=0x7fcbd402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9b69cb361027c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.513+0000 7fcbdb400640 1 -- 172.21.15.97:0/4183055144 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd402cb00 con 0x7fcbdc0cd4b0 2024-01-31T05:01:11.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.513+0000 7fcbdb400640 1 -- 172.21.15.97:0/4183055144 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd402cc60 con 0x7fcbdc0cd4b0 2024-01-31T05:01:11.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.513+0000 7fcbdb400640 1 -- 172.21.15.97:0/4183055144 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40343f0 con 0x7fcbdc0cd4b0 2024-01-31T05:01:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.513+0000 7fcbe3600640 1 -- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0cd4b0 msgr2=0x7fcbdc0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.513+0000 7fcbe3600640 1 --2- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0cd4b0 0x7fcbdc0cd920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009e50 tx=0x7fcbd402bf60 comp rx=0 tx=0).stop 2024-01-31T05:01:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.514+0000 7fcbe3600640 1 -- 172.21.15.97:0/4183055144 shutdown_connections 2024-01-31T05:01:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.514+0000 7fcbe3600640 1 --2- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0cd4b0 0x7fcbdc0cd920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.514+0000 7fcbe3600640 1 --2- 172.21.15.97:0/4183055144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0c9100 0x7fcbdc0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.514+0000 7fcbe3600640 1 -- 172.21.15.97:0/4183055144 >> 172.21.15.97:0/4183055144 conn(0x7fcbdc0cf6e0 msgr2=0x7fcbdc0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:11.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.514+0000 7fcbe3600640 1 -- 172.21.15.97:0/4183055144 shutdown_connections 2024-01-31T05:01:11.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.514+0000 7fcbe3600640 1 -- 172.21.15.97:0/4183055144 wait complete. 2024-01-31T05:01:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.515+0000 7fcbe3600640 1 Processor -- start 2024-01-31T05:01:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 -- start start 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0c9100 0x7fcbdc0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0cd4b0 0x7fcbdc15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 0x7fcbdc06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc1f2f20 con 0x7fcbdc15d370 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc1f69f0 con 0x7fcbdc0c9100 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.516+0000 7fcbe3600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc1f6cd0 con 0x7fcbdc0cd4b0 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 0x7fcbdc06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 0x7fcbdc06e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43066/0 (socket says 172.21.15.97:43066) 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 -- 172.21.15.97:0/1234577933 learned_addr learned my addr 172.21.15.97:0/1234577933 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe0c00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0c9100 0x7fcbdc0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbdbe00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0cd4b0 0x7fcbdc15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 -- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0cd4b0 msgr2=0x7fcbdc15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0cd4b0 0x7fcbdc15da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 -- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0c9100 msgr2=0x7fcbdc0b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0c9100 0x7fcbdc0b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 -- 172.21.15.97:0/1234577933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd4009b30 con 0x7fcbdc15d370 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe0c00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0c9100 0x7fcbdc0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbdbe00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0cd4b0 0x7fcbdc15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.517+0000 7fcbe1600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 0x7fcbdc06e0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4036040 tx=0x7fcbd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.518+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4009510 con 0x7fcbdc15d370 2024-01-31T05:01:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.518+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd403e450 con 0x7fcbdc15d370 2024-01-31T05:01:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.518+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc1f6fb0 con 0x7fcbdc15d370 2024-01-31T05:01:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.518+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd403e820 con 0x7fcbdc15d370 2024-01-31T05:01:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.518+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc0b27f0 con 0x7fcbdc15d370 2024-01-31T05:01:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.519+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc1f7470 con 0x7fcbdc15d370 2024-01-31T05:01:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.520+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fcbd403e980 con 0x7fcbdc15d370 2024-01-31T05:01:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.521+0000 7fcbd8c00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcbb406aed0 0x7fcbb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.521+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcbd40ac720 con 0x7fcbdc15d370 2024-01-31T05:01:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.521+0000 7fcbe0c00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcbb406aed0 0x7fcbb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.522+0000 7fcbe0c00640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcbb406aed0 0x7fcbb406d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc15c0b0 tx=0x7fcbcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.530+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fcbd40771e0 con 0x7fcbdc15d370 2024-01-31T05:01:11.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.728+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbdc067dc0 con 0x7fcbdc15d370 2024-01-31T05:01:11.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.729+0000 7fcbd8c00640 1 -- 172.21.15.97:0/1234577933 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fcbd407a800 con 0x7fcbdc15d370 2024-01-31T05:01:11.729 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:11.729 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcbb406aed0 msgr2=0x7fcbb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcbb406aed0 0x7fcbb406d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc15c0b0 tx=0x7fcbcc009040 comp rx=0 tx=0).stop 2024-01-31T05:01:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 msgr2=0x7fcbdc06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 0x7fcbdc06e0c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4036040 tx=0x7fcbd4004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 shutdown_connections 2024-01-31T05:01:11.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcbb406aed0 0x7fcbb406d390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbdc15d370 0x7fcbdc06e0c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbdc0cd4b0 0x7fcbdc15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 --2- 172.21.15.97:0/1234577933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbdc0c9100 0x7fcbdc0b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 >> 172.21.15.97:0/1234577933 conn(0x7fcbdc0cf6e0 msgr2=0x7fcbdc0c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 shutdown_connections 2024-01-31T05:01:11.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.732+0000 7fcbe3600640 1 -- 172.21.15.97:0/1234577933 wait complete. 2024-01-31T05:01:11.750 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-31T05:01:11.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.934+0000 7fba62c00640 1 Processor -- start 2024-01-31T05:01:11.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.934+0000 7fba62c00640 1 -- start start 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba62c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba5c15d590 0x7fba5c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba62c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c0cfae0 0x7fba5c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba62c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5c0d0460 con 0x7fba5c161080 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba62c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5c0cded0 con 0x7fba5c15d590 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba62c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c15d970 con 0x7fba5c0cfae0 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba5be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba5c15d590 0x7fba5c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba5be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba5c15d590 0x7fba5c15e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43072/0 (socket says 172.21.15.97:43072) 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba5be00640 1 -- 172.21.15.97:0/2863552947 learned_addr learned my addr 172.21.15.97:0/2863552947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.935+0000 7fba53200640 1 --2- 172.21.15.97:0/2863552947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c0cfae0 0x7fba5c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.936+0000 7fba5be00640 1 -- 172.21.15.97:0/2863552947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c0cfae0 msgr2=0x7fba5c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.936+0000 7fba5be00640 1 --2- 172.21.15.97:0/2863552947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c0cfae0 0x7fba5c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.936+0000 7fba5be00640 1 -- 172.21.15.97:0/2863552947 >> v1:172.21.15.188:6789/0 conn(0x7fba5c161080 legacy=0x7fba5c161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:11.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.936+0000 7fba5be00640 1 -- 172.21.15.97:0/2863552947 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba5c0cf210 con 0x7fba5c15d590 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba5be00640 1 --2- 172.21.15.97:0/2863552947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c15d590 0x7fba5c15e540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009870 tx=0x7fba4c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f1c211b8336385 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba5b400640 1 -- 172.21.15.97:0/2863552947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c030840 con 0x7fba5c15d590 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba5b400640 1 -- 172.21.15.97:0/2863552947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c0309a0 con 0x7fba5c15d590 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba62c00640 1 -- 172.21.15.97:0/2863552947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c15d590 msgr2=0x7fba5c15e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba62c00640 1 --2- 172.21.15.97:0/2863552947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c15d590 0x7fba5c15e540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009870 tx=0x7fba4c02b980 comp rx=0 tx=0).stop 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba62c00640 1 -- 172.21.15.97:0/2863552947 shutdown_connections 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba62c00640 1 --2- 172.21.15.97:0/2863552947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c0cfae0 0x7fba5c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba62c00640 1 --2- 172.21.15.97:0/2863552947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c15d590 0x7fba5c15e540 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.937+0000 7fba62c00640 1 -- 172.21.15.97:0/2863552947 >> 172.21.15.97:0/2863552947 conn(0x7fba5c09c560 msgr2=0x7fba5c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.938+0000 7fba62c00640 1 -- 172.21.15.97:0/2863552947 shutdown_connections 2024-01-31T05:01:11.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.938+0000 7fba62c00640 1 -- 172.21.15.97:0/2863552947 wait complete. 2024-01-31T05:01:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.939+0000 7fba62c00640 1 Processor -- start 2024-01-31T05:01:11.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.939+0000 7fba62c00640 1 -- start start 2024-01-31T05:01:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.940+0000 7fba62c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5c0cfae0 0x7fba5c0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.940+0000 7fba62c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c15d590 0x7fba5c14d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.940+0000 7fba62c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 0x7fba5c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.940+0000 7fba62c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0cb3f0 con 0x7fba5c161080 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.940+0000 7fba62c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0ca7e0 con 0x7fba5c0cfae0 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.940+0000 7fba62c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5c0caac0 con 0x7fba5c15d590 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 0x7fba5c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 0x7fba5c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43076/0 (socket says 172.21.15.97:43076) 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 -- 172.21.15.97:0/3927346792 learned_addr learned my addr 172.21.15.97:0/3927346792 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba5be00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5c0cfae0 0x7fba5c0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba53200640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c15d590 0x7fba5c14d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 -- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c15d590 msgr2=0x7fba5c14d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c15d590 0x7fba5c14d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 -- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5c0cfae0 msgr2=0x7fba5c0c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5c0cfae0 0x7fba5c0c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:11.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 -- 172.21.15.97:0/3927346792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0094d0 con 0x7fba5c161080 2024-01-31T05:01:11.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.941+0000 7fba60c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 0x7fba5c0c7b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba5400b970 tx=0x7fba5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.942+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5400cb40 con 0x7fba5c161080 2024-01-31T05:01:11.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.942+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5400cca0 con 0x7fba5c161080 2024-01-31T05:01:11.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.942+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54014680 con 0x7fba5c161080 2024-01-31T05:01:11.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.942+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5c0cada0 con 0x7fba5c161080 2024-01-31T05:01:11.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.942+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5c14db00 con 0x7fba5c161080 2024-01-31T05:01:11.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.943+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5c0cb560 con 0x7fba5c161080 2024-01-31T05:01:11.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.944+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fba5401d3b0 con 0x7fba5c161080 2024-01-31T05:01:11.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.945+0000 7fba58c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3406aed0 0x7fba3406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:11.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.945+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fba54089790 con 0x7fba5c161080 2024-01-31T05:01:11.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.945+0000 7fba5be00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3406aed0 0x7fba3406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:11.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.946+0000 7fba5be00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3406aed0 0x7fba3406d390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba4c036000 tx=0x7fba4c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:11.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:11.950+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fba54054360 con 0x7fba5c161080 2024-01-31T05:01:12.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.159+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba5c15c9d0 con 0x7fba5c161080 2024-01-31T05:01:12.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.161+0000 7fba58c00640 1 -- 172.21.15.97:0/3927346792 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fba54057980 con 0x7fba5c161080 2024-01-31T05:01:12.161 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:12.161 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.163+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3406aed0 msgr2=0x7fba3406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.163+0000 7fba62c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3406aed0 0x7fba3406d390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba4c036000 tx=0x7fba4c03b040 comp rx=0 tx=0).stop 2024-01-31T05:01:12.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.163+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 msgr2=0x7fba5c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.163+0000 7fba62c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 0x7fba5c0c7b90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fba5400b970 tx=0x7fba5400be30 comp rx=0 tx=0).stop 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 shutdown_connections 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3406aed0 0x7fba3406d390 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba5c161080 0x7fba5c0c7b90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba5c15d590 0x7fba5c14d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 --2- 172.21.15.97:0/3927346792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5c0cfae0 0x7fba5c0c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 >> 172.21.15.97:0/3927346792 conn(0x7fba5c09c560 msgr2=0x7fba5c1606d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 shutdown_connections 2024-01-31T05:01:12.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.164+0000 7fba62c00640 1 -- 172.21.15.97:0/3927346792 wait complete. 2024-01-31T05:01:12.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-31T05:01:12.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-31T05:01:12.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-31T05:01:12.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-31T05:01:12.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-31T05:01:12.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-31T05:01:12.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-31T05:01:12.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-31T05:01:12.217 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:01:12.275 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:01:12.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.504+0000 7fed52600640 1 Processor -- start 2024-01-31T05:01:12.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.504+0000 7fed52600640 1 -- start start 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fc229600640 1 Processor -- start 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.506+0000 7fed52600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed4c15d450 0x7fed4c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.506+0000 7fed52600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.506+0000 7fed52600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed4c0d1680 0x7fed4c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed52600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c0b3060 con 0x7fed4c0d1680 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed52600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c0d2190 con 0x7fed4c15d450 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed52600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0b2a30 con 0x7fed4c0d39f0 2024-01-31T05:01:12.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed51c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed4c0d1680 0x7fed4c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed51200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed4c15d450 0x7fed4c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed51200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed4c15d450 0x7fed4c0cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59020/0 (socket says 172.21.15.97:59020) 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.507+0000 7fed51200640 1 -- 172.21.15.97:0/3026588934 learned_addr learned my addr 172.21.15.97:0/3026588934 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.508+0000 7fc229600640 1 -- start start 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.510+0000 7fc229600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2240b1500 0x7fc2241595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.510+0000 7fc229600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc224070420 0x7fc2240ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.510+0000 7fc229600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22406c9a0 con 0x7fc2240ce760 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.510+0000 7fc229600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22406cc80 con 0x7fc2240b1500 2024-01-31T05:01:12.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.510+0000 7fc229600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22406cf60 con 0x7fc224070420 2024-01-31T05:01:12.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.508+0000 7fed51200640 1 -- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 msgr2=0x7fed4c0cfe70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-31T05:01:12.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.508+0000 7fed51200640 1 --2- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c0cfe70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.508+0000 7fed51200640 1 -- 172.21.15.97:0/3026588934 >> v2:172.21.15.97:3300/0 conn(0x7fed4c0d1680 msgr2=0x7fed4c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.508+0000 7fed51200640 1 --2- 172.21.15.97:0/3026588934 >> v2:172.21.15.97:3300/0 conn(0x7fed4c0d1680 0x7fed4c0cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.508+0000 7fed51200640 1 -- 172.21.15.97:0/3026588934 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c0d1c30 con 0x7fed4c15d450 2024-01-31T05:01:12.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.511+0000 7fc222000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc224070420 0x7fc2240ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.511+0000 7fc222000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc224070420 0x7fc2240ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59730/0 (socket says 172.21.15.97:59730) 2024-01-31T05:01:12.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.511+0000 7fc222000640 1 -- 172.21.15.97:0/1205803794 learned_addr learned my addr 172.21.15.97:0/1205803794 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.511+0000 7fc222a00640 1 --2- 172.21.15.97:0/1205803794 >> v2:172.21.15.188:3300/0 conn(0x7fc2240b1500 0x7fc2241595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.511+0000 7fed51200640 1 --2- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c15d450 0x7fed4c0cf930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed4c08d920 tx=0x7fed3403ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628b401bbe1047c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fed4b400640 1 -- 172.21.15.97:0/3026588934 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3403e850 con 0x7fed4c15d450 2024-01-31T05:01:12.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fe9dc400640 1 Processor -- start 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fed4b400640 1 -- 172.21.15.97:0/3026588934 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3403e9b0 con 0x7fed4c15d450 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fed4b400640 1 -- 172.21.15.97:0/3026588934 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3403ed80 con 0x7fed4c15d450 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 -- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c15d450 msgr2=0x7fed4c0cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 --2- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c15d450 0x7fed4c0cf930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed4c08d920 tx=0x7fed3403ca90 comp rx=0 tx=0).stop 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 -- 172.21.15.97:0/3026588934 shutdown_connections 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 --2- 172.21.15.97:0/3026588934 >> v2:172.21.15.97:3300/0 conn(0x7fed4c0d1680 0x7fed4c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 --2- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 --2- 172.21.15.97:0/3026588934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c15d450 0x7fed4c0cf930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.513+0000 7fed52600640 1 -- 172.21.15.97:0/3026588934 >> 172.21.15.97:0/3026588934 conn(0x7fed4c09c400 msgr2=0x7fed4c0d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fc222a00640 1 -- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc224070420 msgr2=0x7fc2240ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fc222a00640 1 --2- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc224070420 0x7fc2240ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fc222a00640 1 -- 172.21.15.97:0/1205803794 >> v1:172.21.15.188:6789/0 conn(0x7fc2240ce760 legacy=0x7fc22406d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.512+0000 7fc222a00640 1 -- 172.21.15.97:0/1205803794 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22406c350 con 0x7fc2240b1500 2024-01-31T05:01:12.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.514+0000 7fed52600640 1 -- 172.21.15.97:0/3026588934 shutdown_connections 2024-01-31T05:01:12.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.514+0000 7fed52600640 1 -- 172.21.15.97:0/3026588934 wait complete. 2024-01-31T05:01:12.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.514+0000 7fe9dc400640 1 -- start start 2024-01-31T05:01:12.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.516+0000 7fe9dc400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 0x7fe9cc04fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.516+0000 7fe9dc400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9cc0dfcb0 0x7fe9cc005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.516+0000 7fe9dc400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9cc0e00e0 con 0x7fe9cc054690 2024-01-31T05:01:12.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.516+0000 7fe9dc400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc053310 con 0x7fe9cc0dfcb0 2024-01-31T05:01:12.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.516+0000 7fe9dc400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0535f0 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fed52600640 1 Processor -- start 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fed52600640 1 -- start start 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fe9da400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9cc0dfcb0 0x7fe9cc005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fe9da400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9cc0dfcb0 0x7fe9cc005ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43088/0 (socket says 172.21.15.97:43088) 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fe9d9a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 0x7fe9cc04fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fe9d9a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 0x7fe9cc04fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59738/0 (socket says 172.21.15.97:59738) 2024-01-31T05:01:12.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.517+0000 7fe9d9a00640 1 -- 172.21.15.97:0/3692640611 learned_addr learned my addr 172.21.15.97:0/3692640611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.519+0000 7fe9da400640 1 -- 172.21.15.97:0/3692640611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 msgr2=0x7fe9cc04fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.519+0000 7fe9da400640 1 --2- 172.21.15.97:0/3692640611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 0x7fe9cc04fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.519+0000 7fe9da400640 1 -- 172.21.15.97:0/3692640611 >> v1:172.21.15.188:6789/0 conn(0x7fe9cc054690 legacy=0x7fe9cc0df5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.519+0000 7fe9da400640 1 -- 172.21.15.97:0/3692640611 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0538d0 con 0x7fe9cc0dfcb0 2024-01-31T05:01:12.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.519+0000 7fe9d9a00640 1 --2- 172.21.15.97:0/3692640611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 0x7fe9cc04fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:12.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.520+0000 7fed52600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c0d1680 0x7fed4c150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.520+0000 7fed52600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.520+0000 7fed52600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed4c15d450 0x7fed4c0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed51200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c0d1680 0x7fed4c150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed52600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0ccf00 con 0x7fed4c15d450 2024-01-31T05:01:12.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed52600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c15a310 con 0x7fed4c0d1680 2024-01-31T05:01:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed52600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c15f700 con 0x7fed4c0d39f0 2024-01-31T05:01:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fc222a00640 1 --2- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2240b1500 0x7fc2241595b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0097a0 tx=0x7fc20c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee54cf35ca904e37 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed4be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed4be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c1511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59750/0 (socket says 172.21.15.97:59750) 2024-01-31T05:01:12.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.521+0000 7fed4be00640 1 -- 172.21.15.97:0/3331005807 learned_addr learned my addr 172.21.15.97:0/3331005807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.522+0000 7fc221600640 1 -- 172.21.15.97:0/1205803794 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c031830 con 0x7fc2240b1500 2024-01-31T05:01:12.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.522+0000 7fc221600640 1 -- 172.21.15.97:0/1205803794 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c031990 con 0x7fc2240b1500 2024-01-31T05:01:12.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.522+0000 7fc221600640 1 -- 172.21.15.97:0/1205803794 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c031d60 con 0x7fc2240b1500 2024-01-31T05:01:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.522+0000 7fed51c00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed4c15d450 0x7fed4c0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.522+0000 7fe9da400640 1 --2- 172.21.15.97:0/3692640611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0dfcb0 0x7fe9cc005ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097a0 tx=0x7fe9bc02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409e25999aff30c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fed4be00640 1 -- 172.21.15.97:0/3331005807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c0d1680 msgr2=0x7fed4c150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fed4be00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed4c0d1680 0x7fed4c150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fed4be00640 1 -- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed4c15d450 msgr2=0x7fed4c0ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fed4be00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed4c15d450 0x7fed4c0ca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fed4be00640 1 -- 172.21.15.97:0/3331005807 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed340094d0 con 0x7fed4c0d39f0 2024-01-31T05:01:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fe9d3e00640 1 -- 172.21.15.97:0/3692640611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc030710 con 0x7fe9cc0dfcb0 2024-01-31T05:01:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fe9d3e00640 1 -- 172.21.15.97:0/3692640611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc02e050 con 0x7fe9cc0dfcb0 2024-01-31T05:01:12.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.523+0000 7fe9d3e00640 1 -- 172.21.15.97:0/3692640611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc030710 con 0x7fe9cc0dfcb0 2024-01-31T05:01:12.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 -- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2240b1500 msgr2=0x7fc2241595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 --2- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2240b1500 0x7fc2241595b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0097a0 tx=0x7fc20c02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 -- 172.21.15.97:0/1205803794 shutdown_connections 2024-01-31T05:01:12.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 --2- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc224070420 0x7fc2240ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 --2- 172.21.15.97:0/1205803794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2240b1500 0x7fc2241595b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 -- 172.21.15.97:0/1205803794 >> 172.21.15.97:0/1205803794 conn(0x7fc2240b0a10 msgr2=0x7fc2240b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fc229600640 1 -- 172.21.15.97:0/1205803794 shutdown_connections 2024-01-31T05:01:12.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fed4be00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c1511a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00b450 tx=0x7fed3c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fe9dc400640 1 -- 172.21.15.97:0/3692640611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0dfcb0 msgr2=0x7fe9cc005ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fc229600640 1 -- 172.21.15.97:0/1205803794 wait complete. 2024-01-31T05:01:12.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.524+0000 7fe9dc400640 1 --2- 172.21.15.97:0/3692640611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0dfcb0 0x7fe9cc005ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0097a0 tx=0x7fe9bc02b710 comp rx=0 tx=0).stop 2024-01-31T05:01:12.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fe9dc400640 1 -- 172.21.15.97:0/3692640611 shutdown_connections 2024-01-31T05:01:12.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fe9dc400640 1 --2- 172.21.15.97:0/3692640611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0dfcb0 0x7fe9cc005ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fe9dc400640 1 --2- 172.21.15.97:0/3692640611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc0df1c0 0x7fe9cc04fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fe9dc400640 1 -- 172.21.15.97:0/3692640611 >> 172.21.15.97:0/3692640611 conn(0x7fe9cc009fe0 msgr2=0x7fe9cc00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fe9dc400640 1 -- 172.21.15.97:0/3692640611 shutdown_connections 2024-01-31T05:01:12.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c011890 con 0x7fed4c0d39f0 2024-01-31T05:01:12.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.525+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c0119f0 con 0x7fed4c0d39f0 2024-01-31T05:01:12.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.526+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c011890 con 0x7fed4c0d39f0 2024-01-31T05:01:12.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.526+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4c15f9e0 con 0x7fed4c0d39f0 2024-01-31T05:01:12.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.526+0000 7fe9dc400640 1 -- 172.21.15.97:0/3692640611 wait complete. 2024-01-31T05:01:12.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.527+0000 7fcac3200640 1 Processor -- start 2024-01-31T05:01:12.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.526+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4c15ffa0 con 0x7fed4c0d39f0 2024-01-31T05:01:12.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.527+0000 7fcac3200640 1 -- start start 2024-01-31T05:01:12.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fe9dc400640 1 Processor -- start 2024-01-31T05:01:12.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fe9dc400640 1 -- start start 2024-01-31T05:01:12.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.528+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed4c0cef40 con 0x7fed4c0d39f0 2024-01-31T05:01:12.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fcac3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcac415d630 0x7fcac4168040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fcac3200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac40d0a30 con 0x7fcac40d3df0 2024-01-31T05:01:12.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fcac3200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac40d07f0 con 0x7fcac40d18f0 2024-01-31T05:01:12.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fcac3200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac40b69c0 con 0x7fcac415d630 2024-01-31T05:01:12.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.529+0000 7fc229600640 1 Processor -- start 2024-01-31T05:01:12.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.530+0000 7fc229600640 1 -- start start 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.530+0000 7fcac2800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcac415d630 0x7fcac4168040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.530+0000 7fcac2800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcac415d630 0x7fcac4168040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43118/0 (socket says 172.21.15.97:43118) 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.530+0000 7fcac2800640 1 -- 172.21.15.97:0/3893246651 learned_addr learned my addr 172.21.15.97:0/3893246651 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.531+0000 7fcac2800640 1 -- 172.21.15.97:0/3893246651 >> v1:172.21.15.188:6789/0 conn(0x7fcac40d18f0 legacy=0x7fcac40d00e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.531+0000 7fcac2800640 1 -- 172.21.15.97:0/3893246651 >> v1:172.21.15.97:6789/0 conn(0x7fcac40d3df0 legacy=0x7fcac415cf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.531+0000 7fcac2800640 1 -- 172.21.15.97:0/3893246651 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac40b6090 con 0x7fcac415d630 2024-01-31T05:01:12.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.530+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fed3c011b60 con 0x7fed4c0d39f0 2024-01-31T05:01:12.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fc229600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc224070420 0x7fc22414d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fc229600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2240b1500 0x7fc2240af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.531+0000 7fed48c00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed2406af30 0x7fed2406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.531+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed3c033e40 con 0x7fed4c0d39f0 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fed48c00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7fed240744f0 0x7fed24076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 --> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed24077000 con 0x7fed240744f0 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed3c053a90 con 0x7fed4c0d39f0 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fc229600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240ce760 0x7fc22414caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fc229600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240ac060 con 0x7fc2240ce760 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.532+0000 7fc229600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240cb0d0 con 0x7fc224070420 2024-01-31T05:01:12.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fc223400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240ce760 0x7fc22414caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7f71c3e00640 1 Processor -- start 2024-01-31T05:01:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fed51200640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed2406af30 0x7fed2406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fcac2800640 1 --2- 172.21.15.97:0/3893246651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac415d630 0x7fcac4168040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431e2fccd930c20e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7f71c3e00640 1 -- start start 2024-01-31T05:01:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fe9dc400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc054690 0x7fe9cc033cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fcac0a00640 1 -- 172.21.15.97:0/3893246651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030860 con 0x7fcac415d630 2024-01-31T05:01:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fe9dc400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0df1c0 0x7fe9cc0d97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fcac0a00640 1 -- 172.21.15.97:0/3893246651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac0309c0 con 0x7fcac415d630 2024-01-31T05:01:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fcac0a00640 1 -- 172.21.15.97:0/3893246651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030d50 con 0x7fcac415d630 2024-01-31T05:01:12.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fe9dc400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc0dfcb0 0x7fe9cc04b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fe9dc400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0362c0 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fc223400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240ce760 0x7fc22414caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43128/0 (socket says 172.21.15.97:43128) 2024-01-31T05:01:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fc223400640 1 -- 172.21.15.97:0/4281843947 learned_addr learned my addr 172.21.15.97:0/4281843947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fc222a00640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc224070420 0x7fc22414d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fc222000640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2240b1500 0x7fc2240af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240cb3b0 con 0x7fc2240b1500 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fc223400640 1 -- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2240b1500 msgr2=0x7fc2240af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fc223400640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2240b1500 0x7fc2240af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fc223400640 1 -- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc224070420 msgr2=0x7fc22414d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fc223400640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc224070420 0x7fc22414d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.534+0000 7fc223400640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c009400 con 0x7fc2240ce760 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fc222a00640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc224070420 0x7fc22414d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:12.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fc223400640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240ce760 0x7fc22414caa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c730 tx=0x7fc21c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fe9dc400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc04b940 con 0x7fe9cc0dfcb0 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fe9dc400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc0d8ff0 con 0x7fe9cc054690 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0df1c0 0x7fe9cc0d97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0df1c0 0x7fe9cc0d97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43132/0 (socket says 172.21.15.97:43132) 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 -- 172.21.15.97:0/468183645 learned_addr learned my addr 172.21.15.97:0/468183645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 -- 172.21.15.97:0/468183645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc054690 msgr2=0x7fe9cc033cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9cc054690 0x7fe9cc033cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 -- 172.21.15.97:0/468183645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc0dfcb0 msgr2=0x7fe9cc04b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fe9d9000640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9cc0dfcb0 0x7fe9cc04b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fe9d9000640 1 -- 172.21.15.97:0/468183645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d404efc0 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fcac3200640 1 -- 172.21.15.97:0/3893246651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac415d630 msgr2=0x7fcac4168040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.535+0000 7fcac3200640 1 --2- 172.21.15.97:0/3893246651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac415d630 0x7fcac4168040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac02b9a0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fcac3200640 1 -- 172.21.15.97:0/3893246651 shutdown_connections 2024-01-31T05:01:12.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fcac3200640 1 --2- 172.21.15.97:0/3893246651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac415d630 0x7fcac4168040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fcac3200640 1 -- 172.21.15.97:0/3893246651 >> 172.21.15.97:0/3893246651 conn(0x7fcac40d30a0 msgr2=0x7fcac40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.533+0000 7fed51c00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7fed240744f0 0x7fed24076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c006d50 con 0x7fc2240ce760 2024-01-31T05:01:12.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fcac3200640 1 -- 172.21.15.97:0/3893246651 shutdown_connections 2024-01-31T05:01:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c003970 con 0x7fc2240ce760 2024-01-31T05:01:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c003d60 con 0x7fc2240ce760 2024-01-31T05:01:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7fcac3200640 1 -- 172.21.15.97:0/3893246651 wait complete. 2024-01-31T05:01:12.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 0x7f71c40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7f71c3e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40d0a30 con 0x7f71c415d630 2024-01-31T05:01:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7f71c3e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40b69c0 con 0x7f71c40d3df0 2024-01-31T05:01:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.536+0000 7f71c3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40b6090 con 0x7f71c40d18f0 2024-01-31T05:01:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.537+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2240abcf0 con 0x7fc2240ce760 2024-01-31T05:01:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.537+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22414da40 con 0x7fc2240ce760 2024-01-31T05:01:12.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.538+0000 7fed51200640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed2406af30 0x7fed2406d3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed34047000 tx=0x7fed34008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.537+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 0x7f71c40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.537+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 0x7f71c40d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59772/0 (socket says 172.21.15.97:59772) 2024-01-31T05:01:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.537+0000 7f71c2a00640 1 -- 172.21.15.97:0/3102705353 learned_addr learned my addr 172.21.15.97:0/3102705353 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.539+0000 7fe9d9000640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0df1c0 0x7fe9cc0d97e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c780 tx=0x7fe9c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.539+0000 7fcac3200640 1 Processor -- start 2024-01-31T05:01:12.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.539+0000 7fcac3200640 1 -- start start 2024-01-31T05:01:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.538+0000 7fed51c00640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7fed240744f0 0x7fed24076930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.539+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4005270 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.540+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc009400 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.540+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc035c70 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.540+0000 7f71c2a00640 1 -- 172.21.15.97:0/3102705353 >> v1:172.21.15.188:6789/0 conn(0x7f71c40d3df0 legacy=0x7f71c415cf20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:01:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.540+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== osd.1 v2:172.21.15.97:6811/2289218318 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fed24077000 con 0x7fed240744f0 2024-01-31T05:01:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.541+0000 7fcac3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac40d18f0 0x7fcac4154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.541+0000 7f71c2a00640 1 -- 172.21.15.97:0/3102705353 >> v1:172.21.15.97:6789/0 conn(0x7f71c415d630 legacy=0x7f71c4167030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7fcac3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40d3df0 0x7fcac40ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7fcac3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac415d630 0x7fcac4157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7fcac3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac40ca6b0 con 0x7fcac40d18f0 2024-01-31T05:01:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7fcac3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4154e50 con 0x7fcac415d630 2024-01-31T05:01:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7fcac3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4155130 con 0x7fcac40d3df0 2024-01-31T05:01:12.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7f71c2a00640 1 -- 172.21.15.97:0/3102705353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40b6370 con 0x7f71c40d18f0 2024-01-31T05:01:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.542+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1f4001030 con 0x7fc2240ce760 2024-01-31T05:01:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.541+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c40053d0 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.541+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4005270 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.543+0000 7fcac1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac40d18f0 0x7fcac4154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.543+0000 7fcac2800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac415d630 0x7fcac4157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.543+0000 7fcac2800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac415d630 0x7fcac4157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59062/0 (socket says 172.21.15.97:59062) 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.543+0000 7fcac2800640 1 -- 172.21.15.97:0/3821467025 learned_addr learned my addr 172.21.15.97:0/3821467025 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.544+0000 7fcac2800640 1 -- 172.21.15.97:0/3821467025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40d3df0 msgr2=0x7fcac40ca170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.544+0000 7fcac2800640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcac40d3df0 0x7fcac40ca170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.544+0000 7fcac2800640 1 -- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac40d18f0 msgr2=0x7fcac4154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.544+0000 7fcac2800640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac40d18f0 0x7fcac4154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.544+0000 7fcac2800640 1 -- 172.21.15.97:0/3821467025 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac0094d0 con 0x7fcac415d630 2024-01-31T05:01:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.544+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9a4001030 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.545+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc21c0050d0 con 0x7fc2240ce760 2024-01-31T05:01:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.545+0000 7fcac2800640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac415d630 0x7fcac4157e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac030810 con 0x7fcac415d630 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac4155410 con 0x7fcac415d630 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac039d10 con 0x7fcac415d630 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac41559d0 con 0x7fcac415d630 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac043610 con 0x7fcac415d630 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcac40b6650 con 0x7fcac415d630 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.545+0000 7f71c2a00640 1 --2- 172.21.15.97:0/3102705353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 0x7f71c40d00e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009e80 tx=0x7f71bc02bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0196a0a88dc22c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7f71c1600640 1 -- 172.21.15.97:0/3102705353 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc02cb70 con 0x7f71c40d18f0 2024-01-31T05:01:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7f71c1600640 1 -- 172.21.15.97:0/3102705353 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc02ccd0 con 0x7f71c40d18f0 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7f71c1600640 1 -- 172.21.15.97:0/3102705353 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc034610 con 0x7f71c40d18f0 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 -- 172.21.15.97:0/3102705353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 msgr2=0x7f71c40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 --2- 172.21.15.97:0/3102705353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 0x7f71c40d00e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009e80 tx=0x7f71bc02bfd0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 -- 172.21.15.97:0/3102705353 shutdown_connections 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 --2- 172.21.15.97:0/3102705353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d18f0 0x7f71c40d00e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 -- 172.21.15.97:0/3102705353 >> 172.21.15.97:0/3102705353 conn(0x7f71c40d30a0 msgr2=0x7f71c40d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 -- 172.21.15.97:0/3102705353 shutdown_connections 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7fc21aa00640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc1fc06af20 0x7fc1fc06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc21c0892b0 con 0x7fc2240ce760 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7fc21aa00640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7fc1fc074560 0x7fc1fc0769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1fc077070 con 0x7fc1fc074560 2024-01-31T05:01:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21c08bbd0 con 0x7fc2240ce760 2024-01-31T05:01:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7f2891e00640 1 Processor -- start 2024-01-31T05:01:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.546+0000 7f2891e00640 1 -- start start 2024-01-31T05:01:12.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe9c400a640 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.548+0000 7f2891e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f288c159700 0x7f288c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.549+0000 7f2891e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f288c0ce430 0x7f288c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.547+0000 7f71c3e00640 1 -- 172.21.15.97:0/3102705353 wait complete. 2024-01-31T05:01:12.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.549+0000 7f2891e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c159ae0 con 0x7f288c06e510 2024-01-31T05:01:12.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.549+0000 7f2891e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288c0ccd70 con 0x7f288c0ce430 2024-01-31T05:01:12.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.549+0000 7f2891e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288c0b35d0 con 0x7f288c159700 2024-01-31T05:01:12.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.549+0000 7fc222000640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7fc1fc074560 0x7fc1fc0769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.549+0000 7fc222a00640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc1fc06af20 0x7fc1fc06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7fe9d1600640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe9ac06af60 0x7fe9ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7f288aa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f288c159700 0x7f288c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7f288b400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f288c06e510 0x7f288c0cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41724/0 (socket says 172.21.15.97:41724) 2024-01-31T05:01:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9c4088c10 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7f288b400640 1 -- 172.21.15.97:0/664891474 learned_addr learned my addr 172.21.15.97:0/664891474 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fe9d1600640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fe9ac0745a0 0x7fe9ac0769e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 --> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9ac0770b0 con 0x7fe9ac0745a0 2024-01-31T05:01:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c408d2a0 con 0x7fe9cc0df1c0 2024-01-31T05:01:12.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7f288a000640 1 --2- 172.21.15.97:0/664891474 >> v2:172.21.15.97:3300/0 conn(0x7f288c0ce430 0x7f288c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fcaac04c3b0 con 0x7fcac415d630 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fcaba000640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca9c06aed0 0x7fca9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcaac0abee0 con 0x7fcac415d630 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fcaba000640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7fca9c074490 0x7fca9c0768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 --> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c076fa0 con 0x7fca9c074490 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.551+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaac0ac250 con 0x7fcac415d630 2024-01-31T05:01:12.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.550+0000 7fc222000640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7fc1fc074560 0x7fc1fc0769a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.552+0000 7fcac1e00640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca9c06aed0 0x7fca9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== osd.0 v2:172.21.15.97:6800/2799621490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc1fc077070 con 0x7fc1fc074560 2024-01-31T05:01:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7fc222a00640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc1fc06af20 0x7fc1fc06d3e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc20c036000 tx=0x7fc20c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.552+0000 7fe9da400640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fe9ac0745a0 0x7fe9ac0769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7fe9da400640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fe9ac0745a0 0x7fe9ac0769e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7fe9d9a00640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe9ac06af60 0x7fe9ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7fcac1400640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7fca9c074490 0x7fca9c0768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.554+0000 7fcac1e00640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca9c06aed0 0x7fca9c06d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009f40 tx=0x7fcabc0094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.554+0000 7fcac1400640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7fca9c074490 0x7fca9c0768d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.554+0000 7f71c3e00640 1 Processor -- start 2024-01-31T05:01:12.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7f288aa00640 1 -- 172.21.15.97:0/664891474 >> v2:172.21.15.97:3300/0 conn(0x7f288c0ce430 msgr2=0x7f288c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.554+0000 7f71c3e00640 1 -- start start 2024-01-31T05:01:12.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7f288aa00640 1 --2- 172.21.15.97:0/664891474 >> v2:172.21.15.97:3300/0 conn(0x7f288c0ce430 0x7f288c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7f288aa00640 1 -- 172.21.15.97:0/664891474 >> v1:172.21.15.97:6789/0 conn(0x7f288c06e510 legacy=0x7f288c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7f2889600640 1 -- 172.21.15.97:0/664891474 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984401529 0 0) 0x7f288c159ae0 con 0x7f288c06e510 2024-01-31T05:01:12.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.553+0000 7f288aa00640 1 -- 172.21.15.97:0/664891474 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0b38b0 con 0x7f288c159700 2024-01-31T05:01:12.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.554+0000 7fe9d9a00640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe9ac06af60 0x7fe9ac06d420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4063d10 tx=0x7fe9d408d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.555+0000 7f288a000640 1 --2- 172.21.15.97:0/664891474 >> v2:172.21.15.97:3300/0 conn(0x7f288c0ce430 0x7f288c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:12.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.555+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== osd.2 v2:172.21.15.97:6808/2991223614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe9ac0770b0 con 0x7fe9ac0745a0 2024-01-31T05:01:12.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.555+0000 7f288aa00640 1 --2- 172.21.15.97:0/664891474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c159700 0x7f288c0abcf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f288c06b670 tx=0x7f287c02b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5dc6db05ddcd4d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.556+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== osd.3 v2:172.21.15.97:6821/4184396147 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca9c076fa0 con 0x7fca9c074490 2024-01-31T05:01:12.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.556+0000 7f2889600640 1 -- 172.21.15.97:0/664891474 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0317e0 con 0x7f288c159700 2024-01-31T05:01:12.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.556+0000 7f2889600640 1 -- 172.21.15.97:0/664891474 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c031940 con 0x7f288c159700 2024-01-31T05:01:12.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.556+0000 7f2889600640 1 -- 172.21.15.97:0/664891474 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c031d10 con 0x7f288c159700 2024-01-31T05:01:12.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.556+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c40d18f0 0x7f71c415f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.556+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d3df0 0x7f71c40b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.557+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c40d18f0 0x7f71c415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.557+0000 7f71c3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c415d630 0x7f71c415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.557+0000 7f71c3e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c415e490 con 0x7f71c40d18f0 2024-01-31T05:01:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.557+0000 7f71c3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c415e770 con 0x7f71c415d630 2024-01-31T05:01:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.557+0000 7f71c3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c415ea50 con 0x7f71c40d3df0 2024-01-31T05:01:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7f71c2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d3df0 0x7f71c40b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7f2891e00640 1 -- 172.21.15.97:0/664891474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c159700 msgr2=0x7f288c0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7f2891e00640 1 --2- 172.21.15.97:0/664891474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c159700 0x7f288c0abcf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f288c06b670 tx=0x7f287c02b880 comp rx=0 tx=0).stop 2024-01-31T05:01:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7f71c2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d3df0 0x7f71c40b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59784/0 (socket says 172.21.15.97:59784) 2024-01-31T05:01:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7f71c2000640 1 -- 172.21.15.97:0/3352908520 learned_addr learned my addr 172.21.15.97:0/3352908520 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7f71c3400640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c415d630 0x7f71c415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.558+0000 7fe18fa00640 1 Processor -- start 2024-01-31T05:01:12.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.559+0000 7fe18fa00640 1 -- start start 2024-01-31T05:01:12.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.559+0000 7f2891e00640 1 -- 172.21.15.97:0/664891474 shutdown_connections 2024-01-31T05:01:12.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.559+0000 7f2891e00640 1 --2- 172.21.15.97:0/664891474 >> v2:172.21.15.97:3300/0 conn(0x7f288c0ce430 0x7f288c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.559+0000 7f2891e00640 1 --2- 172.21.15.97:0/664891474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c159700 0x7f288c0abcf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.559+0000 7f2891e00640 1 -- 172.21.15.97:0/664891474 >> 172.21.15.97:0/664891474 conn(0x7f288c0b0a10 msgr2=0x7f288c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f71c2a00640 1 -- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d3df0 msgr2=0x7f71c40b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f71c2a00640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71c40d3df0 0x7f71c40b7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f71c2a00640 1 -- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c415d630 msgr2=0x7f71c415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f71c2a00640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71c415d630 0x7f71c415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f71c2a00640 1 -- 172.21.15.97:0/3352908520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009b30 con 0x7f71c40d18f0 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f2891e00640 1 -- 172.21.15.97:0/664891474 shutdown_connections 2024-01-31T05:01:12.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.560+0000 7f2891e00640 1 -- 172.21.15.97:0/664891474 wait complete. 2024-01-31T05:01:12.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe18fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1880cff20 0x7fe1880d14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe18fa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18815ec50 con 0x7fe1880cfae0 2024-01-31T05:01:12.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe18fa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1880cded0 con 0x7fe18815d590 2024-01-31T05:01:12.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe18fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18815d970 con 0x7fe1880cff20 2024-01-31T05:01:12.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe187e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe1880cfae0 0x7fe1880d0de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41734/0 (socket says 172.21.15.97:41734) 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe187e00640 1 -- 172.21.15.97:0/4005236004 learned_addr learned my addr 172.21.15.97:0/4005236004 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe18da00640 1 --2- 172.21.15.97:0/4005236004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1880cff20 0x7fe1880d14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe187400640 1 -- 172.21.15.97:0/4005236004 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121877213 0 0) 0x7fe18815ec50 con 0x7fe1880cfae0 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.561+0000 7fe187400640 1 -- 172.21.15.97:0/4005236004 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe168003520 con 0x7fe1880cfae0 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.562+0000 7fe18da00640 1 -- 172.21.15.97:0/4005236004 >> v1:172.21.15.188:6789/0 conn(0x7fe18815d590 legacy=0x7fe18815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.562+0000 7fe18da00640 1 -- 172.21.15.97:0/4005236004 >> v1:172.21.15.97:6789/0 conn(0x7fe1880cfae0 legacy=0x7fe1880d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.562+0000 7fe18da00640 1 -- 172.21.15.97:0/4005236004 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880cf210 con 0x7fe1880cff20 2024-01-31T05:01:12.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.563+0000 7f71c2a00640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c40d18f0 0x7f71c415f530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f71bc032000 tx=0x7f71bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.564+0000 7fe18da00640 1 --2- 172.21.15.97:0/4005236004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1880cff20 0x7fe1880d14f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe1780097d0 tx=0x7fe17802b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95cddc48a53d2ba9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.564+0000 7fe187400640 1 -- 172.21.15.97:0/4005236004 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1800019f0 con 0x7fe1880cff20 2024-01-31T05:01:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.565+0000 7fe187400640 1 -- 172.21.15.97:0/4005236004 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17802e050 con 0x7fe1880cff20 2024-01-31T05:01:12.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.565+0000 7fe187400640 1 -- 172.21.15.97:0/4005236004 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178030730 con 0x7fe1880cff20 2024-01-31T05:01:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.565+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc03e450 con 0x7f71c40d18f0 2024-01-31T05:01:12.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.565+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc03e5b0 con 0x7f71c40d18f0 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.565+0000 7f2891e00640 1 Processor -- start 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.566+0000 7f2891e00640 1 -- start start 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.566+0000 7f2891e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288c06e510 0x7f288c0ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f2891e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c0ce430 0x7f288c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f2891e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f288c159700 0x7f288c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f2891e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c15c210 con 0x7f288c06e510 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f2891e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0af2d0 con 0x7f288c0ce430 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f2891e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c15c460 con 0x7f288c159700 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f288aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288c06e510 0x7f288c0ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f288aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288c06e510 0x7f288c0ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43154/0 (socket says 172.21.15.97:43154) 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f288aa00640 1 -- 172.21.15.97:0/3754161810 learned_addr learned my addr 172.21.15.97:0/3754161810 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f288b400640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f288c159700 0x7f288c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.567+0000 7f288a000640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c0ce430 0x7f288c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.565+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc03e960 con 0x7f71c40d18f0 2024-01-31T05:01:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c415ed30 con 0x7f71c40d18f0 2024-01-31T05:01:12.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c40b7840 con 0x7f71c40d18f0 2024-01-31T05:01:12.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f288aa00640 1 -- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f288c159700 msgr2=0x7f288c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f288aa00640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f288c159700 0x7f288c0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f288aa00640 1 -- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c0ce430 msgr2=0x7f288c15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f288aa00640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f288c0ce430 0x7f288c15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7f288aa00640 1 -- 172.21.15.97:0/3754161810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874009400 con 0x7f288c06e510 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7fe18fa00640 1 -- 172.21.15.97:0/4005236004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1880cff20 msgr2=0x7fe1880d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.568+0000 7fe18fa00640 1 --2- 172.21.15.97:0/4005236004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1880cff20 0x7fe1880d14f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe1780097d0 tx=0x7fe17802b7d0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.569+0000 7fe18fa00640 1 -- 172.21.15.97:0/4005236004 shutdown_connections 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.569+0000 7fe18fa00640 1 --2- 172.21.15.97:0/4005236004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1880cff20 0x7fe1880d14f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.569+0000 7fe18fa00640 1 -- 172.21.15.97:0/4005236004 >> 172.21.15.97:0/4005236004 conn(0x7fe18809c560 msgr2=0x7fe1880d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.569+0000 7fe18fa00640 1 -- 172.21.15.97:0/4005236004 shutdown_connections 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.569+0000 7fe18fa00640 1 -- 172.21.15.97:0/4005236004 wait complete. 2024-01-31T05:01:12.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.571+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71c415e8e0 con 0x7f71c40d18f0 2024-01-31T05:01:12.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.571+0000 7f288aa00640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288c06e510 0x7f288c0ad120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f287c000c00 tx=0x7f287c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.571+0000 7fe18fa00640 1 Processor -- start 2024-01-31T05:01:12.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.572+0000 7fe18fa00640 1 -- start start 2024-01-31T05:01:12.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.572+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c032680 con 0x7f288c06e510 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.572+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c0093a0 con 0x7f288c06e510 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.572+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c0327e0 con 0x7f288c06e510 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.572+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c032b90 con 0x7f288c06e510 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.573+0000 7fe18fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1880cfae0 0x7fe188157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.573+0000 7fe18fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1880cff20 0x7fe1880c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.573+0000 7fe18fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe18815d590 0x7fe1880ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.574+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c15ca20 con 0x7f288c06e510 2024-01-31T05:01:12.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.574+0000 7fe18d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1880cfae0 0x7fe188157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.574+0000 7fe18d000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1880cfae0 0x7fe188157070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43158/0 (socket says 172.21.15.97:43158) 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.575+0000 7fe18d000640 1 -- 172.21.15.97:0/2153392293 learned_addr learned my addr 172.21.15.97:0/2153392293 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.575+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1881f7ba0 con 0x7fe1880cfae0 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.575+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1881f7e80 con 0x7fe1880cff20 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.575+0000 7fe187e00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1880cff20 0x7fe1880c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.575+0000 7fe18da00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe18815d590 0x7fe1880ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.575+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1881f8160 con 0x7fe18815d590 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f288c0b3e70 con 0x7f288c06e510 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f287c032cf0 con 0x7f288c06e510 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.578+0000 7f2883400640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f286406aed0 0x7f286406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.578+0000 7f288a000640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f286406aed0 0x7f286406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.577+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f71bc03eac0 con 0x7f71c40d18f0 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.578+0000 7f71bb400640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f719c06afd0 0x7f719c06d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.579+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f71bc02ce40 con 0x7f71c40d18f0 2024-01-31T05:01:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7fe18da00640 1 -- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1880cff20 msgr2=0x7fe1880c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7fe18da00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1880cff20 0x7fe1880c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7fe18da00640 1 -- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1880cfae0 msgr2=0x7fe188157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7fe18da00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1880cfae0 0x7fe188157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.576+0000 7fe18da00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe178009400 con 0x7fe18815d590 2024-01-31T05:01:12.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.580+0000 7f288a000640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f286406aed0 0x7f286406d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f288c0af1d0 tx=0x7f2874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.580+0000 7f71c2000640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f719c06afd0 0x7f719c06d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.580+0000 7fe18da00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe18815d590 0x7fe1880ccad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe18806b370 tx=0x7fe178004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f287c082f30 con 0x7f288c06e510 2024-01-31T05:01:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17803e860 con 0x7fe18815d590 2024-01-31T05:01:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7f2883400640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7f2864074410 0x7f2864076850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1881f8440 con 0x7fe18815d590 2024-01-31T05:01:12.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1881f7860 con 0x7fe18815d590 2024-01-31T05:01:12.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 --> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2864076f20 con 0x7f2864074410 2024-01-31T05:01:12.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f287c07eb70 con 0x7f288c06e510 2024-01-31T05:01:12.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.581+0000 7f71bb400640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f719c074610 0x7f719c076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.582+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 --> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f719c077120 con 0x7f719c074610 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.582+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71bc0ae3f0 con 0x7f71c40d18f0 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.582+0000 7f71c2000640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f719c06afd0 0x7f719c06d490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71ac006fd0 tx=0x7f71ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.582+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1881f7ff0 con 0x7fe18815d590 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.582+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17803e9c0 con 0x7fe18815d590 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.582+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17803e860 con 0x7fe18815d590 2024-01-31T05:01:12.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.583+0000 7f71c3400640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f719c074610 0x7f719c076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.583+0000 7f288b400640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7f2864074410 0x7f2864076850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.584+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 --> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1f4002d30 con 0x7fc1fc074560 2024-01-31T05:01:12.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.584+0000 7fc21aa00640 1 -- 172.21.15.97:0/4281843947 <== osd.0 v2:172.21.15.97:6800/2799621490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1f4002d30 con 0x7fc1fc074560 2024-01-31T05:01:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.584+0000 7f71c3400640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f719c074610 0x7f719c076a50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.585+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7fc1fc074560 msgr2=0x7fc1fc0769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.585+0000 7fc229600640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7fc1fc074560 0x7fc1fc0769a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc1fc06af20 msgr2=0x7fc1fc06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc1fc06af20 0x7fc1fc06d3e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc20c036000 tx=0x7fc20c008030 comp rx=0 tx=0).stop 2024-01-31T05:01:12.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240ce760 msgr2=0x7fc22414caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 --2- 172.21.15.97:0/4281843947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2240ce760 0x7fc22414caa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c730 tx=0x7fc21c00cbf0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7f288b400640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7f2864074410 0x7f2864076850 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc223400640 1 -- 172.21.15.97:0/4281843947 reap_dead start 2024-01-31T05:01:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 shutdown_connections 2024-01-31T05:01:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 >> 172.21.15.97:0/4281843947 conn(0x7fc2240b0a10 msgr2=0x7fc224071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 shutdown_connections 2024-01-31T05:01:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.586+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe17803eb30 con 0x7fe18815d590 2024-01-31T05:01:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.587+0000 7fc229600640 1 -- 172.21.15.97:0/4281843947 wait complete. 2024-01-31T05:01:12.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.588+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== osd.4 v2:172.21.15.188:6824/2903566089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f719c077120 con 0x7f719c074610 2024-01-31T05:01:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.588+0000 7fe184c00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe16006af30 0x7fe16006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.588+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe1780ab7e0 con 0x7fe18815d590 2024-01-31T05:01:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.589+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 --> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9a4002d30 con 0x7fe9ac0745a0 2024-01-31T05:01:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.589+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== osd.6 v2:172.21.15.188:6800/3025033562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2864076f20 con 0x7f2864074410 2024-01-31T05:01:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.589+0000 7fe184c00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7fe1600744f0 0x7fe160076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.589+0000 7fe18d000640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe16006af30 0x7fe16006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.590+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe160077000 con 0x7fe1600744f0 2024-01-31T05:01:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.590+0000 7fe9d1600640 1 -- 172.21.15.97:0/468183645 <== osd.2 v2:172.21.15.97:6808/2991223614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe9a4002d30 con 0x7fe9ac0745a0 2024-01-31T05:01:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.590+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1780abc30 con 0x7fe18815d590 2024-01-31T05:01:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.590+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 --> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed4c160410 con 0x7fed240744f0 2024-01-31T05:01:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fe9ac0745a0 msgr2=0x7fe9ac0769e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe18d000640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe16006af30 0x7fe16006d3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1880d1c30 tx=0x7fe170009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe187e00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7fe1600744f0 0x7fe160076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9dc400640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fe9ac0745a0 0x7fe9ac0769e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fed48c00640 1 -- 172.21.15.97:0/3331005807 <== osd.1 v2:172.21.15.97:6811/2289218318 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed4c160410 con 0x7fed240744f0 2024-01-31T05:01:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe9ac06af60 msgr2=0x7fe9ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9dc400640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe9ac06af60 0x7fe9ac06d420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4063d10 tx=0x7fe9d408d040 comp rx=0 tx=0).stop 2024-01-31T05:01:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0df1c0 msgr2=0x7fe9cc0d97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9dc400640 1 --2- 172.21.15.97:0/468183645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9cc0df1c0 0x7fe9cc0d97e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c780 tx=0x7fe9c400cc40 comp rx=0 tx=0).stop 2024-01-31T05:01:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.591+0000 7fe9da400640 1 -- 172.21.15.97:0/468183645 reap_dead start 2024-01-31T05:01:12.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 shutdown_connections 2024-01-31T05:01:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 >> 172.21.15.97:0/468183645 conn(0x7fe9cc009fe0 msgr2=0x7fe9cc04fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7fed240744f0 msgr2=0x7fed24076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed52600640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7fed240744f0 0x7fed24076930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed2406af30 msgr2=0x7fed2406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed52600640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed2406af30 0x7fed2406d3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed34047000 tx=0x7fed34008030 comp rx=0 tx=0).stop 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 msgr2=0x7fed4c1511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed52600640 1 --2- 172.21.15.97:0/3331005807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4c0d39f0 0x7fed4c1511a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00b450 tx=0x7fed3c00b910 comp rx=0 tx=0).stop 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.592+0000 7fed51c00640 1 -- 172.21.15.97:0/3331005807 reap_dead start 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 shutdown_connections 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 >> 172.21.15.97:0/3331005807 conn(0x7fed4c09c400 msgr2=0x7fed4c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 shutdown_connections 2024-01-31T05:01:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fe187e00640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7fe1600744f0 0x7fe160076930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fe9dc400640 1 -- 172.21.15.97:0/468183645 wait complete. 2024-01-31T05:01:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 shutdown_connections 2024-01-31T05:01:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.593+0000 7fed52600640 1 -- 172.21.15.97:0/3331005807 wait complete. 2024-01-31T05:01:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.595+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== osd.5 v2:172.21.15.188:6809/576621421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe160077000 con 0x7fe1600744f0 2024-01-31T05:01:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.601+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 --> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcac40d07f0 con 0x7fca9c074490 2024-01-31T05:01:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.607+0000 7fcaba000640 1 -- 172.21.15.97:0/3821467025 <== osd.3 v2:172.21.15.97:6821/4184396147 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcac40d07f0 con 0x7fca9c074490 2024-01-31T05:01:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.611+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7fca9c074490 msgr2=0x7fca9c0768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.611+0000 7fcac3200640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7fca9c074490 0x7fca9c0768d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.613+0000 7fb0f4a00640 1 Processor -- start 2024-01-31T05:01:12.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.613+0000 7fb0f4a00640 1 -- start start 2024-01-31T05:01:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.614+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca9c06aed0 msgr2=0x7fca9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.614+0000 7fcac3200640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca9c06aed0 0x7fca9c06d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009f40 tx=0x7fcabc0094d0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.614+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac415d630 msgr2=0x7fcac4157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.614+0000 7fcac3200640 1 --2- 172.21.15.97:0/3821467025 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcac415d630 0x7fcac4157e60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009870 tx=0x7fcaac004730 comp rx=0 tx=0).stop 2024-01-31T05:01:12.615 INFO:teuthology.orchestra.run.smithi097.stdout:60129542153 2024-01-31T05:01:12.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-31T05:01:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.615+0000 7fcac2800640 1 -- 172.21.15.97:0/3821467025 reap_dead start 2024-01-31T05:01:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.615+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 shutdown_connections 2024-01-31T05:01:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.615+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 >> 172.21.15.97:0/3821467025 conn(0x7fcac40d30a0 msgr2=0x7fcac406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.616+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 shutdown_connections 2024-01-31T05:01:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.616+0000 7fb0f4a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0f00c9ae0 0x7fb0f00cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.616+0000 7fb0ede00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb0f00cc9e0 0x7fb0f00c93d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41740/0 (socket says 172.21.15.97:41740) 2024-01-31T05:01:12.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.616+0000 7fb0ede00640 1 -- 172.21.15.97:0/3440815370 learned_addr learned my addr 172.21.15.97:0/3440815370 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:12.617 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.616+0000 7fcac3200640 1 -- 172.21.15.97:0/3821467025 wait complete. 2024-01-31T05:01:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.617+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f006d000 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.617+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f006c3f0 con 0x7fb0f015c920 2024-01-31T05:01:12.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.617+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f0070a00 con 0x7fb0f00c9ae0 2024-01-31T05:01:12.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-31T05:01:12.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.617+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 --> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f71c415ce60 con 0x7f719c074610 2024-01-31T05:01:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.617+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432314974 0 0) 0x7fb0f006d000 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.617+0000 7fb0ef200640 1 --2- 172.21.15.97:0/3440815370 >> v2:172.21.15.97:3300/0 conn(0x7fb0f00c9ae0 0x7fb0f00cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d0003440 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110579389 0 0) 0x7fb0f006c3f0 con 0x7fb0f015c920 2024-01-31T05:01:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0f006d000 con 0x7fb0f015c920 2024-01-31T05:01:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7fb0ef200640 1 -- 172.21.15.97:0/3440815370 >> v1:172.21.15.188:6789/0 conn(0x7fb0f015c920 legacy=0x7fb0f00702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7fb0ef200640 1 -- 172.21.15.97:0/3440815370 >> v1:172.21.15.97:6789/0 conn(0x7fb0f00cc9e0 legacy=0x7fb0f00c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7fb0ef200640 1 -- 172.21.15.97:0/3440815370 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f006c6d0 con 0x7fb0f00c9ae0 2024-01-31T05:01:12.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.618+0000 7f71bb400640 1 -- 172.21.15.97:0/3352908520 <== osd.4 v2:172.21.15.188:6824/2903566089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f71c415ce60 con 0x7f719c074610 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f719c074610 msgr2=0x7f719c076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3e00640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f719c074610 0x7f719c076a50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f719c06afd0 msgr2=0x7f719c06d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3e00640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f719c06afd0 0x7f719c06d490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71ac006fd0 tx=0x7f71ac008040 comp rx=0 tx=0).stop 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c40d18f0 msgr2=0x7f71c415f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3e00640 1 --2- 172.21.15.97:0/3352908520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c40d18f0 0x7f71c415f530 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f71bc032000 tx=0x7f71bc004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.619+0000 7f71c3400640 1 -- 172.21.15.97:0/3352908520 reap_dead start 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.620+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 shutdown_connections 2024-01-31T05:01:12.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.620+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 >> 172.21.15.97:0/3352908520 conn(0x7f71c40d30a0 msgr2=0x7f71c406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.624 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-01-31T05:01:12.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.620+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 shutdown_connections 2024-01-31T05:01:12.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.620+0000 7f71c3e00640 1 -- 172.21.15.97:0/3352908520 wait complete. 2024-01-31T05:01:12.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.620+0000 7fb0ef200640 1 --2- 172.21.15.97:0/3440815370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00c9ae0 0x7fb0f00cd8d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009d90 tx=0x7fb0e802bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7791140ecd353c3a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.621+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0001670 con 0x7fb0f00c9ae0 2024-01-31T05:01:12.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.621+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e802c7d0 con 0x7fb0f00c9ae0 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.621+0000 7fb0ed400640 1 -- 172.21.15.97:0/3440815370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e802cb60 con 0x7fb0f00c9ae0 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.623+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00c9ae0 msgr2=0x7fb0f00cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.623+0000 7fb0f4a00640 1 --2- 172.21.15.97:0/3440815370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00c9ae0 0x7fb0f00cd8d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009d90 tx=0x7fb0e802bca0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.623+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 shutdown_connections 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.623+0000 7fb0f4a00640 1 --2- 172.21.15.97:0/3440815370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00c9ae0 0x7fb0f00cd8d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.623+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 >> 172.21.15.97:0/3440815370 conn(0x7fb0f00ced80 msgr2=0x7fb0f00cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.623+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 shutdown_connections 2024-01-31T05:01:12.625 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-31T05:01:12.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.624+0000 7fb0f4a00640 1 -- 172.21.15.97:0/3440815370 wait complete. 2024-01-31T05:01:12.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.625+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 --> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f288c073540 con 0x7f2864074410 2024-01-31T05:01:12.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.627+0000 7fb0f4a00640 1 Processor -- start 2024-01-31T05:01:12.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fe18fa00640 1 -- 172.21.15.97:0/2153392293 --> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1880ce730 con 0x7fe1600744f0 2024-01-31T05:01:12.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.630+0000 7fe184c00640 1 -- 172.21.15.97:0/2153392293 <== osd.5 v2:172.21.15.188:6809/576621421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe180001dd0 con 0x7fe1600744f0 2024-01-31T05:01:12.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7fe1600744f0 msgr2=0x7fe160076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fe17e000640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7fe1600744f0 0x7fe160076930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.627+0000 7f2883400640 1 -- 172.21.15.97:0/3754161810 <== osd.6 v2:172.21.15.188:6800/3025033562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f288c073540 con 0x7f2864074410 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.628+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7f2864074410 msgr2=0x7f2864076850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.628+0000 7f2891e00640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7f2864074410 0x7f2864076850 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f286406aed0 msgr2=0x7f286406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f286406aed0 0x7f286406d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f288c0af1d0 tx=0x7f2874008040 comp rx=0 tx=0).stop 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288c06e510 msgr2=0x7f288c0ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 --2- 172.21.15.97:0/3754161810 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288c06e510 0x7f288c0ad120 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f287c000c00 tx=0x7f287c004730 comp rx=0 tx=0).stop 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f288b400640 1 -- 172.21.15.97:0/3754161810 reap_dead start 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 shutdown_connections 2024-01-31T05:01:12.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 >> 172.21.15.97:0/3754161810 conn(0x7f288c0b0a10 msgr2=0x7f288c071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 shutdown_connections 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7f2891e00640 1 -- 172.21.15.97:0/3754161810 wait complete. 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.627+0000 7fb0f4a00640 1 -- start start 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.628+0000 7fb0f4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0f00c9ae0 0x7fb0f01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.628+0000 7fb0f4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00cc9e0 0x7fb0f01584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0ede00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00cc9e0 0x7fb0f01584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0ede00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00cc9e0 0x7fb0f01584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43180/0 (socket says 172.21.15.97:43180) 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0ee800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0f00c9ae0 0x7fb0f01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0f4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0f015c920 0x7fb0f0147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0f4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f01f7b30 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0f4a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00c6fd0 con 0x7fb0f00c9ae0 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0f4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f014d970 con 0x7fb0f015c920 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.629+0000 7fb0ede00640 1 -- 172.21.15.97:0/2138156053 learned_addr learned my addr 172.21.15.97:0/2138156053 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.630+0000 7fb0ef200640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0f015c920 0x7fb0f0147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ede00640 1 -- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0f015c920 msgr2=0x7fb0f0147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ede00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0f015c920 0x7fb0f0147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ede00640 1 -- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0f00c9ae0 msgr2=0x7fb0f01f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ede00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0f00c9ae0 0x7fb0f01f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ede00640 1 -- 172.21.15.97:0/2138156053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8009a70 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ef200640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0f015c920 0x7fb0f0147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.631+0000 7fb0ede00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00cc9e0 0x7fb0f01584c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c840 tx=0x7fb0e000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.632+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00074e0 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe16006af30 msgr2=0x7fe16006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe17e000640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe16006af30 0x7fe16006d3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe1880d1c30 tx=0x7fe170009040 comp rx=0 tx=0).stop 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe18815d590 msgr2=0x7fe1880ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe17e000640 1 --2- 172.21.15.97:0/2153392293 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe18815d590 0x7fe1880ccad0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe18806b370 tx=0x7fe178004730 comp rx=0 tx=0).stop 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe18da00640 1 -- 172.21.15.97:0/2153392293 reap_dead start 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 shutdown_connections 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.636+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 >> 172.21.15.97:0/2153392293 conn(0x7fe18809c560 msgr2=0x7fe1880b6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.639+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 shutdown_connections 2024-01-31T05:01:12.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.639+0000 7fe17e000640 1 -- 172.21.15.97:0/2153392293 wait complete. 2024-01-31T05:01:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.632+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0007640 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.632+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e00049e0 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.633+0000 7fb0f4a00640 1 -- 172.21.15.97:0/2138156053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f014d3b0 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.634+0000 7fb0f4a00640 1 -- 172.21.15.97:0/2138156053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f00c7270 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.635+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0b8001030 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.639+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb0e0004b40 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.641+0000 7fb0e6a00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0c806afa0 0x7fb0c806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.641+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb0e0089910 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.642+0000 7fb0e6a00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7fb0c80745e0 0x7fb0c8076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.642+0000 7fb0ee800640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0c806afa0 0x7fb0c806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.644+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 --> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0c80770f0 con 0x7fb0c80745e0 2024-01-31T05:01:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.644+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e0089cc0 con 0x7fb0f00cc9e0 2024-01-31T05:01:12.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.644+0000 7fb0ee800640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0c806afa0 0x7fb0c806d460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8001010 tx=0x7fb0d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.645+0000 7fb0ef200640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7fb0c80745e0 0x7fb0c8076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.645 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-01-31T05:01:12.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-31T05:01:12.646 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:12.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.647+0000 7fb0ef200640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7fb0c80745e0 0x7fb0c8076a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-31T05:01:12.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.650+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== osd.7 v2:172.21.15.188:6808/2866753840 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0c80770f0 con 0x7fb0c80745e0 2024-01-31T05:01:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:60129542154 2024-01-31T05:01:12.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-31T05:01:12.666 INFO:teuthology.orchestra.run.smithi097.stdout:60129542153 2024-01-31T05:01:12.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-31T05:01:12.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.682+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 --> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0b8002d30 con 0x7fb0c80745e0 2024-01-31T05:01:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.683+0000 7fb0e6a00640 1 -- 172.21.15.97:0/2138156053 <== osd.7 v2:172.21.15.188:6808/2866753840 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb0b8002d30 con 0x7fb0c80745e0 2024-01-31T05:01:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.686+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7fb0c80745e0 msgr2=0x7fb0c8076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.686+0000 7fb0dfe00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7fb0c80745e0 0x7fb0c8076a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.686+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0c806afa0 msgr2=0x7fb0c806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.686+0000 7fb0dfe00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0c806afa0 0x7fb0c806d460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8001010 tx=0x7fb0d8009040 comp rx=0 tx=0).stop 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.686+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00cc9e0 msgr2=0x7fb0f01584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.687+0000 7fb0dfe00640 1 --2- 172.21.15.97:0/2138156053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0f00cc9e0 0x7fb0f01584c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000c840 tx=0x7fb0e000cd00 comp rx=0 tx=0).stop 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.687+0000 7fb0ef200640 1 -- 172.21.15.97:0/2138156053 reap_dead start 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.687+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 shutdown_connections 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.687+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 >> 172.21.15.97:0/2138156053 conn(0x7fb0f00ced80 msgr2=0x7fb0f00c3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.687+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 shutdown_connections 2024-01-31T05:01:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.687+0000 7fb0dfe00640 1 -- 172.21.15.97:0/2138156053 wait complete. 2024-01-31T05:01:12.715 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-01-31T05:01:12.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-31T05:01:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.930+0000 7fea9b600640 1 Processor -- start 2024-01-31T05:01:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.931+0000 7fea9b600640 1 -- start start 2024-01-31T05:01:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.932+0000 7fea9b600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea9415d590 0x7fea9415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.932+0000 7fea9b600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fea940cfae0 0x7fea940cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.932+0000 7fea9b600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940d0460 con 0x7fea94161080 2024-01-31T05:01:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.932+0000 7fea9b600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea940cded0 con 0x7fea9415d590 2024-01-31T05:01:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.932+0000 7fea9b600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9415d970 con 0x7fea940cfae0 2024-01-31T05:01:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.933+0000 7fea98c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea9415d590 0x7fea9415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.933+0000 7fea98c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea9415d590 0x7fea9415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43182/0 (socket says 172.21.15.97:43182) 2024-01-31T05:01:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.933+0000 7fea98c00640 1 -- 172.21.15.97:0/1593774239 learned_addr learned my addr 172.21.15.97:0/1593774239 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.934+0000 7fea93e00640 1 --2- 172.21.15.97:0/1593774239 >> v2:172.21.15.188:3300/0 conn(0x7fea940cfae0 0x7fea940cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.934+0000 7fea98c00640 1 -- 172.21.15.97:0/1593774239 >> v2:172.21.15.188:3300/0 conn(0x7fea940cfae0 msgr2=0x7fea940cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.934+0000 7fea98c00640 1 --2- 172.21.15.97:0/1593774239 >> v2:172.21.15.188:3300/0 conn(0x7fea940cfae0 0x7fea940cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.934+0000 7fea98c00640 1 -- 172.21.15.97:0/1593774239 >> v1:172.21.15.188:6789/0 conn(0x7fea94161080 legacy=0x7fea941614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.934+0000 7fea98c00640 1 -- 172.21.15.97:0/1593774239 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea940cf210 con 0x7fea9415d590 2024-01-31T05:01:12.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.935+0000 7f3b1ea00640 1 Processor -- start 2024-01-31T05:01:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.936+0000 7fea98c00640 1 --2- 172.21.15.97:0/1593774239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415e540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea940d3170 tx=0x7fea7c02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab9d03e4f47913a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.936+0000 7fea93400640 1 -- 172.21.15.97:0/1593774239 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c030880 con 0x7fea9415d590 2024-01-31T05:01:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.936+0000 7fea93400640 1 -- 172.21.15.97:0/1593774239 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c0309e0 con 0x7fea9415d590 2024-01-31T05:01:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.936+0000 7fea93400640 1 -- 172.21.15.97:0/1593774239 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c030d50 con 0x7fea9415d590 2024-01-31T05:01:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.935+0000 7f3b1ea00640 1 -- start start 2024-01-31T05:01:12.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.937+0000 7f3b1ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b180ce930 0x7f3b180cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.938+0000 7fea9b600640 1 -- 172.21.15.97:0/1593774239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 msgr2=0x7fea9415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.938+0000 7fea9b600640 1 --2- 172.21.15.97:0/1593774239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415e540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fea940d3170 tx=0x7fea7c02b9c0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.937+0000 7f3b1ea00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1806c710 con 0x7f3b18159790 2024-01-31T05:01:12.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.937+0000 7f3b1ea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1806c9f0 con 0x7f3b180b37f0 2024-01-31T05:01:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.937+0000 7f3b1ea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b180ccd00 con 0x7f3b180ce930 2024-01-31T05:01:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.938+0000 7fea9b600640 1 -- 172.21.15.97:0/1593774239 shutdown_connections 2024-01-31T05:01:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.938+0000 7fea9b600640 1 --2- 172.21.15.97:0/1593774239 >> v2:172.21.15.188:3300/0 conn(0x7fea940cfae0 0x7fea940cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.938+0000 7fea9b600640 1 --2- 172.21.15.97:0/1593774239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415e540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.938+0000 7fea9b600640 1 -- 172.21.15.97:0/1593774239 >> 172.21.15.97:0/1593774239 conn(0x7fea9409c560 msgr2=0x7fea940d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.939+0000 7f3b17400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3b180b37f0 0x7f3b180ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:54964/0 (socket says 172.21.15.97:54964) 2024-01-31T05:01:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.939+0000 7f3b1ca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b180ce930 0x7f3b180cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.939+0000 7fea9b600640 1 -- 172.21.15.97:0/1593774239 shutdown_connections 2024-01-31T05:01:12.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.939+0000 7fea9b600640 1 -- 172.21.15.97:0/1593774239 wait complete. 2024-01-31T05:01:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.939+0000 7f3b1ca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b180ce930 0x7f3b180cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59108/0 (socket says 172.21.15.97:59108) 2024-01-31T05:01:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.939+0000 7f3b1ca00640 1 -- 172.21.15.97:0/3359601709 learned_addr learned my addr 172.21.15.97:0/3359601709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.940+0000 7f3b1ca00640 1 -- 172.21.15.97:0/3359601709 >> v1:172.21.15.188:6789/0 conn(0x7f3b180b37f0 legacy=0x7f3b180ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.940+0000 7f3b1ca00640 1 -- 172.21.15.97:0/3359601709 >> v1:172.21.15.97:6789/0 conn(0x7f3b18159790 legacy=0x7f3b18070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.941+0000 7f3b1ca00640 1 -- 172.21.15.97:0/3359601709 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1806ccd0 con 0x7f3b180ce930 2024-01-31T05:01:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.943+0000 7fea9b600640 1 Processor -- start 2024-01-31T05:01:12.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.943+0000 7fea9b600640 1 -- start start 2024-01-31T05:01:12.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7f3b1ca00640 1 --2- 172.21.15.97:0/3359601709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180ce930 0x7f3b180cd150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009870 tx=0x7f3b0802ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b2314176b6d8b6f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7fea9b600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea940cfae0 0x7fea940cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7fea9b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7fea9b600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea94161080 0x7fea940c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7fea9b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9409eec0 con 0x7fea9415d590 2024-01-31T05:01:12.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7fea9b600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940cd5f0 con 0x7fea94161080 2024-01-31T05:01:12.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.945+0000 7fea9b600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea940cd310 con 0x7fea940cfae0 2024-01-31T05:01:12.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.947+0000 7fea93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.947+0000 7fea93e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43192/0 (socket says 172.21.15.97:43192) 2024-01-31T05:01:12.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.947+0000 7fea93e00640 1 -- 172.21.15.97:0/3838121046 learned_addr learned my addr 172.21.15.97:0/3838121046 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.947+0000 7fea98c00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea940cfae0 0x7fea940cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.947+0000 7fea99600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea94161080 0x7fea940c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.948+0000 7fea93e00640 1 -- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea940cfae0 msgr2=0x7fea940cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.948+0000 7fea93e00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea940cfae0 0x7fea940cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.949+0000 7fea93e00640 1 -- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea94161080 msgr2=0x7fea940c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.949+0000 7fea93e00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea94161080 0x7fea940c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.949+0000 7fea93e00640 1 -- 172.21.15.97:0/3838121046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c0094d0 con 0x7fea9415d590 2024-01-31T05:01:12.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.949+0000 7fea99600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea94161080 0x7fea940c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:12.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.950+0000 7fea93e00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415ac70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea8400d800 tx=0x7fea8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.950+0000 7f3b16a00640 1 -- 172.21.15.97:0/3359601709 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0802f870 con 0x7f3b180ce930 2024-01-31T05:01:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.950+0000 7f3b16a00640 1 -- 172.21.15.97:0/3359601709 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0802f9d0 con 0x7f3b180ce930 2024-01-31T05:01:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.950+0000 7f3b16a00640 1 -- 172.21.15.97:0/3359601709 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0802fda0 con 0x7f3b180ce930 2024-01-31T05:01:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.950+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400b6c0 con 0x7fea9415d590 2024-01-31T05:01:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.951+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8400b820 con 0x7fea9415d590 2024-01-31T05:01:12.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.951+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400b6c0 con 0x7fea9415d590 2024-01-31T05:01:12.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.951+0000 7f3b1ea00640 1 -- 172.21.15.97:0/3359601709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180ce930 msgr2=0x7f3b180cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.951+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9409f0f0 con 0x7fea9415d590 2024-01-31T05:01:12.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.951+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9409f6b0 con 0x7fea9415d590 2024-01-31T05:01:12.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.952+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/3359601709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180ce930 0x7f3b180cd150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009870 tx=0x7f3b0802ba50 comp rx=0 tx=0).stop 2024-01-31T05:01:12.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.952+0000 7f3b1ea00640 1 -- 172.21.15.97:0/3359601709 shutdown_connections 2024-01-31T05:01:12.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.952+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/3359601709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180ce930 0x7f3b180cd150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.952+0000 7f3b1ea00640 1 -- 172.21.15.97:0/3359601709 >> 172.21.15.97:0/3359601709 conn(0x7f3b180b0a10 msgr2=0x7f3b180b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.953+0000 7f3b1ea00640 1 -- 172.21.15.97:0/3359601709 shutdown_connections 2024-01-31T05:01:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.953+0000 7f3b1ea00640 1 -- 172.21.15.97:0/3359601709 wait complete. 2024-01-31T05:01:12.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.953+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea940677c0 con 0x7fea9415d590 2024-01-31T05:01:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.956+0000 7f3b1ea00640 1 Processor -- start 2024-01-31T05:01:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.957+0000 7fe58e400640 1 Processor -- start 2024-01-31T05:01:12.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.957+0000 7fe58e400640 1 -- start start 2024-01-31T05:01:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.957+0000 7f3b1ea00640 1 -- start start 2024-01-31T05:01:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.958+0000 7f3b1ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 0x7f3b1814c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.958+0000 7f3b1ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b180ce930 0x7f3b1809b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.958+0000 7f3b1ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b18159790 0x7f3b18155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.958+0000 7f3b1ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1809c3c0 con 0x7f3b180ce930 2024-01-31T05:01:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.958+0000 7f3b1ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18153a40 con 0x7f3b180b37f0 2024-01-31T05:01:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.958+0000 7f3b1ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1809be00 con 0x7f3b18159790 2024-01-31T05:01:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.959+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fea8400b990 con 0x7fea9415d590 2024-01-31T05:01:12.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.959+0000 7f3b17400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b180ce930 0x7f3b1809b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.959+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 0x7f3b1814c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.959+0000 7f3b17e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 0x7f3b1814c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59128/0 (socket says 172.21.15.97:59128) 2024-01-31T05:01:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7f3b17e00640 1 -- 172.21.15.97:0/1093312482 learned_addr learned my addr 172.21.15.97:0/1093312482 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.959+0000 7fe58e400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe58815e4f0 0x7fe5880cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.959+0000 7fe58e400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe58815e8d0 0x7fe5880ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7fe58e400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880cc770 con 0x7fe5880ce5f0 2024-01-31T05:01:12.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7fe58e400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880cea80 con 0x7fe58815e4f0 2024-01-31T05:01:12.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7fe58e400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880cc490 con 0x7fe58815e8d0 2024-01-31T05:01:12.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7f3b17e00640 1 -- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b18159790 msgr2=0x7f3b18155c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-31T05:01:12.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7f3b17e00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b18159790 0x7f3b18155c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.960+0000 7f3b17e00640 1 -- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b180ce930 msgr2=0x7f3b1809b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7f3b17e00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b180ce930 0x7f3b1809b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe58815e4f0 0x7fe5880cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe58815e4f0 0x7fe5880cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43208/0 (socket says 172.21.15.97:43208) 2024-01-31T05:01:12.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7f3b17e00640 1 -- 172.21.15.97:0/1093312482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080094d0 con 0x7f3b180b37f0 2024-01-31T05:01:12.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 -- 172.21.15.97:0/3099964196 learned_addr learned my addr 172.21.15.97:0/3099964196 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7f3b17400640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b180ce930 0x7f3b1809b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 -- 172.21.15.97:0/3099964196 >> v2:172.21.15.188:3300/0 conn(0x7fe58815e8d0 msgr2=0x7fe5880ce0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-31T05:01:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 --2- 172.21.15.97:0/3099964196 >> v2:172.21.15.188:3300/0 conn(0x7fe58815e8d0 0x7fe5880ce0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 -- 172.21.15.97:0/3099964196 >> v1:172.21.15.188:6789/0 conn(0x7fe5880ce5f0 legacy=0x7fe5880b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.961+0000 7fe587400640 1 -- 172.21.15.97:0/3099964196 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58806c480 con 0x7fe58815e4f0 2024-01-31T05:01:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.962+0000 7fea90c00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea6c06af30 0x7fea6c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.962+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea84089af0 con 0x7fea9415d590 2024-01-31T05:01:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.962+0000 7f3b17e00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 0x7f3b1814c170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c730 tx=0x7f3b1000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.963+0000 7fea98c00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea6c06af30 0x7fea6c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.963+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10003b70 con 0x7f3b180b37f0 2024-01-31T05:01:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.963+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1809c0e0 con 0x7f3b180b37f0 2024-01-31T05:01:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.962+0000 7f30aa200640 1 Processor -- start 2024-01-31T05:01:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.962+0000 7f30aa200640 1 -- start start 2024-01-31T05:01:12.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1814f430 con 0x7f3b180b37f0 2024-01-31T05:01:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10003cd0 con 0x7f3b180b37f0 2024-01-31T05:01:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.965+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000aa50 con 0x7f3b180b37f0 2024-01-31T05:01:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.965+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae8005290 con 0x7f3b180b37f0 2024-01-31T05:01:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.966+0000 7fe587400640 1 --2- 172.21.15.97:0/3099964196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe5880cbb80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe588067470 tx=0x7fe57802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5508155b37455cc5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f30aa200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f309c039440 0x7f309c006a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f30aa200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f309c052bb0 0x7f309c052fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.966+0000 7fea98c00640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea6c06af30 0x7fea6c06d3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea7c036000 tx=0x7fea7c03b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fea840546c0 con 0x7fea9415d590 2024-01-31T05:01:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7fe586000640 1 -- 172.21.15.97:0/3099964196 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578031800 con 0x7fe58815e4f0 2024-01-31T05:01:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7fe586000640 1 -- 172.21.15.97:0/3099964196 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578031960 con 0x7fe58815e4f0 2024-01-31T05:01:12.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7fe586000640 1 -- 172.21.15.97:0/3099964196 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578031cf0 con 0x7fe58815e4f0 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f30aa200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c006fb0 con 0x7f309c04f4f0 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f30aa200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c050290 con 0x7f309c052bb0 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.964+0000 7f30aa200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c0071e0 con 0x7f309c039440 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.966+0000 7f30ab800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f309c052bb0 0x7f309c052fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.966+0000 7f30ab800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f309c052bb0 0x7f309c052fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43216/0 (socket says 172.21.15.97:43216) 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.966+0000 7f30ab800640 1 -- 172.21.15.97:0/4243182423 learned_addr learned my addr 172.21.15.97:0/4243182423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7f30ab800640 1 -- 172.21.15.97:0/4243182423 >> v2:172.21.15.188:3300/0 conn(0x7f309c039440 msgr2=0x7f309c006a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7f30ab800640 1 --2- 172.21.15.97:0/4243182423 >> v2:172.21.15.188:3300/0 conn(0x7f309c039440 0x7f309c006a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7f30ab800640 1 -- 172.21.15.97:0/4243182423 >> v1:172.21.15.188:6789/0 conn(0x7f309c04f4f0 legacy=0x7f309c04f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.967+0000 7f30ab800640 1 -- 172.21.15.97:0/4243182423 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0074c0 con 0x7f309c052bb0 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.968+0000 7fe58e400640 1 -- 172.21.15.97:0/3099964196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 msgr2=0x7fe5880cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.968+0000 7fe58e400640 1 --2- 172.21.15.97:0/3099964196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe5880cbb80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe588067470 tx=0x7fe57802b8a0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.969+0000 7fe58e400640 1 -- 172.21.15.97:0/3099964196 shutdown_connections 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.969+0000 7fe58e400640 1 --2- 172.21.15.97:0/3099964196 >> v2:172.21.15.188:3300/0 conn(0x7fe58815e8d0 0x7fe5880ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.969+0000 7fe58e400640 1 --2- 172.21.15.97:0/3099964196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe5880cbb80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.969+0000 7fe58e400640 1 -- 172.21.15.97:0/3099964196 >> 172.21.15.97:0/3099964196 conn(0x7fe5880b0a10 msgr2=0x7fe5880b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.969+0000 7fe58e400640 1 -- 172.21.15.97:0/3099964196 shutdown_connections 2024-01-31T05:01:12.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.969+0000 7fe58e400640 1 -- 172.21.15.97:0/3099964196 wait complete. 2024-01-31T05:01:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.970+0000 7f30ab800640 1 --2- 172.21.15.97:0/4243182423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c052fc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f30a4079be0 tx=0x7f30a406a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ba0f9be7c80f4e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.970+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3b1000abb0 con 0x7f3b180b37f0 2024-01-31T05:01:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.971+0000 7f30a3400640 1 -- 172.21.15.97:0/4243182423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40a6df0 con 0x7f309c052bb0 2024-01-31T05:01:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.971+0000 7f30a3400640 1 -- 172.21.15.97:0/4243182423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a40ae440 con 0x7f309c052bb0 2024-01-31T05:01:12.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.971+0000 7f30a3400640 1 -- 172.21.15.97:0/4243182423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a40ae870 con 0x7f309c052bb0 2024-01-31T05:01:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.972+0000 7f3b0fe00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3af006aed0 0x7f3af006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.972+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3b1008a1b0 con 0x7f3b180b37f0 2024-01-31T05:01:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.973+0000 7f3b17400640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3af006aed0 0x7f3af006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.973+0000 7fe58e400640 1 Processor -- start 2024-01-31T05:01:12.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.974+0000 7fe58e400640 1 -- start start 2024-01-31T05:01:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.974+0000 7f30aa200640 1 -- 172.21.15.97:0/4243182423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 msgr2=0x7f309c052fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f30aa200640 1 --2- 172.21.15.97:0/4243182423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c052fc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f30a4079be0 tx=0x7f30a406a440 comp rx=0 tx=0).stop 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f30aa200640 1 -- 172.21.15.97:0/4243182423 shutdown_connections 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f30aa200640 1 --2- 172.21.15.97:0/4243182423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c052fc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f30aa200640 1 --2- 172.21.15.97:0/4243182423 >> v2:172.21.15.188:3300/0 conn(0x7f309c039440 0x7f309c006a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f30aa200640 1 -- 172.21.15.97:0/4243182423 >> 172.21.15.97:0/4243182423 conn(0x7f309c054c90 msgr2=0x7f309c0550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f9269400640 1 Processor -- start 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7fe58e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5880ce5f0 0x7fe58809af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe58e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe588149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.975+0000 7f30aa200640 1 -- 172.21.15.97:0/4243182423 shutdown_connections 2024-01-31T05:01:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe58e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe58815e8d0 0x7fe588154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe58e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58814b210 con 0x7fe58815e4f0 2024-01-31T05:01:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe58e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58814b6a0 con 0x7fe58815e8d0 2024-01-31T05:01:12.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe58e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58814a600 con 0x7fe5880ce5f0 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7f30aa200640 1 -- 172.21.15.97:0/4243182423 wait complete. 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe586a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe588149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe586a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe588149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43222/0 (socket says 172.21.15.97:43222) 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe586a00640 1 -- 172.21.15.97:0/2255256453 learned_addr learned my addr 172.21.15.97:0/2255256453 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7fe587e00640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe58815e8d0 0x7fe588154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7fe587400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5880ce5f0 0x7fe58809af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7f9269400640 1 -- start start 2024-01-31T05:01:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7fe586a00640 1 -- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5880ce5f0 msgr2=0x7fe58809af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7fe586a00640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5880ce5f0 0x7fe58809af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7fe586a00640 1 -- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe58815e8d0 msgr2=0x7fe588154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7fe586a00640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe58815e8d0 0x7fe588154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.977+0000 7fe586a00640 1 -- 172.21.15.97:0/2255256453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780094d0 con 0x7fe58815e4f0 2024-01-31T05:01:12.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.976+0000 7f3b17400640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3af006aed0 0x7f3af006d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b180b3460 tx=0x7f3b00000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.978+0000 7fe586a00640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe588149d20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe57000b6d0 tx=0x7fe57000bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.978+0000 7f30aa200640 1 Processor -- start 2024-01-31T05:01:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.979+0000 7f9269400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f926406b580 0x7f92640d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.979+0000 7f9269400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b960 0x7f926406c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.979+0000 7f9269400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92640d0d00 con 0x7f926406d0e0 2024-01-31T05:01:12.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.979+0000 7f9269400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92640d1810 con 0x7f926406b580 2024-01-31T05:01:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.979+0000 7f9269400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926406c9e0 con 0x7f926406b960 2024-01-31T05:01:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.979+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57000c720 con 0x7fe58815e4f0 2024-01-31T05:01:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.978+0000 7f30aa200640 1 -- start start 2024-01-31T05:01:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f30aa200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 0x7f309c0d5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f30aa200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f309c04f4f0 0x7f309c049bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f30aa200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c046650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57000c880 con 0x7fe58815e4f0 2024-01-31T05:01:12.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57000cc30 con 0x7fe58815e4f0 2024-01-31T05:01:12.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30a8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 0x7f309c0d5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30a8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 0x7f309c0d5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59160/0 (socket says 172.21.15.97:59160) 2024-01-31T05:01:12.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30a3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f309c04f4f0 0x7f309c049bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f9262800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f926406b580 0x7f92640d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f9262800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f926406b580 0x7f92640d1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43232/0 (socket says 172.21.15.97:43232) 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f9262800640 1 -- 172.21.15.97:0/1619310294 learned_addr learned my addr 172.21.15.97:0/1619310294 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7f9263200640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b960 0x7f926406c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30a8e00640 1 -- 172.21.15.97:0/1661146498 learned_addr learned my addr 172.21.15.97:0/1661146498 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30aa200640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c04b280 con 0x7f309c052bb0 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30aa200640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c04a670 con 0x7f309c039440 2024-01-31T05:01:12.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f30aa200640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c04a950 con 0x7f309c04f4f0 2024-01-31T05:01:12.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f9262800640 1 -- 172.21.15.97:0/1619310294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b960 msgr2=0x7f926406c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:12.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f9262800640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b960 0x7f926406c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f9262800640 1 -- 172.21.15.97:0/1619310294 >> v1:172.21.15.188:6789/0 conn(0x7f926406d0e0 legacy=0x7f92640cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.981+0000 7f9262800640 1 -- 172.21.15.97:0/1619310294 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926406bee0 con 0x7f926406b580 2024-01-31T05:01:12.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.982+0000 7f9263200640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b960 0x7f926406c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.982+0000 7f30ab800640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c046650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.982+0000 7f30a3e00640 1 -- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 msgr2=0x7f309c0d5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.982+0000 7f30a3e00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 0x7f309c0d5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.982+0000 7f30a3e00640 1 -- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 msgr2=0x7f309c046650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7f30a3e00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c046650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7f30a3e00640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c04ac30 con 0x7f309c04f4f0 2024-01-31T05:01:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.980+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58814a8e0 con 0x7fe58815e4f0 2024-01-31T05:01:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.982+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58814aea0 con 0x7fe58815e4f0 2024-01-31T05:01:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7f9262800640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b580 0x7f92640d1c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f924c009870 tx=0x7f924c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ea3aec2fdd8014 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58814b3c0 con 0x7fe58815e4f0 2024-01-31T05:01:12.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7f30a8e00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 0x7f309c0d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:12.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7f30ab800640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c046650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:12.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.983+0000 7f9261e00640 1 -- 172.21.15.97:0/1619310294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c031640 con 0x7f926406b580 2024-01-31T05:01:12.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.984+0000 7f9261e00640 1 -- 172.21.15.97:0/1619310294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c0317a0 con 0x7f926406b580 2024-01-31T05:01:12.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.984+0000 7f9261e00640 1 -- 172.21.15.97:0/1619310294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c031640 con 0x7f926406b580 2024-01-31T05:01:12.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.985+0000 7f30a3e00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f309c04f4f0 0x7f309c049bb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f308c009fd0 tx=0x7f308c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.984+0000 7f9269400640 1 -- 172.21.15.97:0/1619310294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b580 msgr2=0x7f92640d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:12.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.984+0000 7f9269400640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b580 0x7f92640d1c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f924c009870 tx=0x7f924c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.985+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3b10054cf0 con 0x7f3b180b37f0 2024-01-31T05:01:12.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.985+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011830 con 0x7f309c04f4f0 2024-01-31T05:01:12.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c0119b0 con 0x7f309c04f4f0 2024-01-31T05:01:12.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f30aa200640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c04af10 con 0x7f309c04f4f0 2024-01-31T05:01:12.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f30aa200640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c0e2ad0 con 0x7f309c04f4f0 2024-01-31T05:01:12.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.987+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe57001d3b0 con 0x7fe58815e4f0 2024-01-31T05:01:12.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f9269400640 1 -- 172.21.15.97:0/1619310294 shutdown_connections 2024-01-31T05:01:12.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f9269400640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b960 0x7f926406c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f9269400640 1 --2- 172.21.15.97:0/1619310294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b580 0x7f92640d1c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:12.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.986+0000 7f9269400640 1 -- 172.21.15.97:0/1619310294 >> 172.21.15.97:0/1619310294 conn(0x7f926409c480 msgr2=0x7f92640d3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:12.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.987+0000 7f9269400640 1 -- 172.21.15.97:0/1619310294 shutdown_connections 2024-01-31T05:01:12.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.987+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011d60 con 0x7f309c04f4f0 2024-01-31T05:01:12.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.989+0000 7fe57f400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe56006afa0 0x7fe56006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.989+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe5700895a0 con 0x7fe58815e4f0 2024-01-31T05:01:12.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.992+0000 7fe587400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe56006afa0 0x7fe56006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.990+0000 7f9269400640 1 -- 172.21.15.97:0/1619310294 wait complete. 2024-01-31T05:01:12.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.991+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c005290 con 0x7f309c04f4f0 2024-01-31T05:01:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.994+0000 7f9269400640 1 Processor -- start 2024-01-31T05:01:12.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.994+0000 7fe587400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe56006afa0 0x7fe56006d460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe578036000 tx=0x7fe57803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:12.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.995+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe570054210 con 0x7fe58815e4f0 2024-01-31T05:01:12.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.995+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f308c01a400 con 0x7f309c04f4f0 2024-01-31T05:01:12.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.996+0000 7f6cf7000640 1 Processor -- start 2024-01-31T05:01:12.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f30a0c00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307c06aed0 0x7f307c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:12.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f308c0899f0 con 0x7f309c04f4f0 2024-01-31T05:01:12.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f30a8e00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307c06aed0 0x7f307c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:12.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f30a8e00640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307c06aed0 0x7f307c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f309c04ea70 tx=0x7f3094009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.996+0000 7f9269400640 1 -- start start 2024-01-31T05:01:13.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f9269400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 0x7f926415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f9269400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b960 0x7f92640b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f9269400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926406d0e0 0x7f92640c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f9269400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92640ccac0 con 0x7f926406b960 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f9269400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926415f400 con 0x7f926406d0e0 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.997+0000 7f9269400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9264203a70 con 0x7f926406b580 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f9262800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 0x7f926415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f9262800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 0x7f926415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59866/0 (socket says 172.21.15.97:59866) 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f9262800640 1 -- 172.21.15.97:0/257580873 learned_addr learned my addr 172.21.15.97:0/257580873 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f9262800640 1 -- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926406d0e0 msgr2=0x7f92640c8c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f9262800640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926406d0e0 0x7f92640c8c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f9262800640 1 -- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b960 msgr2=0x7f92640b80e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f9262800640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b960 0x7f92640b80e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f9262800640 1 -- 172.21.15.97:0/257580873 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0094d0 con 0x7f926406b580 2024-01-31T05:01:13.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.001+0000 7f9262800640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 0x7f926415f890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f924c02b930 tx=0x7f924c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.996+0000 7f6cf7000640 1 -- start start 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f6cf7000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cf015d590 0x7f6cf015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f6cf7000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6cf00cff20 0x7f6cf0161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f6cf7000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf00cded0 con 0x7f6cf00cfae0 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f6cf7000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf015d970 con 0x7f6cf015d590 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.998+0000 7f6cf7000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00cf210 con 0x7f6cf00cff20 2024-01-31T05:01:13.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f6cf5000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6cf00cff20 0x7f6cf0161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f6cefe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cf015d590 0x7f6cf015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f6cefe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cf015d590 0x7f6cf015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43238/0 (socket says 172.21.15.97:43238) 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:12.999+0000 7f6cefe00640 1 -- 172.21.15.97:0/2598296512 learned_addr learned my addr 172.21.15.97:0/2598296512 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.000+0000 7f6cefe00640 1 -- 172.21.15.97:0/2598296512 >> v2:172.21.15.188:3300/0 conn(0x7f6cf00cff20 msgr2=0x7f6cf0161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.000+0000 7f6cefe00640 1 --2- 172.21.15.97:0/2598296512 >> v2:172.21.15.188:3300/0 conn(0x7f6cf00cff20 0x7f6cf0161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.000+0000 7f6cefe00640 1 -- 172.21.15.97:0/2598296512 >> v1:172.21.15.188:6789/0 conn(0x7f6cf00cfae0 legacy=0x7f6cf0161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.000+0000 7f6cefe00640 1 -- 172.21.15.97:0/2598296512 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00ce170 con 0x7f6cf015d590 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.002+0000 7f6cefe00640 1 --2- 172.21.15.97:0/2598296512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf015d590 0x7f6cf015e540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00cdbd0 tx=0x7f6ce002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ae48f9ffbcc1f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.003+0000 7f6ceea00640 1 -- 172.21.15.97:0/2598296512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce002c7d0 con 0x7f6cf015d590 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.003+0000 7f6ceea00640 1 -- 172.21.15.97:0/2598296512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce002c930 con 0x7f6cf015d590 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.003+0000 7f6ceea00640 1 -- 172.21.15.97:0/2598296512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce002cd00 con 0x7f6cf015d590 2024-01-31T05:01:13.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 -- 172.21.15.97:0/2598296512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf015d590 msgr2=0x7f6cf015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2598296512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf015d590 0x7f6cf015e540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00cdbd0 tx=0x7f6ce002bc30 comp rx=0 tx=0).stop 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 -- 172.21.15.97:0/2598296512 shutdown_connections 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2598296512 >> v2:172.21.15.188:3300/0 conn(0x7f6cf00cff20 0x7f6cf0161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2598296512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf015d590 0x7f6cf015e540 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 -- 172.21.15.97:0/2598296512 >> 172.21.15.97:0/2598296512 conn(0x7f6cf009c560 msgr2=0x7f6cf00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 -- 172.21.15.97:0/2598296512 shutdown_connections 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.004+0000 7f6cf7000640 1 -- 172.21.15.97:0/2598296512 wait complete. 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.007+0000 7f6cf7000640 1 Processor -- start 2024-01-31T05:01:13.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.007+0000 7f6cf7000640 1 -- start start 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cf7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 0x7f6cf009dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cf7000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cf00cff20 0x7f6cf00d1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cf7000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cf015d590 0x7f6cf00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cf7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf009d910 con 0x7f6cf00cfae0 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cf7000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00cd560 con 0x7f6cf015d590 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cf7000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf009cf70 con 0x7f6cf00cff20 2024-01-31T05:01:13.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.009+0000 7f6cefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 0x7f6cf009dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.010+0000 7f6cefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 0x7f6cf009dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43244/0 (socket says 172.21.15.97:43244) 2024-01-31T05:01:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.010+0000 7f6cefe00640 1 -- 172.21.15.97:0/2283093900 learned_addr learned my addr 172.21.15.97:0/2283093900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:13.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.010+0000 7f6cf5000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cf015d590 0x7f6cf00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.010+0000 7f6cef400640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cf00cff20 0x7f6cf00d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.011+0000 7f6cefe00640 1 -- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cf00cff20 msgr2=0x7f6cf00d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.011+0000 7f6cefe00640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cf00cff20 0x7f6cf00d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.011+0000 7f6cefe00640 1 -- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cf015d590 msgr2=0x7f6cf00cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.011+0000 7f6cefe00640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cf015d590 0x7f6cf00cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.011+0000 7f6cefe00640 1 -- 172.21.15.97:0/2283093900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00094d0 con 0x7f6cf00cfae0 2024-01-31T05:01:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.011+0000 7f6cf5000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cf015d590 0x7f6cf00cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:13.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f6cefe00640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 0x7f6cf009dd30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00090b0 tx=0x7f6ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0034c80 con 0x7f6cf00cfae0 2024-01-31T05:01:13.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0034de0 con 0x7f6cf00cfae0 2024-01-31T05:01:13.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c003810 con 0x7f926406b580 2024-01-31T05:01:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c03f070 con 0x7f926406b580 2024-01-31T05:01:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c003b70 con 0x7f926406b580 2024-01-31T05:01:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9264203d50 con 0x7f926406b580 2024-01-31T05:01:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.012+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce003f620 con 0x7f6cf00cfae0 2024-01-31T05:01:13.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.013+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926415f090 con 0x7f926406b580 2024-01-31T05:01:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.013+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf009d250 con 0x7f6cf00cfae0 2024-01-31T05:01:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.013+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf00d1570 con 0x7f6cf00cfae0 2024-01-31T05:01:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.014+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f308c054530 con 0x7f309c04f4f0 2024-01-31T05:01:13.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.015+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9264067dc0 con 0x7f926406b580 2024-01-31T05:01:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.017+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00677c0 con 0x7f6cf00cfae0 2024-01-31T05:01:13.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.019+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6ce003b040 con 0x7f6cf00cfae0 2024-01-31T05:01:13.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.019+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f924c02f050 con 0x7f926406b580 2024-01-31T05:01:13.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.023+0000 7f925aa00640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f923c06aed0 0x7f923c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.027+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f924c041070 con 0x7f926406b580 2024-01-31T05:01:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.020+0000 7f6ce7e00640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6cc806afa0 0x7f6cc806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.020+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ce00abbc0 con 0x7f6cf00cfae0 2024-01-31T05:01:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.021+0000 7f6cef400640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6cc806afa0 0x7f6cc806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.023+0000 7f6cef400640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6cc806afa0 0x7f6cc806d460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0071e10 tx=0x7f6cd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.028+0000 7fca87600640 1 Processor -- start 2024-01-31T05:01:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.028+0000 7fca87600640 1 -- start start 2024-01-31T05:01:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.030+0000 7fca87600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca800ce930 0x7fca800cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.031+0000 7fca87600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8006c710 con 0x7fca80159790 2024-01-31T05:01:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.031+0000 7fca87600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8006c9f0 con 0x7fca800b37f0 2024-01-31T05:01:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.031+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6ce00769e0 con 0x7f6cf00cfae0 2024-01-31T05:01:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.031+0000 7fca87600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca800ccd00 con 0x7fca800ce930 2024-01-31T05:01:13.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.031+0000 7fca7fe00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fca800b37f0 0x7fca800ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:55000/0 (socket says 172.21.15.97:55000) 2024-01-31T05:01:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.028+0000 7f925be00640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f923c06aed0 0x7f923c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.031+0000 7fca7fe00640 1 -- 172.21.15.97:0/1287144808 learned_addr learned my addr 172.21.15.97:0/1287144808 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.032+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711171968 0 0) 0x7fca8006c9f0 con 0x7fca800b37f0 2024-01-31T05:01:13.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.032+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca60003520 con 0x7fca800b37f0 2024-01-31T05:01:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.032+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f924c077070 con 0x7f926406b580 2024-01-31T05:01:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7f925be00640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f923c06aed0 0x7f923c06d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92641e9080 tx=0x7f9254007950 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca85600640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca800ce930 0x7fca800cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981325063 0 0) 0x7fca8006c710 con 0x7fca80159790 2024-01-31T05:01:13.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8006c9f0 con 0x7fca80159790 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2351433368 0 0) 0x7fca60003520 con 0x7fca800b37f0 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca800ce930 msgr2=0x7fca800cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca800ce930 0x7fca800cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 >> v1:172.21.15.97:6789/0 conn(0x7fca80159790 legacy=0x7fca80070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8006ccd0 con 0x7fca800b37f0 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fca70001670 con 0x7fca800b37f0 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.033+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 >> v1:172.21.15.188:6789/0 conn(0x7fca800b37f0 legacy=0x7fca800ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca7f400640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca60003970 0x7fca60003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca7f400640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca60004390 0x7fca6000d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca7f400640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6000d950 0x7fca6000fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800ccd00 con 0x7fca60003970 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca600103e0 con 0x7fca60004390 2024-01-31T05:01:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca600105a0 con 0x7fca6000d950 2024-01-31T05:01:13.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.034+0000 7fca84c00640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca60003970 0x7fca60003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.036+0000 7fca84c00640 1 -- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6000d950 msgr2=0x7fca6000fd40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-31T05:01:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.036+0000 7fca84c00640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6000d950 0x7fca6000fd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.036+0000 7fca84c00640 1 -- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca60004390 msgr2=0x7fca6000d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.036+0000 7fca84c00640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca60004390 0x7fca6000d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.036+0000 7fca84c00640 1 -- 172.21.15.97:0/1287144808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8006ccd0 con 0x7fca60003970 2024-01-31T05:01:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.038+0000 7fca84c00640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca60003970 0x7fca60003d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca680018c0 tx=0x7fca600047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db96dcbaac92f0ca server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.039+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68005330 con 0x7fca60003970 2024-01-31T05:01:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.039+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68005490 con 0x7fca60003970 2024-01-31T05:01:13.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.039+0000 7fca7f400640 1 -- 172.21.15.97:0/1287144808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68003830 con 0x7fca60003970 2024-01-31T05:01:13.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.040+0000 7fca87600640 1 -- 172.21.15.97:0/1287144808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca60003970 msgr2=0x7fca60003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.040+0000 7fca87600640 1 --2- 172.21.15.97:0/1287144808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca60003970 0x7fca60003d70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca680018c0 tx=0x7fca600047e0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.041+0000 7fca85600640 1 -- 172.21.15.97:0/1287144808 reap_dead start 2024-01-31T05:01:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.041+0000 7fca87600640 1 -- 172.21.15.97:0/1287144808 shutdown_connections 2024-01-31T05:01:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.041+0000 7fca87600640 1 -- 172.21.15.97:0/1287144808 >> 172.21.15.97:0/1287144808 conn(0x7fca800b0a10 msgr2=0x7fca800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.044+0000 7fca87600640 1 -- 172.21.15.97:0/1287144808 shutdown_connections 2024-01-31T05:01:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.044+0000 7fca87600640 1 -- 172.21.15.97:0/1287144808 wait complete. 2024-01-31T05:01:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.048+0000 7fca87600640 1 Processor -- start 2024-01-31T05:01:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.048+0000 7fca87600640 1 -- start start 2024-01-31T05:01:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.050+0000 7fca87600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca801f2190 0x7fca8009a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca87600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8009a910 0x7fca80071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca87600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca80072240 0x7fca80159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca87600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca800726d0 con 0x7fca80072240 2024-01-31T05:01:13.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca87600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8009ae90 con 0x7fca8009a910 2024-01-31T05:01:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca87600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8009b170 con 0x7fca801f2190 2024-01-31T05:01:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca84c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca801f2190 0x7fca8009a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca84c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca801f2190 0x7fca8009a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59896/0 (socket says 172.21.15.97:59896) 2024-01-31T05:01:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca84c00640 1 -- 172.21.15.97:0/1679395944 learned_addr learned my addr 172.21.15.97:0/1679395944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.054+0000 7fca7fe00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8009a910 0x7fca80071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.055+0000 7fca85600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca80072240 0x7fca80159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.055+0000 7fca7fe00640 1 -- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca801f2190 msgr2=0x7fca8009a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.055+0000 7fca7fe00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca801f2190 0x7fca8009a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.055+0000 7fca7fe00640 1 -- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca80072240 msgr2=0x7fca80159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.055+0000 7fca7fe00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca80072240 0x7fca80159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.055+0000 7fca7fe00640 1 -- 172.21.15.97:0/1679395944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8009b450 con 0x7fca8009a910 2024-01-31T05:01:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.056+0000 7fca85600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca80072240 0x7fca80159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.056+0000 7fca7fe00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8009a910 0x7fca80071d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca7000cac0 tx=0x7fca7000cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.058+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70004cb0 con 0x7fca8009a910 2024-01-31T05:01:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.058+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70004e10 con 0x7fca8009a910 2024-01-31T05:01:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.058+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca700072f0 con 0x7fca8009a910 2024-01-31T05:01:13.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.059+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8009b730 con 0x7fca8009a910 2024-01-31T05:01:13.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.059+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8015bb70 con 0x7fca8009a910 2024-01-31T05:01:13.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.061+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca800b2980 con 0x7fca8009a910 2024-01-31T05:01:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.066+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fca70007450 con 0x7fca8009a910 2024-01-31T05:01:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.068+0000 7fca7cc00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca5806afa0 0x7fca5806d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.068+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fca70089190 con 0x7fca8009a910 2024-01-31T05:01:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.069+0000 7fca84c00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca5806afa0 0x7fca5806d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.071+0000 7fca84c00640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca5806afa0 0x7fca5806d460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca8015b900 tx=0x7fca68009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.080+0000 7eff8be00640 1 Processor -- start 2024-01-31T05:01:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.080+0000 7eff8be00640 1 -- start start 2024-01-31T05:01:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.082+0000 7eff8be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff8c0cfad0 0x7eff8c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.082+0000 7eff8be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7eff8c0cfeb0 0x7eff8c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.082+0000 7eff8be00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8c0c6bd0 con 0x7eff8c0b41f0 2024-01-31T05:01:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.083+0000 7eff8be00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0d1fd0 con 0x7eff8c0cfad0 2024-01-31T05:01:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.083+0000 7eff8be00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8c0d3fe0 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.083+0000 7eff8a000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7eff8c0b41f0 0x7eff8c0b3540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41778/0 (socket says 172.21.15.97:41778) 2024-01-31T05:01:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.083+0000 7eff8a000640 1 -- 172.21.15.97:0/3700754477 learned_addr learned my addr 172.21.15.97:0/3700754477 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.083+0000 7eff8aa00640 1 --2- 172.21.15.97:0/3700754477 >> v2:172.21.15.97:3300/0 conn(0x7eff8c0cfad0 0x7eff8c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff8b400640 1 --2- 172.21.15.97:0/3700754477 >> v2:172.21.15.188:3300/0 conn(0x7eff8c0cfeb0 0x7eff8c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff89600640 1 -- 172.21.15.97:0/3700754477 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335089872 0 0) 0x7eff8c0c6bd0 con 0x7eff8c0b41f0 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff89600640 1 -- 172.21.15.97:0/3700754477 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff6c003520 con 0x7eff8c0b41f0 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff8aa00640 1 -- 172.21.15.97:0/3700754477 >> v2:172.21.15.188:3300/0 conn(0x7eff8c0cfeb0 msgr2=0x7eff8c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff8aa00640 1 --2- 172.21.15.97:0/3700754477 >> v2:172.21.15.188:3300/0 conn(0x7eff8c0cfeb0 0x7eff8c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff8aa00640 1 -- 172.21.15.97:0/3700754477 >> v1:172.21.15.97:6789/0 conn(0x7eff8c0b41f0 legacy=0x7eff8c0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.084+0000 7eff8aa00640 1 -- 172.21.15.97:0/3700754477 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0d3d40 con 0x7eff8c0cfad0 2024-01-31T05:01:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.086+0000 7eff8aa00640 1 --2- 172.21.15.97:0/3700754477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0cfad0 0x7eff8c0b3000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff8c08b930 tx=0x7eff7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6845b3dbd2b5a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.087+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fca70053e00 con 0x7fca8009a910 2024-01-31T05:01:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.090+0000 7eff89600640 1 -- 172.21.15.97:0/3700754477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74031640 con 0x7eff8c0cfad0 2024-01-31T05:01:13.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.090+0000 7eff89600640 1 -- 172.21.15.97:0/3700754477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff740317a0 con 0x7eff8c0cfad0 2024-01-31T05:01:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.091+0000 7eff8be00640 1 -- 172.21.15.97:0/3700754477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0cfad0 msgr2=0x7eff8c0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.091+0000 7eff8be00640 1 --2- 172.21.15.97:0/3700754477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0cfad0 0x7eff8c0b3000 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff8c08b930 tx=0x7eff7402b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.091+0000 7eff8be00640 1 -- 172.21.15.97:0/3700754477 shutdown_connections 2024-01-31T05:01:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.091+0000 7eff8be00640 1 --2- 172.21.15.97:0/3700754477 >> v2:172.21.15.188:3300/0 conn(0x7eff8c0cfeb0 0x7eff8c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.091+0000 7eff8be00640 1 --2- 172.21.15.97:0/3700754477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0cfad0 0x7eff8c0b3000 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff8c08b930 tx=0x7eff7402b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.091+0000 7eff8be00640 1 -- 172.21.15.97:0/3700754477 >> 172.21.15.97:0/3700754477 conn(0x7eff8c0d3080 msgr2=0x7eff8c0d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.095+0000 7eff8be00640 1 -- 172.21.15.97:0/3700754477 shutdown_connections 2024-01-31T05:01:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.099+0000 7eff8be00640 1 -- 172.21.15.97:0/3700754477 wait complete. 2024-01-31T05:01:13.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.102+0000 7eff8be00640 1 Processor -- start 2024-01-31T05:01:13.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.103+0000 7eff8be00640 1 -- start start 2024-01-31T05:01:13.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.108+0000 7eff8be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0b41f0 0x7eff8c1f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.109+0000 7eff8be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 0x7eff8c09e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.109+0000 7eff8be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8c0cbbb0 0x7eff8c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.109+0000 7eff8be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c1f2de0 con 0x7eff8c0b41f0 2024-01-31T05:01:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.109+0000 7eff8a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 0x7eff8c09e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.110+0000 7eff8a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 0x7eff8c09e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59910/0 (socket says 172.21.15.97:59910) 2024-01-31T05:01:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.110+0000 7eff8a000640 1 -- 172.21.15.97:0/1340495482 learned_addr learned my addr 172.21.15.97:0/1340495482 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.110+0000 7eff8b400640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8c0cbbb0 0x7eff8c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.110+0000 7eff8aa00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0b41f0 0x7eff8c1f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.110+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c09e880 con 0x7eff8c0cbbb0 2024-01-31T05:01:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.110+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0b8a90 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.111+0000 7eff8a000640 1 -- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8c0cbbb0 msgr2=0x7eff8c0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.111+0000 7eff8a000640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8c0cbbb0 0x7eff8c0b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.111+0000 7eff8a000640 1 -- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0b41f0 msgr2=0x7eff8c1f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.111+0000 7eff8a000640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0b41f0 0x7eff8c1f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.112+0000 7eff8a000640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74009400 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.112+0000 7eff8aa00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0b41f0 0x7eff8c1f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:13.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.112+0000 7eff8b400640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8c0cbbb0 0x7eff8c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:13.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.113+0000 7eff8a000640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 0x7eff8c09e340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c480 tx=0x7eff7c00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.114+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0074e0 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.114+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c007640 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.114+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c004a00 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.117+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c0b84d0 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.117+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c0b5d40 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.118+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c0b5bb0 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.124+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7eff7c004b60 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.127+0000 7eff82a00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7eff6406af00 0x7eff6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:13.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.127+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7eff7c088e70 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.128+0000 7eff8aa00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7eff6406af00 0x7eff6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:13.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.131+0000 7eff8aa00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7eff6406af00 0x7eff6406d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff74036000 tx=0x7eff7403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:13.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.138+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7eff7c053bb0 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.388+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea9415ff10 con 0x7fea9415d590 2024-01-31T05:01:13.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.390+0000 7fea90c00640 1 -- 172.21.15.97:0/3838121046 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea84057ce0 con 0x7fea9415d590 2024-01-31T05:01:13.392 INFO:teuthology.orchestra.run.smithi097.stdout:64424509447 2024-01-31T05:01:13.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.393+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea6c06af30 msgr2=0x7fea6c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.393+0000 7fea9b600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea6c06af30 0x7fea6c06d3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea7c036000 tx=0x7fea7c03b040 comp rx=0 tx=0).stop 2024-01-31T05:01:13.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.393+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 msgr2=0x7fea9415ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.393+0000 7fea9b600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415ac70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fea8400d800 tx=0x7fea8400dcc0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 shutdown_connections 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea94161080 0x7fea940c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea6c06af30 0x7fea6c06d3f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9415d590 0x7fea9415ac70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 --2- 172.21.15.97:0/3838121046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea940cfae0 0x7fea940cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 >> 172.21.15.97:0/3838121046 conn(0x7fea9409c560 msgr2=0x7fea940b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 shutdown_connections 2024-01-31T05:01:13.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.394+0000 7fea9b600640 1 -- 172.21.15.97:0/3838121046 wait complete. 2024-01-31T05:01:13.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.1 2024-01-31T05:01:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.434+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ae8005810 con 0x7f3b180b37f0 2024-01-31T05:01:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.435+0000 7f3b0fe00640 1 -- 172.21.15.97:0/1093312482 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b1000ae60 con 0x7f3b180b37f0 2024-01-31T05:01:13.435 INFO:teuthology.orchestra.run.smithi097.stdout:60129542152 2024-01-31T05:01:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.436+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f306c005540 con 0x7f309c04f4f0 2024-01-31T05:01:13.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.437+0000 7f30a0c00640 1 -- 172.21.15.97:0/1661146498 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f308c057b50 con 0x7f309c04f4f0 2024-01-31T05:01:13.436 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:13.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.438+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3af006aed0 msgr2=0x7f3af006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.438+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3af006aed0 0x7f3af006d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b180b3460 tx=0x7f3b00000ed0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.438+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 msgr2=0x7f3b1814c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.438+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 0x7f3b1814c170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c730 tx=0x7f3b1000cbf0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 shutdown_connections 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b18159790 0x7f3b18155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3af006aed0 0x7f3af006d390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b180ce930 0x7f3b1809b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 --2- 172.21.15.97:0/1093312482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b180b37f0 0x7f3b1814c170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 >> 172.21.15.97:0/1093312482 conn(0x7f3b180b0a10 msgr2=0x7f3b18070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 shutdown_connections 2024-01-31T05:01:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.439+0000 7f3b1ea00640 1 -- 172.21.15.97:0/1093312482 wait complete. 2024-01-31T05:01:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307c06aed0 msgr2=0x7f307c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307c06aed0 0x7f307c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f309c04ea70 tx=0x7f3094009040 comp rx=0 tx=0).stop 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f309c04f4f0 msgr2=0x7f309c049bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f309c04f4f0 0x7f309c049bb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f308c009fd0 tx=0x7f308c00b870 comp rx=0 tx=0).stop 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 shutdown_connections 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307c06aed0 0x7f307c06d390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f309c052bb0 0x7f309c046650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f309c04f4f0 0x7f309c049bb0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f308c009fd0 tx=0x7f308c00b870 comp rx=0 tx=0).stop 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 --2- 172.21.15.97:0/1661146498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309c039440 0x7f309c0d5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.440+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 >> 172.21.15.97:0/1661146498 conn(0x7f309c054c90 msgr2=0x7f309c0487c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.441+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 shutdown_connections 2024-01-31T05:01:13.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.441+0000 7f309a000640 1 -- 172.21.15.97:0/1661146498 wait complete. 2024-01-31T05:01:13.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.445+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5880b1500 con 0x7fe58815e4f0 2024-01-31T05:01:13.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.446+0000 7fe57f400640 1 -- 172.21.15.97:0/2255256453 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe570019020 con 0x7fe58815e4f0 2024-01-31T05:01:13.445 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.448+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe56006afa0 msgr2=0x7fe56006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.448+0000 7fe58e400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe56006afa0 0x7fe56006d460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe578036000 tx=0x7fe57803c040 comp rx=0 tx=0).stop 2024-01-31T05:01:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.448+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 msgr2=0x7fe588149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.448+0000 7fe58e400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe588149d20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe57000b6d0 tx=0x7fe57000bb90 comp rx=0 tx=0).stop 2024-01-31T05:01:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 shutdown_connections 2024-01-31T05:01:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe58815e8d0 0x7fe588154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe56006afa0 0x7fe56006d460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe58815e4f0 0x7fe588149d20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 --2- 172.21.15.97:0/2255256453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5880ce5f0 0x7fe58809af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 >> 172.21.15.97:0/2255256453 conn(0x7fe5880b0a10 msgr2=0x7fe5880cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 shutdown_connections 2024-01-31T05:01:13.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.449+0000 7fe58e400640 1 -- 172.21.15.97:0/2255256453 wait complete. 2024-01-31T05:01:13.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-01-31T05:01:13.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.0 2024-01-31T05:01:13.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2024-01-31T05:01:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.480+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f92640b33f0 con 0x7f926406b580 2024-01-31T05:01:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.481+0000 7f925aa00640 1 -- 172.21.15.97:0/257580873 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f924c07a690 con 0x7f926406b580 2024-01-31T05:01:13.481 INFO:teuthology.orchestra.run.smithi097.stdout:64424509447 2024-01-31T05:01:13.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.484+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f923c06aed0 msgr2=0x7f923c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.484+0000 7f9269400640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f923c06aed0 0x7f923c06d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92641e9080 tx=0x7f9254007950 comp rx=0 tx=0).stop 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.484+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 msgr2=0x7f926415f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.484+0000 7f9269400640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 0x7f926415f890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f924c02b930 tx=0x7f924c0046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 shutdown_connections 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926406d0e0 0x7f92640c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f923c06aed0 0x7f923c06d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f926406b960 0x7f92640b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 --2- 172.21.15.97:0/257580873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f926406b580 0x7f926415f890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 >> 172.21.15.97:0/257580873 conn(0x7f926409c480 msgr2=0x7f92640b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 shutdown_connections 2024-01-31T05:01:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.485+0000 7f9269400640 1 -- 172.21.15.97:0/257580873 wait complete. 2024-01-31T05:01:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.487+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6cf00cf690 con 0x7f6cf00cfae0 2024-01-31T05:01:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.487+0000 7f6ce7e00640 1 -- 172.21.15.97:0/2283093900 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6ce007a000 con 0x7f6cf00cfae0 2024-01-31T05:01:13.486 INFO:teuthology.orchestra.run.smithi097.stdout:60129542153 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6cc806afa0 msgr2=0x7f6cc806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6cc806afa0 0x7f6cc806d460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0071e10 tx=0x7f6cd8009040 comp rx=0 tx=0).stop 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 msgr2=0x7f6cf009dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 0x7f6cf009dd30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00090b0 tx=0x7f6ce00046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 shutdown_connections 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cf015d590 0x7f6cf00cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cf00cff20 0x7f6cf00d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6cc806afa0 0x7f6cc806d460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 --2- 172.21.15.97:0/2283093900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cf00cfae0 0x7f6cf009dd30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.489+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 >> 172.21.15.97:0/2283093900 conn(0x7f6cf009c560 msgr2=0x7f6cf006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.490+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 shutdown_connections 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.490+0000 7f6cf7000640 1 -- 172.21.15.97:0/2283093900 wait complete. 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.490+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fca800677c0 con 0x7fca8009a910 2024-01-31T05:01:13.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.491+0000 7fca7cc00640 1 -- 172.21.15.97:0/1679395944 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca70018090 con 0x7fca8009a910 2024-01-31T05:01:13.491 INFO:teuthology.orchestra.run.smithi097.stdout:60129542152 2024-01-31T05:01:13.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca5806afa0 msgr2=0x7fca5806d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca5806afa0 0x7fca5806d460 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca8015b900 tx=0x7fca68009030 comp rx=0 tx=0).stop 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8009a910 msgr2=0x7fca80071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8009a910 0x7fca80071d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca7000cac0 tx=0x7fca7000cf80 comp rx=0 tx=0).stop 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 shutdown_connections 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fca5806afa0 0x7fca5806d460 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca80072240 0x7fca80159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca8009a910 0x7fca80071d00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 --2- 172.21.15.97:0/1679395944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca801f2190 0x7fca8009a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 >> 172.21.15.97:0/1679395944 conn(0x7fca800b0a10 msgr2=0x7fca801588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 shutdown_connections 2024-01-31T05:01:13.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.495+0000 7fca87600640 1 -- 172.21.15.97:0/1679395944 wait complete. 2024-01-31T05:01:13.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.6 2024-01-31T05:01:13.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2024-01-31T05:01:13.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.5 2024-01-31T05:01:13.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.543+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7eff8c0676b0 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.544+0000 7eff82a00640 1 -- 172.21.15.97:0/1340495482 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff7c018020 con 0x7eff8c0cfeb0 2024-01-31T05:01:13.544 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7eff6406af00 msgr2=0x7eff6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7eff6406af00 0x7eff6406d3c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff74036000 tx=0x7eff7403c040 comp rx=0 tx=0).stop 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 msgr2=0x7eff8c09e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 0x7eff8c09e340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c480 tx=0x7eff7c00c940 comp rx=0 tx=0).stop 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 shutdown_connections 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff8c0cbbb0 0x7eff8c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff8c0cfeb0 0x7eff8c09e340 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7eff6406af00 0x7eff6406d3c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 --2- 172.21.15.97:0/1340495482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff8c0b41f0 0x7eff8c1f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 >> 172.21.15.97:0/1340495482 conn(0x7eff8c0d3080 msgr2=0x7eff8c160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 shutdown_connections 2024-01-31T05:01:13.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:13.547+0000 7eff8be00640 1 -- 172.21.15.97:0/1340495482 wait complete. 2024-01-31T05:01:13.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2024-01-31T05:01:14.420 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-31T05:01:14.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-31T05:01:14.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-31T05:01:14.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-31T05:01:14.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-31T05:01:14.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-31T05:01:14.512 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-31T05:01:14.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-31T05:01:14.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.698+0000 7f9986200640 1 Processor -- start 2024-01-31T05:01:14.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.699+0000 7f9986200640 1 -- start start 2024-01-31T05:01:14.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.701+0000 7f9986200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99800b1500 0x7f9980159520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.701+0000 7f9986200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998006c9f0 con 0x7f99800708d0 2024-01-31T05:01:14.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.701+0000 7f9986200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998006ccd0 con 0x7f9980070420 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.701+0000 7f9986200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99800ccd00 con 0x7f99800b1500 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.702+0000 7f997ea00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9980070420 0x7f99800b37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:55008/0 (socket says 172.21.15.97:55008) 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.702+0000 7f997ea00640 1 -- 172.21.15.97:0/1613920557 learned_addr learned my addr 172.21.15.97:0/1613920557 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.703+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120740349 0 0) 0x7f998006c9f0 con 0x7f99800708d0 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.703+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9960003520 con 0x7f99800708d0 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.703+0000 7f997f400640 1 --2- 172.21.15.97:0/1613920557 >> v2:172.21.15.188:3300/0 conn(0x7f99800b1500 0x7f9980159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3174503201 0 0) 0x7f9960003520 con 0x7f99800708d0 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 >> v2:172.21.15.188:3300/0 conn(0x7f99800b1500 msgr2=0x7f9980159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 --2- 172.21.15.97:0/1613920557 >> v2:172.21.15.188:3300/0 conn(0x7f99800b1500 0x7f9980159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 >> v1:172.21.15.188:6789/0 conn(0x7f9980070420 legacy=0x7f99800b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998006cfb0 con 0x7f99800708d0 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f9970001670 con 0x7f99800708d0 2024-01-31T05:01:14.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 >> v1:172.21.15.97:6789/0 conn(0x7f99800708d0 legacy=0x7f99800ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9960003970 0x7f9960003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.704+0000 7f997e000640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99600047c0 0x7f9960004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.705+0000 7f997e000640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960005180 0x7f996000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.705+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99800ccd00 con 0x7f9960003970 2024-01-31T05:01:14.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.705+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996000c050 con 0x7f99600047c0 2024-01-31T05:01:14.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.705+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996000c210 con 0x7f9960005180 2024-01-31T05:01:14.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997fe00640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960005180 0x7f996000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997f400640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9960003970 0x7f9960003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997f400640 1 -- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960005180 msgr2=0x7f996000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997f400640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960005180 0x7f996000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997f400640 1 -- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99600047c0 msgr2=0x7f9960004c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-31T05:01:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997f400640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99600047c0 0x7f9960004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.706+0000 7f997f400640 1 -- 172.21.15.97:0/1613920557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960003520 con 0x7f9960003970 2024-01-31T05:01:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.708+0000 7f997fe00640 1 -- 172.21.15.97:0/1613920557 reap_dead start 2024-01-31T05:01:14.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.708+0000 7f997f400640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9960003970 0x7f9960003d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f99600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c41f3f1caefea2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.709+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997802ccf0 con 0x7f9960003970 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.709+0000 7f997e000640 1 -- 172.21.15.97:0/1613920557 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997802ce50 con 0x7f9960003970 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.710+0000 7f9986200640 1 -- 172.21.15.97:0/1613920557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9960003970 msgr2=0x7f9960003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.710+0000 7f9986200640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9960003970 0x7f9960003d50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f99600057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.711+0000 7f9986200640 1 -- 172.21.15.97:0/1613920557 shutdown_connections 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.711+0000 7f9986200640 1 --2- 172.21.15.97:0/1613920557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9960003970 0x7f9960003d50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.711+0000 7f9986200640 1 -- 172.21.15.97:0/1613920557 >> 172.21.15.97:0/1613920557 conn(0x7f99800b0a10 msgr2=0x7f99800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.711+0000 7f9986200640 1 -- 172.21.15.97:0/1613920557 shutdown_connections 2024-01-31T05:01:14.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.711+0000 7f9986200640 1 -- 172.21.15.97:0/1613920557 wait complete. 2024-01-31T05:01:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.713+0000 7f9986200640 1 Processor -- start 2024-01-31T05:01:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.714+0000 7f9986200640 1 -- start start 2024-01-31T05:01:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.715+0000 7f9986200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960003970 0x7f9980099b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.715+0000 7f9986200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99800cbcc0 0x7f998015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.715+0000 7f9986200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 0x7f998015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.716+0000 7f997fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 0x7f998015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.716+0000 7f997fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 0x7f998015c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43298/0 (socket says 172.21.15.97:43298) 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.716+0000 7f997ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99800cbcc0 0x7f998015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f997f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960003970 0x7f9980099b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f997ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99800cbcc0 0x7f998015bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59238/0 (socket says 172.21.15.97:59238) 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f997f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960003970 0x7f9980099b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59924/0 (socket says 172.21.15.97:59924) 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f997fe00640 1 -- 172.21.15.97:0/3199890038 learned_addr learned my addr 172.21.15.97:0/3199890038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9980144300 con 0x7f99800cc140 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99801436f0 con 0x7f99800cbcc0 2024-01-31T05:01:14.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.717+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99801439d0 con 0x7f9960003970 2024-01-31T05:01:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.718+0000 7f997fe00640 1 -- 172.21.15.97:0/3199890038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960003970 msgr2=0x7f9980099b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.718+0000 7f997fe00640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960003970 0x7f9980099b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.719+0000 7f997fe00640 1 -- 172.21.15.97:0/3199890038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99800cbcc0 msgr2=0x7f998015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.719+0000 7f997fe00640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99800cbcc0 0x7f998015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.719+0000 7f997fe00640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9980143cb0 con 0x7f99800cc140 2024-01-31T05:01:14.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.720+0000 7f997fe00640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 0x7f998015c4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f99800734d0 tx=0x7f997000fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.720+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000fe40 con 0x7f99800cc140 2024-01-31T05:01:14.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.720+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99700052e0 con 0x7f99800cc140 2024-01-31T05:01:14.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.720+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970003830 con 0x7f99800cc140 2024-01-31T05:01:14.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.722+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9980143f90 con 0x7f99800cc140 2024-01-31T05:01:14.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.722+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9980147610 con 0x7f99800cc140 2024-01-31T05:01:14.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.724+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9980144150 con 0x7f99800cc140 2024-01-31T05:01:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.727+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9970003990 con 0x7f99800cc140 2024-01-31T05:01:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.729+0000 7f9977400640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f995806afd0 0x7f995806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.729+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9970089600 con 0x7f99800cc140 2024-01-31T05:01:14.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.730+0000 7f997f400640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f995806afd0 0x7f995806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.732+0000 7f997f400640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f995806afd0 0x7f995806d490 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f9978030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.734+0000 7f6686a00640 1 Processor -- start 2024-01-31T05:01:14.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.734+0000 7f6686a00640 1 -- start start 2024-01-31T05:01:14.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.735+0000 7fd88f200640 1 Processor -- start 2024-01-31T05:01:14.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.735+0000 7fd88f200640 1 -- start start 2024-01-31T05:01:14.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.736+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9970054270 con 0x7f99800cc140 2024-01-31T05:01:14.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.736+0000 7f6686a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66800ce1e0 0x7f66800b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.736+0000 7f6686a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66800b3490 con 0x7f66800cbb40 2024-01-31T05:01:14.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.736+0000 7f6686a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668006e1c0 con 0x7f6680161b50 2024-01-31T05:01:14.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.736+0000 7f6686a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6680162110 con 0x7f66800ce1e0 2024-01-31T05:01:14.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.737+0000 7f6684a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f66800cbb40 0x7f66800cbf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41794/0 (socket says 172.21.15.97:41794) 2024-01-31T05:01:14.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.737+0000 7f6684a00640 1 -- 172.21.15.97:0/3047610996 learned_addr learned my addr 172.21.15.97:0/3047610996 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:14.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.738+0000 7f667fe00640 1 --2- 172.21.15.97:0/3047610996 >> v2:172.21.15.188:3300/0 conn(0x7f66800ce1e0 0x7f66800b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.738+0000 7fd88f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd88003e840 0x7fd880053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.738+0000 7fd88f200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd880054130 0x7fd8800068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.738+0000 7f667ea00640 1 -- 172.21.15.97:0/3047610996 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940853166 0 0) 0x7f66800b3490 con 0x7f66800cbb40 2024-01-31T05:01:14.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.738+0000 7f667ea00640 1 -- 172.21.15.97:0/3047610996 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6660003520 con 0x7f66800cbb40 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7fd88d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd88003e840 0x7fd880053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7fd88d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd88003e840 0x7fd880053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43310/0 (socket says 172.21.15.97:43310) 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7fd88d400640 1 -- 172.21.15.97:0/3810719366 learned_addr learned my addr 172.21.15.97:0/3810719366 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd880058570 con 0x7fd8800e30c0 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800061a0 con 0x7fd88003e840 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800545a0 con 0x7fd880054130 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7fd88e800640 1 --2- 172.21.15.97:0/3810719366 >> v2:172.21.15.188:3300/0 conn(0x7fd880054130 0x7fd8800068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7fd88d400640 1 -- 172.21.15.97:0/3810719366 >> v2:172.21.15.188:3300/0 conn(0x7fd880054130 msgr2=0x7fd8800068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7fd88d400640 1 --2- 172.21.15.97:0/3810719366 >> v2:172.21.15.188:3300/0 conn(0x7fd880054130 0x7fd8800068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7f667fe00640 1 -- 172.21.15.97:0/3047610996 >> v1:172.21.15.188:6789/0 conn(0x7f6680161b50 legacy=0x7f66800ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7f667ea00640 1 -- 172.21.15.97:0/3047610996 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3042841562 0 0) 0x7f6660003520 con 0x7f66800cbb40 2024-01-31T05:01:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7f667fe00640 1 -- 172.21.15.97:0/3047610996 >> v1:172.21.15.97:6789/0 conn(0x7f66800cbb40 legacy=0x7f66800cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.739+0000 7f667fe00640 1 -- 172.21.15.97:0/3047610996 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66801623f0 con 0x7f66800ce1e0 2024-01-31T05:01:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7f2bcee00640 1 Processor -- start 2024-01-31T05:01:14.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7fd88d400640 1 -- 172.21.15.97:0/3810719366 >> v1:172.21.15.97:6789/0 conn(0x7fd8800e30c0 legacy=0x7fd880057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7f2bcee00640 1 -- start start 2024-01-31T05:01:14.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.740+0000 7fd88d400640 1 -- 172.21.15.97:0/3810719366 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880058850 con 0x7fd88003e840 2024-01-31T05:01:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.741+0000 7f667fe00640 1 --2- 172.21.15.97:0/3047610996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66800b2f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66680097a0 tx=0x7f666802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6662b4861d4f98 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.742+0000 7f667ea00640 1 -- 172.21.15.97:0/3047610996 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678001670 con 0x7f66800ce1e0 2024-01-31T05:01:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.742+0000 7f667ea00640 1 -- 172.21.15.97:0/3047610996 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666802e050 con 0x7f66800ce1e0 2024-01-31T05:01:14.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.742+0000 7f667ea00640 1 -- 172.21.15.97:0/3047610996 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668030710 con 0x7f66800ce1e0 2024-01-31T05:01:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.742+0000 7fd88d400640 1 --2- 172.21.15.97:0/3810719366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88003e840 0x7fd880053bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8800e7cd0 tx=0x7fd87802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed482ace84f1df1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bcee00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bc815d590 0x7f2bc815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bcee00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2bc80cfae0 0x7f2bc80cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bcee00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80d0460 con 0x7f2bc8161080 2024-01-31T05:01:14.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bcee00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc80cded0 con 0x7f2bc80cfae0 2024-01-31T05:01:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bcee00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc815d970 con 0x7f2bc815d590 2024-01-31T05:01:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f6686a00640 1 -- 172.21.15.97:0/3047610996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 msgr2=0x7f66800b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bc7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bc815d590 0x7f2bc815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7f2bc7400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2bc80cfae0 0x7f2bc80cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.743+0000 7fd88ca00640 1 -- 172.21.15.97:0/3810719366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878030860 con 0x7fd88003e840 2024-01-31T05:01:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f6686a00640 1 --2- 172.21.15.97:0/3047610996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66800b2f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66680097a0 tx=0x7f666802b790 comp rx=0 tx=0).stop 2024-01-31T05:01:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f2bc7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bc815d590 0x7f2bc815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59248/0 (socket says 172.21.15.97:59248) 2024-01-31T05:01:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f2bc7e00640 1 -- 172.21.15.97:0/2899033120 learned_addr learned my addr 172.21.15.97:0/2899033120 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7fd88ca00640 1 -- 172.21.15.97:0/3810719366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780309c0 con 0x7fd88003e840 2024-01-31T05:01:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f6686a00640 1 -- 172.21.15.97:0/3047610996 shutdown_connections 2024-01-31T05:01:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f6686a00640 1 --2- 172.21.15.97:0/3047610996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66800b2f50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f6686a00640 1 -- 172.21.15.97:0/3047610996 >> 172.21.15.97:0/3047610996 conn(0x7f66800b0a10 msgr2=0x7f66800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88003e840 msgr2=0x7fd880053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 --2- 172.21.15.97:0/3810719366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88003e840 0x7fd880053bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8800e7cd0 tx=0x7fd87802b9a0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f2bc7e00640 1 -- 172.21.15.97:0/2899033120 >> v2:172.21.15.97:3300/0 conn(0x7f2bc80cfae0 msgr2=0x7f2bc80cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f2bc7e00640 1 --2- 172.21.15.97:0/2899033120 >> v2:172.21.15.97:3300/0 conn(0x7f2bc80cfae0 0x7f2bc80cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f2bc7e00640 1 -- 172.21.15.97:0/2899033120 >> v1:172.21.15.188:6789/0 conn(0x7f2bc8161080 legacy=0x7f2bc81614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.744+0000 7f2bc7e00640 1 -- 172.21.15.97:0/2899033120 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80cf210 con 0x7f2bc815d590 2024-01-31T05:01:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 shutdown_connections 2024-01-31T05:01:14.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 --2- 172.21.15.97:0/3810719366 >> v2:172.21.15.188:3300/0 conn(0x7fd880054130 0x7fd8800068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 --2- 172.21.15.97:0/3810719366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd88003e840 0x7fd880053bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 >> 172.21.15.97:0/3810719366 conn(0x7fd8800559e0 msgr2=0x7fd880055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.745+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 shutdown_connections 2024-01-31T05:01:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.746+0000 7fd88f200640 1 -- 172.21.15.97:0/3810719366 wait complete. 2024-01-31T05:01:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.746+0000 7f6686a00640 1 -- 172.21.15.97:0/3047610996 shutdown_connections 2024-01-31T05:01:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.746+0000 7f6686a00640 1 -- 172.21.15.97:0/3047610996 wait complete. 2024-01-31T05:01:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.747+0000 7f2bc7e00640 1 --2- 172.21.15.97:0/2899033120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc815e540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808b830 tx=0x7f2bc002bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36cf1e35d48bf9f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.747+0000 7f2bc6a00640 1 -- 172.21.15.97:0/2899033120 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc002cb30 con 0x7f2bc815d590 2024-01-31T05:01:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.747+0000 7f2bc6a00640 1 -- 172.21.15.97:0/2899033120 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc002cc90 con 0x7f2bc815d590 2024-01-31T05:01:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.747+0000 7f2bc6a00640 1 -- 172.21.15.97:0/2899033120 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0034610 con 0x7f2bc815d590 2024-01-31T05:01:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 -- 172.21.15.97:0/2899033120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 msgr2=0x7f2bc815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2899033120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc815e540 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2bc808b830 tx=0x7f2bc002bf90 comp rx=0 tx=0).stop 2024-01-31T05:01:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7fd88f200640 1 Processor -- start 2024-01-31T05:01:14.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7fd88f200640 1 -- start start 2024-01-31T05:01:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 -- 172.21.15.97:0/2899033120 shutdown_connections 2024-01-31T05:01:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2899033120 >> v2:172.21.15.97:3300/0 conn(0x7f2bc80cfae0 0x7f2bc80cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2899033120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc815e540 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 -- 172.21.15.97:0/2899033120 >> 172.21.15.97:0/2899033120 conn(0x7f2bc809c560 msgr2=0x7f2bc80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.749+0000 7f2bcee00640 1 -- 172.21.15.97:0/2899033120 shutdown_connections 2024-01-31T05:01:14.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.750+0000 7f2bcee00640 1 -- 172.21.15.97:0/2899033120 wait complete. 2024-01-31T05:01:14.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.751+0000 7f6686a00640 1 Processor -- start 2024-01-31T05:01:14.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.751+0000 7f6686a00640 1 -- start start 2024-01-31T05:01:14.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.751+0000 7fd88f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd88003e840 0x7fd880179c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.751+0000 7fd88f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 0x7fd880052580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8800e30c0 0x7fd8800500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7f6686a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 0x7f668009c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7f6686a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66801f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 0x7fd880052580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 0x7fd880052580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59260/0 (socket says 172.21.15.97:59260) 2024-01-31T05:01:14.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7f6686a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6680161b50 0x7f66800cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88d400640 1 -- 172.21.15.97:0/540301721 learned_addr learned my addr 172.21.15.97:0/540301721 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88004f880 con 0x7fd8800e30c0 2024-01-31T05:01:14.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880052ac0 con 0x7fd880054130 2024-01-31T05:01:14.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd880052fb0 con 0x7fd88003e840 2024-01-31T05:01:14.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7f6686a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801ed090 con 0x7f66800cbb40 2024-01-31T05:01:14.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7f6686a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66801f90f0 con 0x7f66800ce1e0 2024-01-31T05:01:14.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.752+0000 7f6686a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66801f8850 con 0x7f6680161b50 2024-01-31T05:01:14.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.753+0000 7fd88de00640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd88003e840 0x7fd880179c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.753+0000 7f2bcee00640 1 Processor -- start 2024-01-31T05:01:14.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.753+0000 7fd88e800640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8800e30c0 0x7fd8800500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.753+0000 7f2bcee00640 1 -- start start 2024-01-31T05:01:14.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7f6684a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6680161b50 0x7f66800cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7fd88d400640 1 -- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd88003e840 msgr2=0x7fd880179c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7fd88d400640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd88003e840 0x7fd880179c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7fd88d400640 1 -- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8800e30c0 msgr2=0x7fd8800500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7fd88d400640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8800e30c0 0x7fd8800500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7fd88d400640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780094d0 con 0x7fd880054130 2024-01-31T05:01:14.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7f667fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 0x7f668009c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7f667fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 0x7f668009c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43334/0 (socket says 172.21.15.97:43334) 2024-01-31T05:01:14.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.754+0000 7f667fe00640 1 -- 172.21.15.97:0/99848287 learned_addr learned my addr 172.21.15.97:0/99848287 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.755+0000 7fd88d400640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 0x7fd880052580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd878000c00 tx=0x7fd878004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.755+0000 7f667f400640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66801f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.755+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878039de0 con 0x7fd880054130 2024-01-31T05:01:14.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.755+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8800dd8b0 con 0x7fd880054130 2024-01-31T05:01:14.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880052cd0 con 0x7fd880054130 2024-01-31T05:01:14.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.755+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878043410 con 0x7fd880054130 2024-01-31T05:01:14.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878039de0 con 0x7fd880054130 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7f667fe00640 1 -- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6680161b50 msgr2=0x7f66800cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7f667fe00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6680161b50 0x7f66800cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7f667fe00640 1 -- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 msgr2=0x7f66801f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7f667fe00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66801f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7f667fe00640 1 -- 172.21.15.97:0/99848287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6668009400 con 0x7f66800cbb40 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.756+0000 7f6684a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6680161b50 0x7f66800cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.757+0000 7f667fe00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 0x7f668009c360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f66680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.757+0000 7f2bcee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc80cfae0 0x7f2bc81ff480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.757+0000 7f2bcee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc80c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.757+0000 7f2bcee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 0x7f2bc815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.757+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd858005290 con 0x7fd880054130 2024-01-31T05:01:14.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666803c880 con 0x7f66800cbb40 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bcce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 0x7f2bc815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bcce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 0x7f2bc815f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43340/0 (socket says 172.21.15.97:43340) 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bcce00640 1 -- 172.21.15.97:0/2817881321 learned_addr learned my addr 172.21.15.97:0/2817881321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666803c9e0 con 0x7f66800cbb40 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666803cd70 con 0x7f66800cbb40 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80ca2e0 con 0x7f2bc8161080 2024-01-31T05:01:14.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bc7e00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc80cfae0 0x7f2bc81ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc81ff9c0 con 0x7f2bc815d590 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bc7400640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc80c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.758+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80ca4c0 con 0x7f2bc80cfae0 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66801f8b30 con 0x7f66800cbb40 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66800705c0 con 0x7f66800cbb40 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f2bc7400640 1 -- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc80cfae0 msgr2=0x7f2bc81ff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f2bc7400640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc80cfae0 0x7f2bc81ff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f2bc7400640 1 -- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 msgr2=0x7f2bc815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f2bc7400640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 0x7f2bc815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f2bc7400640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009b30 con 0x7f2bc815d590 2024-01-31T05:01:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.759+0000 7f2bcce00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 0x7f2bc815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.761+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd878030660 con 0x7fd880054130 2024-01-31T05:01:14.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.761+0000 7f2bc7400640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc80c9540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000d520 tx=0x7f2bb000d9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.762+0000 7fd886000640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd86006af60 0x7fd86006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.763+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8780ab800 con 0x7fd880054130 2024-01-31T05:01:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.763+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6680073520 con 0x7f66800cbb40 2024-01-31T05:01:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.763+0000 7fd88de00640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd86006af60 0x7fd86006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.764+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6668039040 con 0x7f66800cbb40 2024-01-31T05:01:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.764+0000 7fd88de00640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd86006af60 0x7fd86006d420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd8880641e0 tx=0x7fd888068430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.765+0000 7f6677e00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f665806af20 0x7f665806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.765+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66680ab310 con 0x7f66800cbb40 2024-01-31T05:01:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.766+0000 7f667f400640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f665806af20 0x7f665806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.766+0000 7f667f400640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f665806af20 0x7f665806d3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f668009cb20 tx=0x7f6670009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.773+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00125f0 con 0x7f2bc815d590 2024-01-31T05:01:14.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.773+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb0012750 con 0x7f2bc815d590 2024-01-31T05:01:14.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.773+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0012b00 con 0x7f2bc815d590 2024-01-31T05:01:14.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.773+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd878076500 con 0x7fd880054130 2024-01-31T05:01:14.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.773+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc80ca7a0 con 0x7f2bc815d590 2024-01-31T05:01:14.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.773+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc80cad60 con 0x7f2bc815d590 2024-01-31T05:01:14.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.775+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6668076230 con 0x7f66800cbb40 2024-01-31T05:01:14.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.776+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc80677c0 con 0x7f2bc815d590 2024-01-31T05:01:14.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.778+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2bb0012c60 con 0x7f2bc815d590 2024-01-31T05:01:14.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.780+0000 7f2bbfe00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2ba006afd0 0x7f2ba006d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.780+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2bb0089e40 con 0x7f2bc815d590 2024-01-31T05:01:14.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.781+0000 7f2bc7e00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2ba006afd0 0x7f2ba006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.783+0000 7f2bc7e00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2ba006afd0 0x7f2ba006d490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0035000 tx=0x7f2bc0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.790+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2bb0054a90 con 0x7f2bc815d590 2024-01-31T05:01:14.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.822+0000 7fea5c000640 1 Processor -- start 2024-01-31T05:01:14.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.823+0000 7fea5c000640 1 -- start start 2024-01-31T05:01:14.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.823+0000 7f24d7000640 1 Processor -- start 2024-01-31T05:01:14.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.823+0000 7f24d7000640 1 -- start start 2024-01-31T05:01:14.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.825+0000 7f24d7000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d015d590 0x7f24d015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.825+0000 7f24d7000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24d00cff20 0x7f24d0161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.825+0000 7f24d7000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d00cded0 con 0x7f24d00cfae0 2024-01-31T05:01:14.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.825+0000 7f24d7000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d015d970 con 0x7f24d00cff20 2024-01-31T05:01:14.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.825+0000 7f24d7000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00cf210 con 0x7f24d015d590 2024-01-31T05:01:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.826+0000 7f24d5000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24d00cff20 0x7f24d0161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.826+0000 7f24d5000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24d00cff20 0x7f24d0161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59288/0 (socket says 172.21.15.97:59288) 2024-01-31T05:01:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.826+0000 7f24d5000640 1 -- 172.21.15.97:0/1482666006 learned_addr learned my addr 172.21.15.97:0/1482666006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.826+0000 7f24cfe00640 1 --2- 172.21.15.97:0/1482666006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d015d590 0x7f24d015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.827+0000 7fea5c000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea540cfca0 0x7fea540b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.827+0000 7f24cfe00640 1 -- 172.21.15.97:0/1482666006 >> v2:172.21.15.188:3300/0 conn(0x7f24d00cff20 msgr2=0x7f24d0161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.827+0000 7f24cfe00640 1 --2- 172.21.15.97:0/1482666006 >> v2:172.21.15.188:3300/0 conn(0x7f24d00cff20 0x7f24d0161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.827+0000 7f24cfe00640 1 -- 172.21.15.97:0/1482666006 >> v1:172.21.15.97:6789/0 conn(0x7f24d00cfae0 legacy=0x7f24d0161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.827+0000 7f24cfe00640 1 -- 172.21.15.97:0/1482666006 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00ce170 con 0x7f24d015d590 2024-01-31T05:01:14.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.828+0000 7fea5c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540d3bb0 0x7fea540b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fe7b9600640 1 Processor -- start 2024-01-31T05:01:14.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fea5c000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540d1b20 con 0x7fea540b3e90 2024-01-31T05:01:14.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fe7b9600640 1 -- start start 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fea5c000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea540d1e00 con 0x7fea540cfca0 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fea5c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540d20e0 con 0x7fea540d3bb0 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fea58c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540d3bb0 0x7fea540b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fea59600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea540cfca0 0x7fea540b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.829+0000 7fea5a000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fea540b3e90 0x7fea540ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41820/0 (socket says 172.21.15.97:41820) 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.830+0000 7fea5a000640 1 -- 172.21.15.97:0/3414324320 learned_addr learned my addr 172.21.15.97:0/3414324320 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:14.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.830+0000 7f24cfe00640 1 --2- 172.21.15.97:0/1482666006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d015d590 0x7f24d015e540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f24b80097a0 tx=0x7f24b802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198a34c11af2a049 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fea59600640 1 -- 172.21.15.97:0/3414324320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540d3bb0 msgr2=0x7fea540b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.830+0000 7f24cea00640 1 -- 172.21.15.97:0/1482666006 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8030800 con 0x7f24d015d590 2024-01-31T05:01:14.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fea59600640 1 --2- 172.21.15.97:0/3414324320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540d3bb0 0x7fea540b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fe7b9600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7b415d3a0 0x7fe7b4161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fe7b9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b406a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fe7b9600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7b40af3c0 0x7fe7b406ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fe7b9600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b40cdd30 con 0x7fe7b415d3a0 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fe7b9600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b406ef80 con 0x7fe7b40af3c0 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fe7b9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4160e50 con 0x7fe7b40d0920 2024-01-31T05:01:14.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fea59600640 1 -- 172.21.15.97:0/3414324320 >> v1:172.21.15.97:6789/0 conn(0x7fea540b3e90 legacy=0x7fea540ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fea59600640 1 -- 172.21.15.97:0/3414324320 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540d4120 con 0x7fea540cfca0 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fea58c00640 1 --2- 172.21.15.97:0/3414324320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540d3bb0 0x7fea540b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7fea53e00640 1 -- 172.21.15.97:0/3414324320 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311353482 0 0) 0x7fea540d1b20 con 0x7fea540b3e90 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7f24cea00640 1 -- 172.21.15.97:0/1482666006 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8030960 con 0x7f24d015d590 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.831+0000 7f24cea00640 1 -- 172.21.15.97:0/1482666006 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8030800 con 0x7f24d015d590 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.832+0000 7f24d7000640 1 -- 172.21.15.97:0/1482666006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d015d590 msgr2=0x7f24d015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.832+0000 7f24d7000640 1 --2- 172.21.15.97:0/1482666006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d015d590 0x7f24d015e540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f24b80097a0 tx=0x7f24b802b940 comp rx=0 tx=0).stop 2024-01-31T05:01:14.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7fe7b2a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7b415d3a0 0x7fe7b4161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7f24d7000640 1 -- 172.21.15.97:0/1482666006 shutdown_connections 2024-01-31T05:01:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7f24d7000640 1 --2- 172.21.15.97:0/1482666006 >> v2:172.21.15.188:3300/0 conn(0x7f24d00cff20 0x7f24d0161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7f24d7000640 1 --2- 172.21.15.97:0/1482666006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d015d590 0x7f24d015e540 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7f24d7000640 1 -- 172.21.15.97:0/1482666006 >> 172.21.15.97:0/1482666006 conn(0x7f24d009c560 msgr2=0x7f24d00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7fe7b3400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7b40af3c0 0x7fe7b406ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7fe7b3400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7b40af3c0 0x7fe7b406ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:59290/0 (socket says 172.21.15.97:59290) 2024-01-31T05:01:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7fe7b3400640 1 -- 172.21.15.97:0/2421746826 learned_addr learned my addr 172.21.15.97:0/2421746826 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7f24d7000640 1 -- 172.21.15.97:0/1482666006 shutdown_connections 2024-01-31T05:01:14.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7f24d7000640 1 -- 172.21.15.97:0/1482666006 wait complete. 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.833+0000 7fea59600640 1 --2- 172.21.15.97:0/3414324320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540cfca0 0x7fea540b3150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fea44009a00 tx=0x7fea4402bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6e702a19528dae server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.834+0000 7fe7b3400640 1 -- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 msgr2=0x7fe7b406a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.834+0000 7fe7b3400640 1 --2- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b406a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.834+0000 7fe7b3400640 1 -- 172.21.15.97:0/2421746826 >> v2:172.21.15.97:3300/0 conn(0x7fe7b415d3a0 msgr2=0x7fe7b4161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.834+0000 7fe7b3400640 1 --2- 172.21.15.97:0/2421746826 >> v2:172.21.15.97:3300/0 conn(0x7fe7b415d3a0 0x7fe7b4161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.834+0000 7fe7b3400640 1 -- 172.21.15.97:0/2421746826 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4161130 con 0x7fe7b40af3c0 2024-01-31T05:01:14.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.834+0000 7fe7b2a00640 1 --2- 172.21.15.97:0/2421746826 >> v2:172.21.15.97:3300/0 conn(0x7fe7b415d3a0 0x7fe7b4161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.836+0000 7fea53e00640 1 -- 172.21.15.97:0/3414324320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4402d6e0 con 0x7fea540cfca0 2024-01-31T05:01:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.836+0000 7fea53e00640 1 -- 172.21.15.97:0/3414324320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4402d840 con 0x7fea540cfca0 2024-01-31T05:01:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.836+0000 7fea53e00640 1 -- 172.21.15.97:0/3414324320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4402dbd0 con 0x7fea540cfca0 2024-01-31T05:01:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.836+0000 7f24d7000640 1 Processor -- start 2024-01-31T05:01:14.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.836+0000 7f24d7000640 1 -- start start 2024-01-31T05:01:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.837+0000 7fea5c000640 1 -- 172.21.15.97:0/3414324320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540cfca0 msgr2=0x7fea540b3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.837+0000 7fea5c000640 1 --2- 172.21.15.97:0/3414324320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540cfca0 0x7fea540b3150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fea44009a00 tx=0x7fea4402bb30 comp rx=0 tx=0).stop 2024-01-31T05:01:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.837+0000 7fea5c000640 1 -- 172.21.15.97:0/3414324320 shutdown_connections 2024-01-31T05:01:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.837+0000 7fea5c000640 1 --2- 172.21.15.97:0/3414324320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540d3bb0 0x7fea540b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.837+0000 7fea5c000640 1 --2- 172.21.15.97:0/3414324320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540cfca0 0x7fea540b3150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.837+0000 7fea5c000640 1 -- 172.21.15.97:0/3414324320 >> 172.21.15.97:0/3414324320 conn(0x7fea5409c570 msgr2=0x7fea540d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.838+0000 7fea5c000640 1 -- 172.21.15.97:0/3414324320 shutdown_connections 2024-01-31T05:01:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.838+0000 7fea5c000640 1 -- 172.21.15.97:0/3414324320 wait complete. 2024-01-31T05:01:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.838+0000 7f24d7000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d00cfae0 0x7f24d01fd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7f24d7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 0x7f24d006b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.838+0000 7fe7b3400640 1 --2- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b40af3c0 0x7fe7b406ac50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40b2ba0 tx=0x7fe79c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce3a6429fe25d62b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7f24d7000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d015d590 0x7f24d006b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7f24d7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0204250 con 0x7f24d00cff20 2024-01-31T05:01:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7f24d7000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00a0e50 con 0x7f24d015d590 2024-01-31T05:01:14.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7f24d7000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d0203f70 con 0x7f24d00cfae0 2024-01-31T05:01:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7fe7b1600640 1 -- 172.21.15.97:0/2421746826 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c02d040 con 0x7fe7b40af3c0 2024-01-31T05:01:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7fe7b1600640 1 -- 172.21.15.97:0/2421746826 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c031810 con 0x7fe7b40af3c0 2024-01-31T05:01:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.839+0000 7fe7b1600640 1 -- 172.21.15.97:0/2421746826 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c02d040 con 0x7fe7b40af3c0 2024-01-31T05:01:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 -- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b40af3c0 msgr2=0x7fe7b406ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 --2- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b40af3c0 0x7fe7b406ac50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40b2ba0 tx=0x7fe79c02b8b0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 0x7f24d006b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24d5000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d015d590 0x7f24d006b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d00cfae0 0x7f24d01fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d00cfae0 0x7f24d01fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59980/0 (socket says 172.21.15.97:59980) 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cfe00640 1 -- 172.21.15.97:0/3564913965 learned_addr learned my addr 172.21.15.97:0/3564913965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 0x7f24d006b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43356/0 (socket says 172.21.15.97:43356) 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 -- 172.21.15.97:0/3564913965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d00cfae0 msgr2=0x7f24d01fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d00cfae0 0x7f24d01fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 -- 172.21.15.97:0/3564913965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d015d590 msgr2=0x7f24d006b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d015d590 0x7f24d006b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.840+0000 7f24cf400640 1 -- 172.21.15.97:0/3564913965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009400 con 0x7f24d00cff20 2024-01-31T05:01:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7f24cf400640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 0x7f24d006b410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c780 tx=0x7f24c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 -- 172.21.15.97:0/2421746826 shutdown_connections 2024-01-31T05:01:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c00050d0 con 0x7f24d00cff20 2024-01-31T05:01:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c0005230 con 0x7f24d00cff20 2024-01-31T05:01:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000a640 con 0x7f24d00cff20 2024-01-31T05:01:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.842+0000 7fea5c000640 1 Processor -- start 2024-01-31T05:01:14.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.842+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01fff40 con 0x7f24d00cff20 2024-01-31T05:01:14.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.842+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d0200500 con 0x7f24d00cff20 2024-01-31T05:01:14.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 --2- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b40af3c0 0x7fe7b406ac50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 --2- 172.21.15.97:0/2421746826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b406a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 --2- 172.21.15.97:0/2421746826 >> v2:172.21.15.97:3300/0 conn(0x7fe7b415d3a0 0x7fe7b4161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.841+0000 7fe7b9600640 1 -- 172.21.15.97:0/2421746826 >> 172.21.15.97:0/2421746826 conn(0x7fe7b40cfe30 msgr2=0x7fe7b40d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.843+0000 7fe7b9600640 1 -- 172.21.15.97:0/2421746826 shutdown_connections 2024-01-31T05:01:14.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.843+0000 7fe7b9600640 1 -- 172.21.15.97:0/2421746826 wait complete. 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.843+0000 7fea5c000640 1 -- start start 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540b3e90 0x7fea541fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5c000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea540cfca0 0x7fea540c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 0x7fea540b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540b0270 con 0x7fea540d3bb0 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5c000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea541fb190 con 0x7fea540cfca0 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea541fb470 con 0x7fea540b3e90 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 0x7fea540b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea5a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 0x7fea540b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43360/0 (socket says 172.21.15.97:43360) 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea59600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540b3e90 0x7fea541fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7fea59600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540b3e90 0x7fea541fa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59986/0 (socket says 172.21.15.97:59986) 2024-01-31T05:01:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.846+0000 7fea59600640 1 -- 172.21.15.97:0/3870307937 learned_addr learned my addr 172.21.15.97:0/3870307937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.845+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a0005290 con 0x7f24d00cff20 2024-01-31T05:01:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.846+0000 7fea58c00640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea540cfca0 0x7fea540c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.847+0000 7fea5a000640 1 -- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540b3e90 msgr2=0x7fea541fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.847+0000 7fea5a000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540b3e90 0x7fea541fa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.847+0000 7fea5a000640 1 -- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea540cfca0 msgr2=0x7fea540c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.847+0000 7fea5a000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea540cfca0 0x7fea540c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.847+0000 7fea5a000640 1 -- 172.21.15.97:0/3870307937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009400 con 0x7fea540d3bb0 2024-01-31T05:01:14.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.847+0000 7fe7b9600640 1 Processor -- start 2024-01-31T05:01:14.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.849+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f24c000a7a0 con 0x7f24d00cff20 2024-01-31T05:01:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.849+0000 7fea5a000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 0x7fea540b0700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00c6d0 tx=0x7fea4c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.848+0000 7fe7b9600640 1 -- start start 2024-01-31T05:01:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.850+0000 7fe7b9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40af3c0 0x7fe7b4071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.850+0000 7fe7b9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b40cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.850+0000 7fe7b9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b415d3a0 0x7fe7b40cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40b4680 con 0x7fe7b40af3c0 2024-01-31T05:01:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.850+0000 7f24c7e00640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f24a806aed0 0x7f24a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.850+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f24c00893b0 con 0x7f24d00cff20 2024-01-31T05:01:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b40cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b40cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:60000/0 (socket says 172.21.15.97:60000) 2024-01-31T05:01:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b2000640 1 -- 172.21.15.97:0/3103740549 learned_addr learned my addr 172.21.15.97:0/3103740549 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b3400640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b415d3a0 0x7fe7b40cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40b4960 con 0x7fe7b415d3a0 2024-01-31T05:01:14.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40b4c40 con 0x7fe7b40d0920 2024-01-31T05:01:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.851+0000 7f24cfe00640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f24a806aed0 0x7f24a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.852+0000 7fe7b2a00640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40af3c0 0x7fe7b4071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.853+0000 7fe7b3400640 1 -- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 msgr2=0x7fe7b40cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.853+0000 7f24cfe00640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f24a806aed0 0x7f24a806d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24b80091d0 tx=0x7f24b8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.853+0000 7fe7b3400640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b40cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.853+0000 7fe7b3400640 1 -- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40af3c0 msgr2=0x7fe7b4071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.853+0000 7fe7b3400640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40af3c0 0x7fe7b4071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.853+0000 7fe7b3400640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a4009400 con 0x7fe7b415d3a0 2024-01-31T05:01:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.854+0000 7fe7b2a00640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40af3c0 0x7fe7b4071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:14.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.854+0000 7fe7b3400640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b415d3a0 0x7fe7b40cb660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe79c033000 tx=0x7fe79c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.855+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003610 con 0x7fe7b415d3a0 2024-01-31T05:01:14.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.855+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c03e070 con 0x7fe7b415d3a0 2024-01-31T05:01:14.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.855+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c0094d0 con 0x7fe7b415d3a0 2024-01-31T05:01:14.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.856+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0039d0 con 0x7fe7b415d3a0 2024-01-31T05:01:14.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.856+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b40720c0 con 0x7fe7b415d3a0 2024-01-31T05:01:14.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.858+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004d60 con 0x7fea540d3bb0 2024-01-31T05:01:14.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.858+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c00a640 con 0x7fea540d3bb0 2024-01-31T05:01:14.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.858+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea540aff00 con 0x7fea540d3bb0 2024-01-31T05:01:14.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.858+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f24c0053ef0 con 0x7f24d00cff20 2024-01-31T05:01:14.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.858+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004d60 con 0x7fea540d3bb0 2024-01-31T05:01:14.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.860+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea540c9db0 con 0x7fea540d3bb0 2024-01-31T05:01:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.860+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe79c003b30 con 0x7fe7b415d3a0 2024-01-31T05:01:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.861+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b4072310 con 0x7fe7b415d3a0 2024-01-31T05:01:14.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.861+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea540b03e0 con 0x7fea540d3bb0 2024-01-31T05:01:14.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.862+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fea4c00a7f0 con 0x7fea540d3bb0 2024-01-31T05:01:14.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.862+0000 7fe7aaa00640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe78c06aed0 0x7fe78c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.862+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe79c0ab560 con 0x7fe7b415d3a0 2024-01-31T05:01:14.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.863+0000 7fe7b2a00640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe78c06aed0 0x7fe78c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.866+0000 7fea51600640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea2c06af30 0x7fea2c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.866+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea4c088c90 con 0x7fea540d3bb0 2024-01-31T05:01:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.867+0000 7fe7b2a00640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe78c06aed0 0x7fe78c06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009c80 tx=0x7fe7a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.867+0000 7fea59600640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea2c06af30 0x7fea2c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.869+0000 7fea59600640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea2c06af30 0x7fea2c06d3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea44000c00 tx=0x7fea44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.871+0000 7fb65d200640 1 Processor -- start 2024-01-31T05:01:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.871+0000 7fb65d200640 1 -- start start 2024-01-31T05:01:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.873+0000 7fb65d200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6580cfee0 0x7fb6580ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.873+0000 7fb65d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6580b3800 0x7fb6580ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.873+0000 7fb65d200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6580d21c0 con 0x7fb6580b3420 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.873+0000 7fb65d200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6580cdcb0 con 0x7fb6580cfee0 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fb65d200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6580d3bc0 con 0x7fb6580b3800 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fb656a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6580cfee0 0x7fb6580ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fb657400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb6580b3420 0x7fb6580cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:55052/0 (socket says 172.21.15.97:55052) 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fb657400640 1 -- 172.21.15.97:0/3525903056 learned_addr learned my addr 172.21.15.97:0/3525903056 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fb656a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6580cfee0 0x7fb6580ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43376/0 (socket says 172.21.15.97:43376) 2024-01-31T05:01:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fb657e00640 1 --2- 172.21.15.97:0/3525903056 >> v2:172.21.15.188:3300/0 conn(0x7fb6580b3800 0x7fb6580ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.874+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fea4c0536d0 con 0x7fea540d3bb0 2024-01-31T05:01:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe79c0760a0 con 0x7fe7b415d3a0 2024-01-31T05:01:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fb656000640 1 -- 172.21.15.97:0/3525903056 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721761757 0 0) 0x7fb648001490 con 0x7fb6580b3420 2024-01-31T05:01:14.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fb656000640 1 -- 172.21.15.97:0/3525903056 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb638003520 con 0x7fb6580b3420 2024-01-31T05:01:14.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fb656a00640 1 -- 172.21.15.97:0/3525903056 >> v2:172.21.15.188:3300/0 conn(0x7fb6580b3800 msgr2=0x7fb6580ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fb656a00640 1 --2- 172.21.15.97:0/3525903056 >> v2:172.21.15.188:3300/0 conn(0x7fb6580b3800 0x7fb6580ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fb656a00640 1 -- 172.21.15.97:0/3525903056 >> v1:172.21.15.188:6789/0 conn(0x7fb6580b3420 legacy=0x7fb6580cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.875+0000 7fb656a00640 1 -- 172.21.15.97:0/3525903056 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580d3ea0 con 0x7fb6580cfee0 2024-01-31T05:01:14.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.880+0000 7fb656a00640 1 --2- 172.21.15.97:0/3525903056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6580cfee0 0x7fb6580ce190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6580ca0f0 tx=0x7fb64002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58501354010af377 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.881+0000 7fb656000640 1 -- 172.21.15.97:0/3525903056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64002e070 con 0x7fb6580cfee0 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.881+0000 7fb656000640 1 -- 172.21.15.97:0/3525903056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6400315a0 con 0x7fb6580cfee0 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.881+0000 7fb656000640 1 -- 172.21.15.97:0/3525903056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640031930 con 0x7fb6580cfee0 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.882+0000 7fb65d200640 1 -- 172.21.15.97:0/3525903056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6580cfee0 msgr2=0x7fb6580ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.882+0000 7fb65d200640 1 --2- 172.21.15.97:0/3525903056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6580cfee0 0x7fb6580ce190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6580ca0f0 tx=0x7fb64002b840 comp rx=0 tx=0).stop 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.883+0000 7fb65d200640 1 -- 172.21.15.97:0/3525903056 shutdown_connections 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.883+0000 7fb65d200640 1 --2- 172.21.15.97:0/3525903056 >> v2:172.21.15.188:3300/0 conn(0x7fb6580b3800 0x7fb6580ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.884+0000 7fb65d200640 1 --2- 172.21.15.97:0/3525903056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6580cfee0 0x7fb6580ce190 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6580ca0f0 tx=0x7fb64002b840 comp rx=0 tx=0).stop 2024-01-31T05:01:14.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.884+0000 7fb65d200640 1 -- 172.21.15.97:0/3525903056 >> 172.21.15.97:0/3525903056 conn(0x7fb6580d30a0 msgr2=0x7fb6580d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:14.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.887+0000 7fb65d200640 1 -- 172.21.15.97:0/3525903056 shutdown_connections 2024-01-31T05:01:14.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.887+0000 7fb65d200640 1 -- 172.21.15.97:0/3525903056 wait complete. 2024-01-31T05:01:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.890+0000 7fb65d200640 1 Processor -- start 2024-01-31T05:01:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.890+0000 7fb65d200640 1 -- start start 2024-01-31T05:01:14.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.892+0000 7fb65d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6580b3420 0x7fb6580afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.892+0000 7fb65d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6580b3800 0x7fb6581e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.892+0000 7fb65d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 0x7fb6580cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb657400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6580b3420 0x7fb6580afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb657e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 0x7fb6580cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb657e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 0x7fb6580cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43392/0 (socket says 172.21.15.97:43392) 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb657400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6580b3420 0x7fb6580afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:60010/0 (socket says 172.21.15.97:60010) 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb657e00640 1 -- 172.21.15.97:0/217495142 learned_addr learned my addr 172.21.15.97:0/217495142 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6581f7370 con 0x7fb6581f6ee0 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580b0280 con 0x7fb6580b3800 2024-01-31T05:01:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.893+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6580b0560 con 0x7fb6580b3420 2024-01-31T05:01:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.894+0000 7fb657e00640 1 -- 172.21.15.97:0/217495142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6580b3420 msgr2=0x7fb6580afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.894+0000 7fb657e00640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6580b3420 0x7fb6580afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.894+0000 7fb657e00640 1 -- 172.21.15.97:0/217495142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6580b3800 msgr2=0x7fb6581e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.894+0000 7fb657e00640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6580b3800 0x7fb6581e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.894+0000 7fb657e00640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009400 con 0x7fb6581f6ee0 2024-01-31T05:01:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.894+0000 7fb657e00640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 0x7fb6580cc5b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e850 tx=0x7fb65000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.895+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650009630 con 0x7fb6581f6ee0 2024-01-31T05:01:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.895+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65000c940 con 0x7fb6581f6ee0 2024-01-31T05:01:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.895+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000ccb0 con 0x7fb6581f6ee0 2024-01-31T05:01:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.898+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6580b0840 con 0x7fb6581f6ee0 2024-01-31T05:01:14.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.898+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6580ccaf0 con 0x7fb6581f6ee0 2024-01-31T05:01:14.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.901+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6580b0d60 con 0x7fb6581f6ee0 2024-01-31T05:01:14.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.902+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb650013470 con 0x7fb6581f6ee0 2024-01-31T05:01:14.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.904+0000 7fb64f400640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb63006af20 0x7fb63006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:14.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.904+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb650089a70 con 0x7fb6581f6ee0 2024-01-31T05:01:14.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.906+0000 7fb657400640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb63006af20 0x7fb63006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:14.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.907+0000 7fb657400640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb63006af20 0x7fb63006d3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6480012e0 tx=0x7fb6480018d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:14.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:14.912+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb6580b0d60 con 0x7fb6581f6ee0 2024-01-31T05:01:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.113+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f99800b1250 con 0x7f99800cc140 2024-01-31T05:01:15.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.115+0000 7f9977400640 1 -- 172.21.15.97:0/3199890038 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9970057890 con 0x7f99800cc140 2024-01-31T05:01:15.115 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.118+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f995806afd0 msgr2=0x7f995806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.119+0000 7f9986200640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f995806afd0 0x7f995806d490 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f9978030000 comp rx=0 tx=0).stop 2024-01-31T05:01:15.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.119+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 msgr2=0x7f998015c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.120+0000 7f9986200640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 0x7f998015c4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f99800734d0 tx=0x7f997000fb90 comp rx=0 tx=0).stop 2024-01-31T05:01:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.120+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 shutdown_connections 2024-01-31T05:01:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.120+0000 7f9986200640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f995806afd0 0x7f995806d490 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.120+0000 7f9986200640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99800cc140 0x7f998015c4c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.121+0000 7f9986200640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99800cbcc0 0x7f998015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.121+0000 7f9986200640 1 --2- 172.21.15.97:0/3199890038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9960003970 0x7f9980099b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.121+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 >> 172.21.15.97:0/3199890038 conn(0x7f99800b0a10 msgr2=0x7f99800ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.121+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 shutdown_connections 2024-01-31T05:01:15.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.121+0000 7f9986200640 1 -- 172.21.15.97:0/3199890038 wait complete. 2024-01-31T05:01:15.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-01-31T05:01:15.147 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.177+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2bc80c68a0 con 0x7f2bc815d590 2024-01-31T05:01:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.178+0000 7f2bbfe00640 1 -- 172.21.15.97:0/2817881321 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bb00580b0 con 0x7f2bc815d590 2024-01-31T05:01:15.177 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-01-31T05:01:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.181+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2ba006afd0 msgr2=0x7f2ba006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.181+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2ba006afd0 0x7f2ba006d490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0035000 tx=0x7f2bc0030000 comp rx=0 tx=0).stop 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.181+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 msgr2=0x7f2bc80c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.181+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc80c9540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000d520 tx=0x7f2bb000d9e0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 shutdown_connections 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2ba006afd0 0x7f2ba006d490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bc8161080 0x7f2bc815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc815d590 0x7f2bc80c9540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 --2- 172.21.15.97:0/2817881321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc80cfae0 0x7f2bc81ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 >> 172.21.15.97:0/2817881321 conn(0x7f2bc809c560 msgr2=0x7f2bc806cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 shutdown_connections 2024-01-31T05:01:15.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.182+0000 7f2bcee00640 1 -- 172.21.15.97:0/2817881321 wait complete. 2024-01-31T05:01:15.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-01-31T05:01:15.206 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.208+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6680070420 con 0x7f66800cbb40 2024-01-31T05:01:15.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.208+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd858005810 con 0x7fd880054130 2024-01-31T05:01:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.208+0000 7f6677e00640 1 -- 172.21.15.97:0/99848287 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6680070420 con 0x7f66800cbb40 2024-01-31T05:01:15.208 INFO:teuthology.orchestra.run.smithi097.stdout:60129542153 2024-01-31T05:01:15.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.209+0000 7fd886000640 1 -- 172.21.15.97:0/540301721 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd878079b20 con 0x7fd880054130 2024-01-31T05:01:15.209 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f665806af20 msgr2=0x7f665806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f665806af20 0x7f665806d3e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f668009cb20 tx=0x7f6670009040 comp rx=0 tx=0).stop 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 msgr2=0x7f668009c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 0x7f668009c360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f66680046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 shutdown_connections 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6680161b50 0x7f66800cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66800ce1e0 0x7f66801f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f665806af20 0x7f665806d3e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 --2- 172.21.15.97:0/99848287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66800cbb40 0x7f668009c360 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6668000c00 tx=0x7f66680046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 >> 172.21.15.97:0/99848287 conn(0x7f66800b0a10 msgr2=0x7f66800ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 shutdown_connections 2024-01-31T05:01:15.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.211+0000 7f6686a00640 1 -- 172.21.15.97:0/99848287 wait complete. 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.212+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd86006af60 msgr2=0x7fd86006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.212+0000 7fd88f200640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd86006af60 0x7fd86006d420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd8880641e0 tx=0x7fd888068430 comp rx=0 tx=0).stop 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.212+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 msgr2=0x7fd880052580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.212+0000 7fd88f200640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 0x7fd880052580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd878000c00 tx=0x7fd878004730 comp rx=0 tx=0).stop 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 shutdown_connections 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd86006af60 0x7fd86006d420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8800e30c0 0x7fd8800500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd880054130 0x7fd880052580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 --2- 172.21.15.97:0/540301721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd88003e840 0x7fd880179c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 >> 172.21.15.97:0/540301721 conn(0x7fd8800559e0 msgr2=0x7fd8800390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 shutdown_connections 2024-01-31T05:01:15.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.213+0000 7fd88f200640 1 -- 172.21.15.97:0/540301721 wait complete. 2024-01-31T05:01:15.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.0 2024-01-31T05:01:15.228 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.234+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb6580b0d60 con 0x7fb6581f6ee0 2024-01-31T05:01:15.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.234+0000 7fb64f400640 1 -- 172.21.15.97:0/217495142 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6580b0d60 con 0x7fb6581f6ee0 2024-01-31T05:01:15.234 INFO:teuthology.orchestra.run.smithi097.stdout:64424509449 2024-01-31T05:01:15.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-01-31T05:01:15.237 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb63006af20 msgr2=0x7fb63006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb63006af20 0x7fb63006d3e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6480012e0 tx=0x7fb6480018d0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 msgr2=0x7fb6580cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 0x7fb6580cc5b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb65000e850 tx=0x7fb65000ed10 comp rx=0 tx=0).stop 2024-01-31T05:01:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 shutdown_connections 2024-01-31T05:01:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb63006af20 0x7fb63006d3e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6581f6ee0 0x7fb6580cc5b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6580b3800 0x7fb6581e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 --2- 172.21.15.97:0/217495142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6580b3420 0x7fb6580afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 >> 172.21.15.97:0/217495142 conn(0x7fb6580d30a0 msgr2=0x7fb6580c7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 shutdown_connections 2024-01-31T05:01:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.237+0000 7fb65d200640 1 -- 172.21.15.97:0/217495142 wait complete. 2024-01-31T05:01:15.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.251+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe7b40b5160 con 0x7fe7b415d3a0 2024-01-31T05:01:15.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.252+0000 7fe7aaa00640 1 -- 172.21.15.97:0/3103740549 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe7b40b5160 con 0x7fe7b415d3a0 2024-01-31T05:01:15.252 INFO:teuthology.orchestra.run.smithi097.stdout:64424509448 2024-01-31T05:01:15.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-01-31T05:01:15.253 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe78c06aed0 msgr2=0x7fe78c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe78c06aed0 0x7fe78c06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009c80 tx=0x7fe7a4008040 comp rx=0 tx=0).stop 2024-01-31T05:01:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b415d3a0 msgr2=0x7fe7b40cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b415d3a0 0x7fe7b40cb660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe79c033000 tx=0x7fe79c0046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 shutdown_connections 2024-01-31T05:01:15.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7b415d3a0 0x7fe7b40cb660 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7b40d0920 0x7fe7b40cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe78c06aed0 0x7fe78c06d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 --2- 172.21.15.97:0/3103740549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7b40af3c0 0x7fe7b4071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.254+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 >> 172.21.15.97:0/3103740549 conn(0x7fe7b40cfe30 msgr2=0x7fe7b406d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.255+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 shutdown_connections 2024-01-31T05:01:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.255+0000 7fe7b9600640 1 -- 172.21.15.97:0/3103740549 wait complete. 2024-01-31T05:01:15.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2024-01-31T05:01:15.270 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.290+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fea54067dc0 con 0x7fea540d3bb0 2024-01-31T05:01:15.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.291+0000 7fea51600640 1 -- 172.21.15.97:0/3870307937 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea4c018070 con 0x7fea540d3bb0 2024-01-31T05:01:15.290 INFO:teuthology.orchestra.run.smithi097.stdout:60129542154 2024-01-31T05:01:15.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea2c06af30 msgr2=0x7fea2c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea2c06af30 0x7fea2c06d3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea44000c00 tx=0x7fea44008030 comp rx=0 tx=0).stop 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 msgr2=0x7fea540b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 0x7fea540b0700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00c6d0 tx=0x7fea4c00cb90 comp rx=0 tx=0).stop 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 shutdown_connections 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea2c06af30 0x7fea2c06d3f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea540d3bb0 0x7fea540b0700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea540cfca0 0x7fea540c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 --2- 172.21.15.97:0/3870307937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea540b3e90 0x7fea541fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 >> 172.21.15.97:0/3870307937 conn(0x7fea5409c570 msgr2=0x7fea5406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 shutdown_connections 2024-01-31T05:01:15.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.294+0000 7fea5c000640 1 -- 172.21.15.97:0/3870307937 wait complete. 2024-01-31T05:01:15.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.296+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f24a0005540 con 0x7f24d00cff20 2024-01-31T05:01:15.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.296+0000 7f24c7e00640 1 -- 172.21.15.97:0/3564913965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24c0057510 con 0x7f24d00cff20 2024-01-31T05:01:15.296 INFO:teuthology.orchestra.run.smithi097.stdout:60129542153 2024-01-31T05:01:15.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f24a806aed0 msgr2=0x7f24a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f24a806aed0 0x7f24a806d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24b80091d0 tx=0x7f24b8008030 comp rx=0 tx=0).stop 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 msgr2=0x7f24d006b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 0x7f24d006b410 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f24c000c780 tx=0x7f24c000cc40 comp rx=0 tx=0).stop 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 shutdown_connections 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d015d590 0x7f24d006b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f24a806aed0 0x7f24a806d390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24d00cff20 0x7f24d006b410 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 --2- 172.21.15.97:0/3564913965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d00cfae0 0x7f24d01fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 >> 172.21.15.97:0/3564913965 conn(0x7f24d009c560 msgr2=0x7f24d006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 shutdown_connections 2024-01-31T05:01:15.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.300+0000 7f24d7000640 1 -- 172.21.15.97:0/3564913965 wait complete. 2024-01-31T05:01:15.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.6 2024-01-31T05:01:15.316 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.5 2024-01-31T05:01:15.320 DEBUG:teuthology.parallel:result is None 2024-01-31T05:01:15.321 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-31T05:01:15.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:01:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.568+0000 7f0c29200640 1 Processor -- start 2024-01-31T05:01:15.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.568+0000 7f0c29200640 1 -- start start 2024-01-31T05:01:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c29200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c2415d4e0 0x7f0c240b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c29200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c29200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2415d8c0 con 0x7f0c240d1560 2024-01-31T05:01:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c29200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c240b0b20 con 0x7f0c2415d4e0 2024-01-31T05:01:15.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c29200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240b22f0 con 0x7f0c240d0d60 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c23400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c2415d4e0 0x7f0c240b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c23e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0c240d1560 0x7f0c240b5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33130/0 (socket says 172.21.15.97:33130) 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.570+0000 7f0c23e00640 1 -- 172.21.15.97:0/4149332721 learned_addr learned my addr 172.21.15.97:0/4149332721 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.571+0000 7f0c22a00640 1 --2- 172.21.15.97:0/4149332721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.571+0000 7f0c23400640 1 -- 172.21.15.97:0/4149332721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 msgr2=0x7f0c240b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.571+0000 7f0c23400640 1 --2- 172.21.15.97:0/4149332721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.571+0000 7f0c23400640 1 -- 172.21.15.97:0/4149332721 >> v1:172.21.15.97:6789/0 conn(0x7f0c240d1560 legacy=0x7f0c240b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.571+0000 7f0c23400640 1 -- 172.21.15.97:0/4149332721 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c240b1920 con 0x7f0c2415d4e0 2024-01-31T05:01:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.571+0000 7f0c22000640 1 -- 172.21.15.97:0/4149332721 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845929798 0 0) 0x7f0c1c001490 con 0x7f0c240d1560 2024-01-31T05:01:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.572+0000 7f0c23400640 1 --2- 172.21.15.97:0/4149332721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b05e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c140097d0 tx=0x7f0c1402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a2b15c97ccf46d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.572+0000 7f0c22000640 1 -- 172.21.15.97:0/4149332721 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14031890 con 0x7f0c2415d4e0 2024-01-31T05:01:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.572+0000 7f0c22000640 1 -- 172.21.15.97:0/4149332721 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c140319f0 con 0x7f0c2415d4e0 2024-01-31T05:01:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.572+0000 7f0c22000640 1 -- 172.21.15.97:0/4149332721 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14031d80 con 0x7f0c2415d4e0 2024-01-31T05:01:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 -- 172.21.15.97:0/4149332721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 msgr2=0x7f0c240b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 --2- 172.21.15.97:0/4149332721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b05e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c140097d0 tx=0x7f0c1402b930 comp rx=0 tx=0).stop 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 -- 172.21.15.97:0/4149332721 shutdown_connections 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 --2- 172.21.15.97:0/4149332721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 --2- 172.21.15.97:0/4149332721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b05e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 -- 172.21.15.97:0/4149332721 >> 172.21.15.97:0/4149332721 conn(0x7f0c240d3010 msgr2=0x7f0c240d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 -- 172.21.15.97:0/4149332721 shutdown_connections 2024-01-31T05:01:15.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.573+0000 7f0c29200640 1 -- 172.21.15.97:0/4149332721 wait complete. 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.574+0000 7f0c29200640 1 Processor -- start 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.574+0000 7f0c29200640 1 -- start start 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.575+0000 7f0c29200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.575+0000 7f0c29200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c240d1560 0x7f0c240b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.575+0000 7f0c29200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.575+0000 7f0c29200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240ca970 con 0x7f0c2415d4e0 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.575+0000 7f0c29200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240b3290 con 0x7f0c240d1560 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.575+0000 7f0c29200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240b50f0 con 0x7f0c240d0d60 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34336/0 (socket says 172.21.15.97:34336) 2024-01-31T05:01:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 -- 172.21.15.97:0/3126550997 learned_addr learned my addr 172.21.15.97:0/3126550997 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c22a00640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c240d1560 0x7f0c240b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23400640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 -- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 msgr2=0x7f0c240b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 -- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c240d1560 msgr2=0x7f0c240b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c240d1560 0x7f0c240b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 -- 172.21.15.97:0/3126550997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14009400 con 0x7f0c2415d4e0 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.576+0000 7f0c23e00640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b7380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c7f0 tx=0x7f0c1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.577+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00a5e0 con 0x7f0c2415d4e0 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.577+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c00a760 con 0x7f0c2415d4e0 2024-01-31T05:01:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.577+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c240b53d0 con 0x7f0c2415d4e0 2024-01-31T05:01:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.577+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00ab50 con 0x7f0c2415d4e0 2024-01-31T05:01:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.577+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c240cae60 con 0x7f0c2415d4e0 2024-01-31T05:01:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.578+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c240676b0 con 0x7f0c2415d4e0 2024-01-31T05:01:15.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.579+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f0c1c00acb0 con 0x7f0c2415d4e0 2024-01-31T05:01:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.580+0000 7f0c1b400640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0bfc06aed0 0x7f0bfc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.580+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0c1c088f70 con 0x7f0c2415d4e0 2024-01-31T05:01:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.580+0000 7f0c23400640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0bfc06aed0 0x7f0bfc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.581+0000 7f0c23400640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0bfc06aed0 0x7f0bfc06d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c14036000 tx=0x7f0c1403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.584+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f0c1c053b40 con 0x7f0c2415d4e0 2024-01-31T05:01:15.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.758+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c240b0ff0 con 0x7f0bfc06aed0 2024-01-31T05:01:15.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.768+0000 7f0c1b400640 1 -- 172.21.15.97:0/3126550997 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345167 (secure 0 0 0) 0x7f0c14044400 con 0x7f0bfc06aed0 2024-01-31T05:01:15.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:15.774 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:01:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0bfc06aed0 msgr2=0x7f0bfc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0bfc06aed0 0x7f0bfc06d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0c14036000 tx=0x7f0c1403c040 comp rx=0 tx=0).stop 2024-01-31T05:01:15.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 msgr2=0x7f0c240b7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b7380 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c7f0 tx=0x7f0c1c00ccb0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 shutdown_connections 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0bfc06aed0 0x7f0bfc06d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c2415d4e0 0x7f0c240b7380 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c240d1560 0x7f0c240b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 --2- 172.21.15.97:0/3126550997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c240d0d60 0x7f0c240b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 >> 172.21.15.97:0/3126550997 conn(0x7f0c240d3010 msgr2=0x7f0c2415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 shutdown_connections 2024-01-31T05:01:15.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:15.775+0000 7f0c29200640 1 -- 172.21.15.97:0/3126550997 wait complete. 2024-01-31T05:01:15.791 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-01-31T05:01:15.560002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2038574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004784"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679907+0000","last_change":"2024-01-31T05:00:45.571880+0000","last_active":"2024-01-31T05:00:51.679907+0000","last_peered":"2024-01-31T05:00:51.679907+0000","last_clean":"2024-01-31T05:00:51.679907+0000","last_became_active":"2024-01-31T05:00:45.571633+0000","last_became_peered":"2024-01-31T05:00:45.571633+0000","last_unstale":"2024-01-31T05:00:51.679907+0000","last_undegraded":"2024-01-31T05:00:51.679907+0000","last_fullsized":"2024-01-31T05:00:51.679907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681114+0000","last_change":"2024-01-31T05:00:46.593889+0000","last_active":"2024-01-31T05:00:51.681114+0000","last_peered":"2024-01-31T05:00:51.681114+0000","last_clean":"2024-01-31T05:00:51.681114+0000","last_became_active":"2024-01-31T05:00:46.593583+0000","last_became_peered":"2024-01-31T05:00:46.593583+0000","last_unstale":"2024-01-31T05:00:51.681114+0000","last_undegraded":"2024-01-31T05:00:51.681114+0000","last_fullsized":"2024-01-31T05:00:51.681114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914795+0000","last_change":"2024-01-31T05:00:50.653143+0000","last_active":"2024-01-31T05:00:51.914795+0000","last_peered":"2024-01-31T05:00:51.914795+0000","last_clean":"2024-01-31T05:00:51.914795+0000","last_became_active":"2024-01-31T05:00:50.652831+0000","last_became_peered":"2024-01-31T05:00:50.652831+0000","last_unstale":"2024-01-31T05:00:51.914795+0000","last_undegraded":"2024-01-31T05:00:51.914795+0000","last_fullsized":"2024-01-31T05:00:51.914795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:44.981765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681288+0000","last_change":"2024-01-31T05:00:45.578379+0000","last_active":"2024-01-31T05:00:51.681288+0000","last_peered":"2024-01-31T05:00:51.681288+0000","last_clean":"2024-01-31T05:00:51.681288+0000","last_became_active":"2024-01-31T05:00:45.574393+0000","last_became_peered":"2024-01-31T05:00:45.574393+0000","last_unstale":"2024-01-31T05:00:51.681288+0000","last_undegraded":"2024-01-31T05:00:51.681288+0000","last_fullsized":"2024-01-31T05:00:51.681288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681650+0000","last_change":"2024-01-31T05:00:46.573630+0000","last_active":"2024-01-31T05:00:51.681650+0000","last_peered":"2024-01-31T05:00:51.681650+0000","last_clean":"2024-01-31T05:00:51.681650+0000","last_became_active":"2024-01-31T05:00:46.572199+0000","last_became_peered":"2024-01-31T05:00:46.572199+0000","last_unstale":"2024-01-31T05:00:51.681650+0000","last_undegraded":"2024-01-31T05:00:51.681650+0000","last_fullsized":"2024-01-31T05:00:51.681650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912863+0000","last_change":"2024-01-31T05:00:50.644804+0000","last_active":"2024-01-31T05:00:51.912863+0000","last_peered":"2024-01-31T05:00:51.912863+0000","last_clean":"2024-01-31T05:00:51.912863+0000","last_became_active":"2024-01-31T05:00:50.640815+0000","last_became_peered":"2024-01-31T05:00:50.640815+0000","last_unstale":"2024-01-31T05:00:51.912863+0000","last_undegraded":"2024-01-31T05:00:51.912863+0000","last_fullsized":"2024-01-31T05:00:51.912863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:03:57.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915593+0000","last_change":"2024-01-31T05:00:45.582233+0000","last_active":"2024-01-31T05:00:51.915593+0000","last_peered":"2024-01-31T05:00:51.915593+0000","last_clean":"2024-01-31T05:00:51.915593+0000","last_became_active":"2024-01-31T05:00:45.578094+0000","last_became_peered":"2024-01-31T05:00:45.578094+0000","last_unstale":"2024-01-31T05:00:51.915593+0000","last_undegraded":"2024-01-31T05:00:51.915593+0000","last_fullsized":"2024-01-31T05:00:51.915593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:21:00.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913671+0000","last_change":"2024-01-31T05:00:46.589171+0000","last_active":"2024-01-31T05:00:51.913671+0000","last_peered":"2024-01-31T05:00:51.913671+0000","last_clean":"2024-01-31T05:00:51.913671+0000","last_became_active":"2024-01-31T05:00:46.588749+0000","last_became_peered":"2024-01-31T05:00:46.588749+0000","last_unstale":"2024-01-31T05:00:51.913671+0000","last_undegraded":"2024-01-31T05:00:51.913671+0000","last_fullsized":"2024-01-31T05:00:51.913671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914061+0000","last_change":"2024-01-31T05:00:50.635774+0000","last_active":"2024-01-31T05:00:51.914061+0000","last_peered":"2024-01-31T05:00:51.914061+0000","last_clean":"2024-01-31T05:00:51.914061+0000","last_became_active":"2024-01-31T05:00:50.635404+0000","last_became_peered":"2024-01-31T05:00:50.635404+0000","last_unstale":"2024-01-31T05:00:51.914061+0000","last_undegraded":"2024-01-31T05:00:51.914061+0000","last_fullsized":"2024-01-31T05:00:51.914061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:01:05.575230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915820+0000","last_change":"2024-01-31T05:00:45.583051+0000","last_active":"2024-01-31T05:00:51.915820+0000","last_peered":"2024-01-31T05:00:51.915820+0000","last_clean":"2024-01-31T05:00:51.915820+0000","last_became_active":"2024-01-31T05:00:45.582464+0000","last_became_peered":"2024-01-31T05:00:45.582464+0000","last_unstale":"2024-01-31T05:00:51.915820+0000","last_undegraded":"2024-01-31T05:00:51.915820+0000","last_fullsized":"2024-01-31T05:00:51.915820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:46:05.159804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674284+0000","last_change":"2024-01-31T05:00:46.593796+0000","last_active":"2024-01-31T05:00:51.674284+0000","last_peered":"2024-01-31T05:00:51.674284+0000","last_clean":"2024-01-31T05:00:51.674284+0000","last_became_active":"2024-01-31T05:00:46.593485+0000","last_became_peered":"2024-01-31T05:00:46.593485+0000","last_unstale":"2024-01-31T05:00:51.674284+0000","last_undegraded":"2024-01-31T05:00:51.674284+0000","last_fullsized":"2024-01-31T05:00:51.674284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:51:45.113263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912620+0000","last_change":"2024-01-31T05:00:50.645582+0000","last_active":"2024-01-31T05:00:51.912620+0000","last_peered":"2024-01-31T05:00:51.912620+0000","last_clean":"2024-01-31T05:00:51.912620+0000","last_became_active":"2024-01-31T05:00:50.645198+0000","last_became_peered":"2024-01-31T05:00:50.645198+0000","last_unstale":"2024-01-31T05:00:51.912620+0000","last_undegraded":"2024-01-31T05:00:51.912620+0000","last_fullsized":"2024-01-31T05:00:51.912620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:07:24.039515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673371+0000","last_change":"2024-01-31T05:00:46.581785+0000","last_active":"2024-01-31T05:00:51.673371+0000","last_peered":"2024-01-31T05:00:51.673371+0000","last_clean":"2024-01-31T05:00:51.673371+0000","last_became_active":"2024-01-31T05:00:46.581463+0000","last_became_peered":"2024-01-31T05:00:46.581463+0000","last_unstale":"2024-01-31T05:00:51.673371+0000","last_undegraded":"2024-01-31T05:00:51.673371+0000","last_fullsized":"2024-01-31T05:00:51.673371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:24:26.658435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673762+0000","last_change":"2024-01-31T05:00:45.566782+0000","last_active":"2024-01-31T05:00:51.673762+0000","last_peered":"2024-01-31T05:00:51.673762+0000","last_clean":"2024-01-31T05:00:51.673762+0000","last_became_active":"2024-01-31T05:00:45.565360+0000","last_became_peered":"2024-01-31T05:00:45.565360+0000","last_unstale":"2024-01-31T05:00:51.673762+0000","last_undegraded":"2024-01-31T05:00:51.673762+0000","last_fullsized":"2024-01-31T05:00:51.673762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:45.457514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674294+0000","last_change":"2024-01-31T05:00:50.639468+0000","last_active":"2024-01-31T05:00:51.674294+0000","last_peered":"2024-01-31T05:00:51.674294+0000","last_clean":"2024-01-31T05:00:51.674294+0000","last_became_active":"2024-01-31T05:00:50.639035+0000","last_became_peered":"2024-01-31T05:00:50.639035+0000","last_unstale":"2024-01-31T05:00:51.674294+0000","last_undegraded":"2024-01-31T05:00:51.674294+0000","last_fullsized":"2024-01-31T05:00:51.674294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:43:03.844139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915694+0000","last_change":"2024-01-31T05:00:46.583107+0000","last_active":"2024-01-31T05:00:51.915694+0000","last_peered":"2024-01-31T05:00:51.915694+0000","last_clean":"2024-01-31T05:00:51.915694+0000","last_became_active":"2024-01-31T05:00:46.582506+0000","last_became_peered":"2024-01-31T05:00:46.582506+0000","last_unstale":"2024-01-31T05:00:51.915694+0000","last_undegraded":"2024-01-31T05:00:51.915694+0000","last_fullsized":"2024-01-31T05:00:51.915694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917300+0000","last_change":"2024-01-31T05:00:45.585334+0000","last_active":"2024-01-31T05:00:51.917300+0000","last_peered":"2024-01-31T05:00:51.917300+0000","last_clean":"2024-01-31T05:00:51.917300+0000","last_became_active":"2024-01-31T05:00:45.585034+0000","last_became_peered":"2024-01-31T05:00:45.585034+0000","last_unstale":"2024-01-31T05:00:51.917300+0000","last_undegraded":"2024-01-31T05:00:51.917300+0000","last_fullsized":"2024-01-31T05:00:51.917300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915319+0000","last_change":"2024-01-31T05:00:50.642526+0000","last_active":"2024-01-31T05:00:51.915319+0000","last_peered":"2024-01-31T05:00:51.915319+0000","last_clean":"2024-01-31T05:00:51.915319+0000","last_became_active":"2024-01-31T05:00:50.642120+0000","last_became_peered":"2024-01-31T05:00:50.642120+0000","last_unstale":"2024-01-31T05:00:51.915319+0000","last_undegraded":"2024-01-31T05:00:51.915319+0000","last_fullsized":"2024-01-31T05:00:51.915319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674674+0000","last_change":"2024-01-31T05:00:46.585812+0000","last_active":"2024-01-31T05:00:51.674674+0000","last_peered":"2024-01-31T05:00:51.674674+0000","last_clean":"2024-01-31T05:00:51.674674+0000","last_became_active":"2024-01-31T05:00:46.585556+0000","last_became_peered":"2024-01-31T05:00:46.585556+0000","last_unstale":"2024-01-31T05:00:51.674674+0000","last_undegraded":"2024-01-31T05:00:51.674674+0000","last_fullsized":"2024-01-31T05:00:51.674674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:37.158256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676683+0000","last_change":"2024-01-31T05:00:45.580944+0000","last_active":"2024-01-31T05:00:51.676683+0000","last_peered":"2024-01-31T05:00:51.676683+0000","last_clean":"2024-01-31T05:00:51.676683+0000","last_became_active":"2024-01-31T05:00:45.580698+0000","last_became_peered":"2024-01-31T05:00:45.580698+0000","last_unstale":"2024-01-31T05:00:51.676683+0000","last_undegraded":"2024-01-31T05:00:51.676683+0000","last_fullsized":"2024-01-31T05:00:51.676683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:57.421278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916224+0000","last_change":"2024-01-31T05:00:50.659576+0000","last_active":"2024-01-31T05:00:51.916224+0000","last_peered":"2024-01-31T05:00:51.916224+0000","last_clean":"2024-01-31T05:00:51.916224+0000","last_became_active":"2024-01-31T05:00:50.659067+0000","last_became_peered":"2024-01-31T05:00:50.659067+0000","last_unstale":"2024-01-31T05:00:51.916224+0000","last_undegraded":"2024-01-31T05:00:51.916224+0000","last_fullsized":"2024-01-31T05:00:51.916224+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:11.777909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673016+0000","last_change":"2024-01-31T05:00:46.586716+0000","last_active":"2024-01-31T05:00:51.673016+0000","last_peered":"2024-01-31T05:00:51.673016+0000","last_clean":"2024-01-31T05:00:51.673016+0000","last_became_active":"2024-01-31T05:00:46.582889+0000","last_became_peered":"2024-01-31T05:00:46.582889+0000","last_unstale":"2024-01-31T05:00:51.673016+0000","last_undegraded":"2024-01-31T05:00:51.673016+0000","last_fullsized":"2024-01-31T05:00:51.673016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:07.424800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.572569+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.571841+0000","last_became_peered":"2024-01-31T05:00:45.571841+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914775+0000","last_change":"2024-01-31T05:00:50.637862+0000","last_active":"2024-01-31T05:00:51.914775+0000","last_peered":"2024-01-31T05:00:51.914775+0000","last_clean":"2024-01-31T05:00:51.914775+0000","last_became_active":"2024-01-31T05:00:50.637439+0000","last_became_peered":"2024-01-31T05:00:50.637439+0000","last_unstale":"2024-01-31T05:00:51.914775+0000","last_undegraded":"2024-01-31T05:00:51.914775+0000","last_fullsized":"2024-01-31T05:00:51.914775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676808+0000","last_change":"2024-01-31T05:00:46.588199+0000","last_active":"2024-01-31T05:00:51.676808+0000","last_peered":"2024-01-31T05:00:51.676808+0000","last_clean":"2024-01-31T05:00:51.676808+0000","last_became_active":"2024-01-31T05:00:46.585236+0000","last_became_peered":"2024-01-31T05:00:46.585236+0000","last_unstale":"2024-01-31T05:00:51.676808+0000","last_undegraded":"2024-01-31T05:00:51.676808+0000","last_fullsized":"2024-01-31T05:00:51.676808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:16.863565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675061+0000","last_change":"2024-01-31T05:00:45.577787+0000","last_active":"2024-01-31T05:00:51.675061+0000","last_peered":"2024-01-31T05:00:51.675061+0000","last_clean":"2024-01-31T05:00:51.675061+0000","last_became_active":"2024-01-31T05:00:45.577538+0000","last_became_peered":"2024-01-31T05:00:45.577538+0000","last_unstale":"2024-01-31T05:00:51.675061+0000","last_undegraded":"2024-01-31T05:00:51.675061+0000","last_fullsized":"2024-01-31T05:00:51.675061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:15.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912746+0000","last_change":"2024-01-31T05:00:50.630730+0000","last_active":"2024-01-31T05:00:51.912746+0000","last_peered":"2024-01-31T05:00:51.912746+0000","last_clean":"2024-01-31T05:00:51.912746+0000","last_became_active":"2024-01-31T05:00:50.629780+0000","last_became_peered":"2024-01-31T05:00:50.629780+0000","last_unstale":"2024-01-31T05:00:51.912746+0000","last_undegraded":"2024-01-31T05:00:51.912746+0000","last_fullsized":"2024-01-31T05:00:51.912746+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:07:58.674398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679833+0000","last_change":"2024-01-31T05:00:46.583111+0000","last_active":"2024-01-31T05:00:51.679833+0000","last_peered":"2024-01-31T05:00:51.679833+0000","last_clean":"2024-01-31T05:00:51.679833+0000","last_became_active":"2024-01-31T05:00:46.582827+0000","last_became_peered":"2024-01-31T05:00:46.582827+0000","last_unstale":"2024-01-31T05:00:51.679833+0000","last_undegraded":"2024-01-31T05:00:51.679833+0000","last_fullsized":"2024-01-31T05:00:51.679833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:40:42.367021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679389+0000","last_change":"2024-01-31T05:00:45.580550+0000","last_active":"2024-01-31T05:00:51.679389+0000","last_peered":"2024-01-31T05:00:51.679389+0000","last_clean":"2024-01-31T05:00:51.679389+0000","last_became_active":"2024-01-31T05:00:45.579549+0000","last_became_peered":"2024-01-31T05:00:45.579549+0000","last_unstale":"2024-01-31T05:00:51.679389+0000","last_undegraded":"2024-01-31T05:00:51.679389+0000","last_fullsized":"2024-01-31T05:00:51.679389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:43.683478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674795+0000","last_change":"2024-01-31T05:00:50.649333+0000","last_active":"2024-01-31T05:00:51.674795+0000","last_peered":"2024-01-31T05:00:51.674795+0000","last_clean":"2024-01-31T05:00:51.674795+0000","last_became_active":"2024-01-31T05:00:50.648927+0000","last_became_peered":"2024-01-31T05:00:50.648927+0000","last_unstale":"2024-01-31T05:00:51.674795+0000","last_undegraded":"2024-01-31T05:00:51.674795+0000","last_fullsized":"2024-01-31T05:00:51.674795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918351+0000","last_change":"2024-01-31T05:00:46.581800+0000","last_active":"2024-01-31T05:00:51.918351+0000","last_peered":"2024-01-31T05:00:51.918351+0000","last_clean":"2024-01-31T05:00:51.918351+0000","last_became_active":"2024-01-31T05:00:46.581435+0000","last_became_peered":"2024-01-31T05:00:46.581435+0000","last_unstale":"2024-01-31T05:00:51.918351+0000","last_undegraded":"2024-01-31T05:00:51.918351+0000","last_fullsized":"2024-01-31T05:00:51.918351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:55:29.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674998+0000","last_change":"2024-01-31T05:00:45.570866+0000","last_active":"2024-01-31T05:00:51.674998+0000","last_peered":"2024-01-31T05:00:51.674998+0000","last_clean":"2024-01-31T05:00:51.674998+0000","last_became_active":"2024-01-31T05:00:45.570644+0000","last_became_peered":"2024-01-31T05:00:45.570644+0000","last_unstale":"2024-01-31T05:00:51.674998+0000","last_undegraded":"2024-01-31T05:00:51.674998+0000","last_fullsized":"2024-01-31T05:00:51.674998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:54.101266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917676+0000","last_change":"2024-01-31T05:00:50.641145+0000","last_active":"2024-01-31T05:00:51.917676+0000","last_peered":"2024-01-31T05:00:51.917676+0000","last_clean":"2024-01-31T05:00:51.917676+0000","last_became_active":"2024-01-31T05:00:50.638626+0000","last_became_peered":"2024-01-31T05:00:50.638626+0000","last_unstale":"2024-01-31T05:00:51.917676+0000","last_undegraded":"2024-01-31T05:00:51.917676+0000","last_fullsized":"2024-01-31T05:00:51.917676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:46:01.447285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914572+0000","last_change":"2024-01-31T05:00:46.589437+0000","last_active":"2024-01-31T05:00:51.914572+0000","last_peered":"2024-01-31T05:00:51.914572+0000","last_clean":"2024-01-31T05:00:51.914572+0000","last_became_active":"2024-01-31T05:00:46.589134+0000","last_became_peered":"2024-01-31T05:00:46.589134+0000","last_unstale":"2024-01-31T05:00:51.914572+0000","last_undegraded":"2024-01-31T05:00:51.914572+0000","last_fullsized":"2024-01-31T05:00:51.914572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680322+0000","last_change":"2024-01-31T05:00:45.579033+0000","last_active":"2024-01-31T05:00:51.680322+0000","last_peered":"2024-01-31T05:00:51.680322+0000","last_clean":"2024-01-31T05:00:51.680322+0000","last_became_active":"2024-01-31T05:00:45.578730+0000","last_became_peered":"2024-01-31T05:00:45.578730+0000","last_unstale":"2024-01-31T05:00:51.680322+0000","last_undegraded":"2024-01-31T05:00:51.680322+0000","last_fullsized":"2024-01-31T05:00:51.680322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916714+0000","last_change":"2024-01-31T05:00:50.639581+0000","last_active":"2024-01-31T05:00:51.916714+0000","last_peered":"2024-01-31T05:00:51.916714+0000","last_clean":"2024-01-31T05:00:51.916714+0000","last_became_active":"2024-01-31T05:00:50.639192+0000","last_became_peered":"2024-01-31T05:00:50.639192+0000","last_unstale":"2024-01-31T05:00:51.916714+0000","last_undegraded":"2024-01-31T05:00:51.916714+0000","last_fullsized":"2024-01-31T05:00:51.916714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:28.385448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915081+0000","last_change":"2024-01-31T05:00:46.590496+0000","last_active":"2024-01-31T05:00:51.915081+0000","last_peered":"2024-01-31T05:00:51.915081+0000","last_clean":"2024-01-31T05:00:51.915081+0000","last_became_active":"2024-01-31T05:00:46.587386+0000","last_became_peered":"2024-01-31T05:00:46.587386+0000","last_unstale":"2024-01-31T05:00:51.915081+0000","last_undegraded":"2024-01-31T05:00:51.915081+0000","last_fullsized":"2024-01-31T05:00:51.915081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674054+0000","last_change":"2024-01-31T05:00:45.580484+0000","last_active":"2024-01-31T05:00:51.674054+0000","last_peered":"2024-01-31T05:00:51.674054+0000","last_clean":"2024-01-31T05:00:51.674054+0000","last_became_active":"2024-01-31T05:00:45.580179+0000","last_became_peered":"2024-01-31T05:00:45.580179+0000","last_unstale":"2024-01-31T05:00:51.674054+0000","last_undegraded":"2024-01-31T05:00:51.674054+0000","last_fullsized":"2024-01-31T05:00:51.674054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:54.780283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918220+0000","last_change":"2024-01-31T05:00:50.633651+0000","last_active":"2024-01-31T05:00:51.918220+0000","last_peered":"2024-01-31T05:00:51.918220+0000","last_clean":"2024-01-31T05:00:51.918220+0000","last_became_active":"2024-01-31T05:00:50.633176+0000","last_became_peered":"2024-01-31T05:00:50.633176+0000","last_unstale":"2024-01-31T05:00:51.918220+0000","last_undegraded":"2024-01-31T05:00:51.918220+0000","last_fullsized":"2024-01-31T05:00:51.918220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:41.984643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675265+0000","last_change":"2024-01-31T05:00:46.578636+0000","last_active":"2024-01-31T05:00:51.675265+0000","last_peered":"2024-01-31T05:00:51.675265+0000","last_clean":"2024-01-31T05:00:51.675265+0000","last_became_active":"2024-01-31T05:00:46.577232+0000","last_became_peered":"2024-01-31T05:00:46.577232+0000","last_unstale":"2024-01-31T05:00:51.675265+0000","last_undegraded":"2024-01-31T05:00:51.675265+0000","last_fullsized":"2024-01-31T05:00:51.675265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:37:22.658839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915084+0000","last_change":"2024-01-31T05:00:45.574367+0000","last_active":"2024-01-31T05:00:51.915084+0000","last_peered":"2024-01-31T05:00:51.915084+0000","last_clean":"2024-01-31T05:00:51.915084+0000","last_became_active":"2024-01-31T05:00:45.574010+0000","last_became_peered":"2024-01-31T05:00:45.574010+0000","last_unstale":"2024-01-31T05:00:51.915084+0000","last_undegraded":"2024-01-31T05:00:51.915084+0000","last_fullsized":"2024-01-31T05:00:51.915084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:12:03.585718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918910+0000","last_change":"2024-01-31T05:00:50.642311+0000","last_active":"2024-01-31T05:00:51.918910+0000","last_peered":"2024-01-31T05:00:51.918910+0000","last_clean":"2024-01-31T05:00:51.918910+0000","last_became_active":"2024-01-31T05:00:50.641937+0000","last_became_peered":"2024-01-31T05:00:50.641937+0000","last_unstale":"2024-01-31T05:00:51.918910+0000","last_undegraded":"2024-01-31T05:00:51.918910+0000","last_fullsized":"2024-01-31T05:00:51.918910+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:57.841272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916940+0000","last_change":"2024-01-31T05:00:46.589298+0000","last_active":"2024-01-31T05:00:51.916940+0000","last_peered":"2024-01-31T05:00:51.916940+0000","last_clean":"2024-01-31T05:00:51.916940+0000","last_became_active":"2024-01-31T05:00:46.589007+0000","last_became_peered":"2024-01-31T05:00:46.589007+0000","last_unstale":"2024-01-31T05:00:51.916940+0000","last_undegraded":"2024-01-31T05:00:51.916940+0000","last_fullsized":"2024-01-31T05:00:51.916940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:53:18.216173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673208+0000","last_change":"2024-01-31T05:00:45.573415+0000","last_active":"2024-01-31T05:00:51.673208+0000","last_peered":"2024-01-31T05:00:51.673208+0000","last_clean":"2024-01-31T05:00:51.673208+0000","last_became_active":"2024-01-31T05:00:45.573080+0000","last_became_peered":"2024-01-31T05:00:45.573080+0000","last_unstale":"2024-01-31T05:00:51.673208+0000","last_undegraded":"2024-01-31T05:00:51.673208+0000","last_fullsized":"2024-01-31T05:00:51.673208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:22.222326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680907+0000","last_change":"2024-01-31T05:00:50.642960+0000","last_active":"2024-01-31T05:00:51.680907+0000","last_peered":"2024-01-31T05:00:51.680907+0000","last_clean":"2024-01-31T05:00:51.680907+0000","last_became_active":"2024-01-31T05:00:50.640025+0000","last_became_peered":"2024-01-31T05:00:50.640025+0000","last_unstale":"2024-01-31T05:00:51.680907+0000","last_undegraded":"2024-01-31T05:00:51.680907+0000","last_fullsized":"2024-01-31T05:00:51.680907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:28:26.315820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677292+0000","last_change":"2024-01-31T05:00:46.592032+0000","last_active":"2024-01-31T05:00:51.677292+0000","last_peered":"2024-01-31T05:00:51.677292+0000","last_clean":"2024-01-31T05:00:51.677292+0000","last_became_active":"2024-01-31T05:00:46.588029+0000","last_became_peered":"2024-01-31T05:00:46.588029+0000","last_unstale":"2024-01-31T05:00:51.677292+0000","last_undegraded":"2024-01-31T05:00:51.677292+0000","last_fullsized":"2024-01-31T05:00:51.677292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:42.890080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673909+0000","last_change":"2024-01-31T05:00:45.562238+0000","last_active":"2024-01-31T05:00:51.673909+0000","last_peered":"2024-01-31T05:00:51.673909+0000","last_clean":"2024-01-31T05:00:51.673909+0000","last_became_active":"2024-01-31T05:00:45.561726+0000","last_became_peered":"2024-01-31T05:00:45.561726+0000","last_unstale":"2024-01-31T05:00:51.673909+0000","last_undegraded":"2024-01-31T05:00:51.673909+0000","last_fullsized":"2024-01-31T05:00:51.673909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:13.468671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912656+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:00:51.912656+0000","last_peered":"2024-01-31T05:00:51.912656+0000","last_clean":"2024-01-31T05:00:51.912656+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:00:51.912656+0000","last_undegraded":"2024-01-31T05:00:51.912656+0000","last_fullsized":"2024-01-31T05:00:51.912656+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674754+0000","last_change":"2024-01-31T05:00:50.629234+0000","last_active":"2024-01-31T05:00:51.674754+0000","last_peered":"2024-01-31T05:00:51.674754+0000","last_clean":"2024-01-31T05:00:51.674754+0000","last_became_active":"2024-01-31T05:00:50.628308+0000","last_became_peered":"2024-01-31T05:00:50.628308+0000","last_unstale":"2024-01-31T05:00:51.674754+0000","last_undegraded":"2024-01-31T05:00:51.674754+0000","last_fullsized":"2024-01-31T05:00:51.674754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:21:35.300095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674590+0000","last_change":"2024-01-31T05:00:46.583227+0000","last_active":"2024-01-31T05:00:51.674590+0000","last_peered":"2024-01-31T05:00:51.674590+0000","last_clean":"2024-01-31T05:00:51.674590+0000","last_became_active":"2024-01-31T05:00:46.582532+0000","last_became_peered":"2024-01-31T05:00:46.582532+0000","last_unstale":"2024-01-31T05:00:51.674590+0000","last_undegraded":"2024-01-31T05:00:51.674590+0000","last_fullsized":"2024-01-31T05:00:51.674590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:53:51.592799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918513+0000","last_change":"2024-01-31T05:00:45.583848+0000","last_active":"2024-01-31T05:00:51.918513+0000","last_peered":"2024-01-31T05:00:51.918513+0000","last_clean":"2024-01-31T05:00:51.918513+0000","last_became_active":"2024-01-31T05:00:45.583492+0000","last_became_peered":"2024-01-31T05:00:45.583492+0000","last_unstale":"2024-01-31T05:00:51.918513+0000","last_undegraded":"2024-01-31T05:00:51.918513+0000","last_fullsized":"2024-01-31T05:00:51.918513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912922+0000","last_change":"2024-01-31T05:00:50.642193+0000","last_active":"2024-01-31T05:00:51.912922+0000","last_peered":"2024-01-31T05:00:51.912922+0000","last_clean":"2024-01-31T05:00:51.912922+0000","last_became_active":"2024-01-31T05:00:50.639911+0000","last_became_peered":"2024-01-31T05:00:50.639911+0000","last_unstale":"2024-01-31T05:00:51.912922+0000","last_undegraded":"2024-01-31T05:00:51.912922+0000","last_fullsized":"2024-01-31T05:00:51.912922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:50:44.744734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915799+0000","last_change":"2024-01-31T05:00:46.580861+0000","last_active":"2024-01-31T05:00:51.915799+0000","last_peered":"2024-01-31T05:00:51.915799+0000","last_clean":"2024-01-31T05:00:51.915799+0000","last_became_active":"2024-01-31T05:00:46.580588+0000","last_became_peered":"2024-01-31T05:00:46.580588+0000","last_unstale":"2024-01-31T05:00:51.915799+0000","last_undegraded":"2024-01-31T05:00:51.915799+0000","last_fullsized":"2024-01-31T05:00:51.915799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681122+0000","last_change":"2024-01-31T05:00:45.567443+0000","last_active":"2024-01-31T05:00:51.681122+0000","last_peered":"2024-01-31T05:00:51.681122+0000","last_clean":"2024-01-31T05:00:51.681122+0000","last_became_active":"2024-01-31T05:00:45.567160+0000","last_became_peered":"2024-01-31T05:00:45.567160+0000","last_unstale":"2024-01-31T05:00:51.681122+0000","last_undegraded":"2024-01-31T05:00:51.681122+0000","last_fullsized":"2024-01-31T05:00:51.681122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679437+0000","last_change":"2024-01-31T05:00:50.647210+0000","last_active":"2024-01-31T05:00:51.679437+0000","last_peered":"2024-01-31T05:00:51.679437+0000","last_clean":"2024-01-31T05:00:51.679437+0000","last_became_active":"2024-01-31T05:00:50.646792+0000","last_became_peered":"2024-01-31T05:00:50.646792+0000","last_unstale":"2024-01-31T05:00:51.679437+0000","last_undegraded":"2024-01-31T05:00:51.679437+0000","last_fullsized":"2024-01-31T05:00:51.679437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915671+0000","last_change":"2024-01-31T05:00:46.593737+0000","last_active":"2024-01-31T05:00:51.915671+0000","last_peered":"2024-01-31T05:00:51.915671+0000","last_clean":"2024-01-31T05:00:51.915671+0000","last_became_active":"2024-01-31T05:00:46.593484+0000","last_became_peered":"2024-01-31T05:00:46.593484+0000","last_unstale":"2024-01-31T05:00:51.915671+0000","last_undegraded":"2024-01-31T05:00:51.915671+0000","last_fullsized":"2024-01-31T05:00:51.915671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:38:14.536828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679315+0000","last_change":"2024-01-31T05:00:45.582469+0000","last_active":"2024-01-31T05:00:51.679315+0000","last_peered":"2024-01-31T05:00:51.679315+0000","last_clean":"2024-01-31T05:00:51.679315+0000","last_became_active":"2024-01-31T05:00:45.582151+0000","last_became_peered":"2024-01-31T05:00:45.582151+0000","last_unstale":"2024-01-31T05:00:51.679315+0000","last_undegraded":"2024-01-31T05:00:51.679315+0000","last_fullsized":"2024-01-31T05:00:51.679315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:36:17.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672556+0000","last_change":"2024-01-31T05:00:50.642993+0000","last_active":"2024-01-31T05:00:51.672556+0000","last_peered":"2024-01-31T05:00:51.672556+0000","last_clean":"2024-01-31T05:00:51.672556+0000","last_became_active":"2024-01-31T05:00:50.642558+0000","last_became_peered":"2024-01-31T05:00:50.642558+0000","last_unstale":"2024-01-31T05:00:51.672556+0000","last_undegraded":"2024-01-31T05:00:51.672556+0000","last_fullsized":"2024-01-31T05:00:51.672556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:07:54.060532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914870+0000","last_change":"2024-01-31T05:00:46.597235+0000","last_active":"2024-01-31T05:00:51.914870+0000","last_peered":"2024-01-31T05:00:51.914870+0000","last_clean":"2024-01-31T05:00:51.914870+0000","last_became_active":"2024-01-31T05:00:46.592597+0000","last_became_peered":"2024-01-31T05:00:46.592597+0000","last_unstale":"2024-01-31T05:00:51.914870+0000","last_undegraded":"2024-01-31T05:00:51.914870+0000","last_fullsized":"2024-01-31T05:00:51.914870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:55.281235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915788+0000","last_change":"2024-01-31T05:00:45.583433+0000","last_active":"2024-01-31T05:00:51.915788+0000","last_peered":"2024-01-31T05:00:51.915788+0000","last_clean":"2024-01-31T05:00:51.915788+0000","last_became_active":"2024-01-31T05:00:45.582873+0000","last_became_peered":"2024-01-31T05:00:45.582873+0000","last_unstale":"2024-01-31T05:00:51.915788+0000","last_undegraded":"2024-01-31T05:00:51.915788+0000","last_fullsized":"2024-01-31T05:00:51.915788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:44:30.759721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913367+0000","last_change":"2024-01-31T05:00:50.651478+0000","last_active":"2024-01-31T05:00:51.913367+0000","last_peered":"2024-01-31T05:00:51.913367+0000","last_clean":"2024-01-31T05:00:51.913367+0000","last_became_active":"2024-01-31T05:00:50.650775+0000","last_became_peered":"2024-01-31T05:00:50.650775+0000","last_unstale":"2024-01-31T05:00:51.913367+0000","last_undegraded":"2024-01-31T05:00:51.913367+0000","last_fullsized":"2024-01-31T05:00:51.913367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672742+0000","last_change":"2024-01-31T05:00:46.588386+0000","last_active":"2024-01-31T05:00:51.672742+0000","last_peered":"2024-01-31T05:00:51.672742+0000","last_clean":"2024-01-31T05:00:51.672742+0000","last_became_active":"2024-01-31T05:00:46.588150+0000","last_became_peered":"2024-01-31T05:00:46.588150+0000","last_unstale":"2024-01-31T05:00:51.672742+0000","last_undegraded":"2024-01-31T05:00:51.672742+0000","last_fullsized":"2024-01-31T05:00:51.672742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:38.932253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674904+0000","last_change":"2024-01-31T05:00:45.572910+0000","last_active":"2024-01-31T05:00:51.674904+0000","last_peered":"2024-01-31T05:00:51.674904+0000","last_clean":"2024-01-31T05:00:51.674904+0000","last_became_active":"2024-01-31T05:00:45.570281+0000","last_became_peered":"2024-01-31T05:00:45.570281+0000","last_unstale":"2024-01-31T05:00:51.674904+0000","last_undegraded":"2024-01-31T05:00:51.674904+0000","last_fullsized":"2024-01-31T05:00:51.674904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681309+0000","last_change":"2024-01-31T05:00:50.646215+0000","last_active":"2024-01-31T05:00:51.681309+0000","last_peered":"2024-01-31T05:00:51.681309+0000","last_clean":"2024-01-31T05:00:51.681309+0000","last_became_active":"2024-01-31T05:00:50.640453+0000","last_became_peered":"2024-01-31T05:00:50.640453+0000","last_unstale":"2024-01-31T05:00:51.681309+0000","last_undegraded":"2024-01-31T05:00:51.681309+0000","last_fullsized":"2024-01-31T05:00:51.681309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:20.595164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912715+0000","last_change":"2024-01-31T05:00:46.582233+0000","last_active":"2024-01-31T05:00:51.912715+0000","last_peered":"2024-01-31T05:00:51.912715+0000","last_clean":"2024-01-31T05:00:51.912715+0000","last_became_active":"2024-01-31T05:00:46.581941+0000","last_became_peered":"2024-01-31T05:00:46.581941+0000","last_unstale":"2024-01-31T05:00:51.912715+0000","last_undegraded":"2024-01-31T05:00:51.912715+0000","last_fullsized":"2024-01-31T05:00:51.912715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914727+0000","last_change":"2024-01-31T05:00:45.586099+0000","last_active":"2024-01-31T05:00:51.914727+0000","last_peered":"2024-01-31T05:00:51.914727+0000","last_clean":"2024-01-31T05:00:51.914727+0000","last_became_active":"2024-01-31T05:00:45.583149+0000","last_became_peered":"2024-01-31T05:00:45.583149+0000","last_unstale":"2024-01-31T05:00:51.914727+0000","last_undegraded":"2024-01-31T05:00:51.914727+0000","last_fullsized":"2024-01-31T05:00:51.914727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681581+0000","last_change":"2024-01-31T05:00:50.644931+0000","last_active":"2024-01-31T05:00:51.681581+0000","last_peered":"2024-01-31T05:00:51.681581+0000","last_clean":"2024-01-31T05:00:51.681581+0000","last_became_active":"2024-01-31T05:00:50.644554+0000","last_became_peered":"2024-01-31T05:00:50.644554+0000","last_unstale":"2024-01-31T05:00:51.681581+0000","last_undegraded":"2024-01-31T05:00:51.681581+0000","last_fullsized":"2024-01-31T05:00:51.681581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:23:08.930803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673424+0000","last_change":"2024-01-31T05:00:46.582885+0000","last_active":"2024-01-31T05:00:51.673424+0000","last_peered":"2024-01-31T05:00:51.673424+0000","last_clean":"2024-01-31T05:00:51.673424+0000","last_became_active":"2024-01-31T05:00:46.582585+0000","last_became_peered":"2024-01-31T05:00:46.582585+0000","last_unstale":"2024-01-31T05:00:51.673424+0000","last_undegraded":"2024-01-31T05:00:51.673424+0000","last_fullsized":"2024-01-31T05:00:51.673424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:15:39.727609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673905+0000","last_change":"2024-01-31T05:00:45.570263+0000","last_active":"2024-01-31T05:00:51.673905+0000","last_peered":"2024-01-31T05:00:51.673905+0000","last_clean":"2024-01-31T05:00:51.673905+0000","last_became_active":"2024-01-31T05:00:45.568414+0000","last_became_peered":"2024-01-31T05:00:45.568414+0000","last_unstale":"2024-01-31T05:00:51.673905+0000","last_undegraded":"2024-01-31T05:00:51.673905+0000","last_fullsized":"2024-01-31T05:00:51.673905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:02.658111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918156+0000","last_change":"2024-01-31T05:00:50.640753+0000","last_active":"2024-01-31T05:00:51.918156+0000","last_peered":"2024-01-31T05:00:51.918156+0000","last_clean":"2024-01-31T05:00:51.918156+0000","last_became_active":"2024-01-31T05:00:50.640179+0000","last_became_peered":"2024-01-31T05:00:50.640179+0000","last_unstale":"2024-01-31T05:00:51.918156+0000","last_undegraded":"2024-01-31T05:00:51.918156+0000","last_fullsized":"2024-01-31T05:00:51.918156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:39.555756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916848+0000","last_change":"2024-01-31T05:00:46.596210+0000","last_active":"2024-01-31T05:00:51.916848+0000","last_peered":"2024-01-31T05:00:51.916848+0000","last_clean":"2024-01-31T05:00:51.916848+0000","last_became_active":"2024-01-31T05:00:46.595941+0000","last_became_peered":"2024-01-31T05:00:46.595941+0000","last_unstale":"2024-01-31T05:00:51.916848+0000","last_undegraded":"2024-01-31T05:00:51.916848+0000","last_fullsized":"2024-01-31T05:00:51.916848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:09.854108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916254+0000","last_change":"2024-01-31T05:00:45.577952+0000","last_active":"2024-01-31T05:00:51.916254+0000","last_peered":"2024-01-31T05:00:51.916254+0000","last_clean":"2024-01-31T05:00:51.916254+0000","last_became_active":"2024-01-31T05:00:45.577632+0000","last_became_peered":"2024-01-31T05:00:45.577632+0000","last_unstale":"2024-01-31T05:00:51.916254+0000","last_undegraded":"2024-01-31T05:00:51.916254+0000","last_fullsized":"2024-01-31T05:00:51.916254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673947+0000","last_change":"2024-01-31T05:00:50.639630+0000","last_active":"2024-01-31T05:00:51.673947+0000","last_peered":"2024-01-31T05:00:51.673947+0000","last_clean":"2024-01-31T05:00:51.673947+0000","last_became_active":"2024-01-31T05:00:50.639187+0000","last_became_peered":"2024-01-31T05:00:50.639187+0000","last_unstale":"2024-01-31T05:00:51.673947+0000","last_undegraded":"2024-01-31T05:00:51.673947+0000","last_fullsized":"2024-01-31T05:00:51.673947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:41:38.465126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915506+0000","last_change":"2024-01-31T05:00:46.595225+0000","last_active":"2024-01-31T05:00:51.915506+0000","last_peered":"2024-01-31T05:00:51.915506+0000","last_clean":"2024-01-31T05:00:51.915506+0000","last_became_active":"2024-01-31T05:00:46.594892+0000","last_became_peered":"2024-01-31T05:00:46.594892+0000","last_unstale":"2024-01-31T05:00:51.915506+0000","last_undegraded":"2024-01-31T05:00:51.915506+0000","last_fullsized":"2024-01-31T05:00:51.915506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:01:05.347110+0000","last_change":"2024-01-31T05:00:45.575073+0000","last_active":"2024-01-31T05:01:05.347110+0000","last_peered":"2024-01-31T05:01:05.347110+0000","last_clean":"2024-01-31T05:01:05.347110+0000","last_became_active":"2024-01-31T05:00:45.574725+0000","last_became_peered":"2024-01-31T05:00:45.574725+0000","last_unstale":"2024-01-31T05:01:05.347110+0000","last_undegraded":"2024-01-31T05:01:05.347110+0000","last_fullsized":"2024-01-31T05:01:05.347110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674916+0000","last_change":"2024-01-31T05:00:50.640744+0000","last_active":"2024-01-31T05:00:51.674916+0000","last_peered":"2024-01-31T05:00:51.674916+0000","last_clean":"2024-01-31T05:00:51.674916+0000","last_became_active":"2024-01-31T05:00:50.640181+0000","last_became_peered":"2024-01-31T05:00:50.640181+0000","last_unstale":"2024-01-31T05:00:51.674916+0000","last_undegraded":"2024-01-31T05:00:51.674916+0000","last_fullsized":"2024-01-31T05:00:51.674916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673600+0000","last_change":"2024-01-31T05:00:46.587774+0000","last_active":"2024-01-31T05:00:51.673600+0000","last_peered":"2024-01-31T05:00:51.673600+0000","last_clean":"2024-01-31T05:00:51.673600+0000","last_became_active":"2024-01-31T05:00:46.587495+0000","last_became_peered":"2024-01-31T05:00:46.587495+0000","last_unstale":"2024-01-31T05:00:51.673600+0000","last_undegraded":"2024-01-31T05:00:51.673600+0000","last_fullsized":"2024-01-31T05:00:51.673600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:19:31.923714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917086+0000","last_change":"2024-01-31T05:00:45.577387+0000","last_active":"2024-01-31T05:00:51.917086+0000","last_peered":"2024-01-31T05:00:51.917086+0000","last_clean":"2024-01-31T05:00:51.917086+0000","last_became_active":"2024-01-31T05:00:45.577035+0000","last_became_peered":"2024-01-31T05:00:45.577035+0000","last_unstale":"2024-01-31T05:00:51.917086+0000","last_undegraded":"2024-01-31T05:00:51.917086+0000","last_fullsized":"2024-01-31T05:00:51.917086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914130+0000","last_change":"2024-01-31T05:00:50.641531+0000","last_active":"2024-01-31T05:00:51.914130+0000","last_peered":"2024-01-31T05:00:51.914130+0000","last_clean":"2024-01-31T05:00:51.914130+0000","last_became_active":"2024-01-31T05:00:50.641164+0000","last_became_peered":"2024-01-31T05:00:50.641164+0000","last_unstale":"2024-01-31T05:00:51.914130+0000","last_undegraded":"2024-01-31T05:00:51.914130+0000","last_fullsized":"2024-01-31T05:00:51.914130+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681764+0000","last_change":"2024-01-31T05:00:46.584219+0000","last_active":"2024-01-31T05:00:51.681764+0000","last_peered":"2024-01-31T05:00:51.681764+0000","last_clean":"2024-01-31T05:00:51.681764+0000","last_became_active":"2024-01-31T05:00:46.583942+0000","last_became_peered":"2024-01-31T05:00:46.583942+0000","last_unstale":"2024-01-31T05:00:51.681764+0000","last_undegraded":"2024-01-31T05:00:51.681764+0000","last_fullsized":"2024-01-31T05:00:51.681764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915433+0000","last_change":"2024-01-31T05:00:45.576533+0000","last_active":"2024-01-31T05:00:51.915433+0000","last_peered":"2024-01-31T05:00:51.915433+0000","last_clean":"2024-01-31T05:00:51.915433+0000","last_became_active":"2024-01-31T05:00:45.576327+0000","last_became_peered":"2024-01-31T05:00:45.576327+0000","last_unstale":"2024-01-31T05:00:51.915433+0000","last_undegraded":"2024-01-31T05:00:51.915433+0000","last_fullsized":"2024-01-31T05:00:51.915433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:09:58.811691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914596+0000","last_change":"2024-01-31T05:00:50.654987+0000","last_active":"2024-01-31T05:00:51.914596+0000","last_peered":"2024-01-31T05:00:51.914596+0000","last_clean":"2024-01-31T05:00:51.914596+0000","last_became_active":"2024-01-31T05:00:50.654461+0000","last_became_peered":"2024-01-31T05:00:50.654461+0000","last_unstale":"2024-01-31T05:00:51.914596+0000","last_undegraded":"2024-01-31T05:00:51.914596+0000","last_fullsized":"2024-01-31T05:00:51.914596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914829+0000","last_change":"2024-01-31T05:00:46.597875+0000","last_active":"2024-01-31T05:00:51.914829+0000","last_peered":"2024-01-31T05:00:51.914829+0000","last_clean":"2024-01-31T05:00:51.914829+0000","last_became_active":"2024-01-31T05:00:46.597494+0000","last_became_peered":"2024-01-31T05:00:46.597494+0000","last_unstale":"2024-01-31T05:00:51.914829+0000","last_undegraded":"2024-01-31T05:00:51.914829+0000","last_fullsized":"2024-01-31T05:00:51.914829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:14.648640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.574121+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.573715+0000","last_became_peered":"2024-01-31T05:00:45.573715+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676164+0000","last_change":"2024-01-31T05:00:50.642176+0000","last_active":"2024-01-31T05:00:51.676164+0000","last_peered":"2024-01-31T05:00:51.676164+0000","last_clean":"2024-01-31T05:00:51.676164+0000","last_became_active":"2024-01-31T05:00:50.639134+0000","last_became_peered":"2024-01-31T05:00:50.639134+0000","last_unstale":"2024-01-31T05:00:51.676164+0000","last_undegraded":"2024-01-31T05:00:51.676164+0000","last_fullsized":"2024-01-31T05:00:51.676164+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:11.485002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676564+0000","last_change":"2024-01-31T05:00:50.649633+0000","last_active":"2024-01-31T05:00:51.676564+0000","last_peered":"2024-01-31T05:00:51.676564+0000","last_clean":"2024-01-31T05:00:51.676564+0000","last_became_active":"2024-01-31T05:00:50.647001+0000","last_became_peered":"2024-01-31T05:00:50.647001+0000","last_unstale":"2024-01-31T05:00:51.676564+0000","last_undegraded":"2024-01-31T05:00:51.676564+0000","last_fullsized":"2024-01-31T05:00:51.676564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:42.992455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676393+0000","last_change":"2024-01-31T05:00:46.567858+0000","last_active":"2024-01-31T05:00:51.676393+0000","last_peered":"2024-01-31T05:00:51.676393+0000","last_clean":"2024-01-31T05:00:51.676393+0000","last_became_active":"2024-01-31T05:00:46.567351+0000","last_became_peered":"2024-01-31T05:00:46.567351+0000","last_unstale":"2024-01-31T05:00:51.676393+0000","last_undegraded":"2024-01-31T05:00:51.676393+0000","last_fullsized":"2024-01-31T05:00:51.676393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:07.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673099+0000","last_change":"2024-01-31T05:00:45.562694+0000","last_active":"2024-01-31T05:00:51.673099+0000","last_peered":"2024-01-31T05:00:51.673099+0000","last_clean":"2024-01-31T05:00:51.673099+0000","last_became_active":"2024-01-31T05:00:45.560991+0000","last_became_peered":"2024-01-31T05:00:45.560991+0000","last_unstale":"2024-01-31T05:00:51.673099+0000","last_undegraded":"2024-01-31T05:00:51.673099+0000","last_fullsized":"2024-01-31T05:00:51.673099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674525+0000","last_change":"2024-01-31T05:00:50.638454+0000","last_active":"2024-01-31T05:00:51.674525+0000","last_peered":"2024-01-31T05:00:51.674525+0000","last_clean":"2024-01-31T05:00:51.674525+0000","last_became_active":"2024-01-31T05:00:50.637600+0000","last_became_peered":"2024-01-31T05:00:50.637600+0000","last_unstale":"2024-01-31T05:00:51.674525+0000","last_undegraded":"2024-01-31T05:00:51.674525+0000","last_fullsized":"2024-01-31T05:00:51.674525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:16.246206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676944+0000","last_change":"2024-01-31T05:00:46.574103+0000","last_active":"2024-01-31T05:00:51.676944+0000","last_peered":"2024-01-31T05:00:51.676944+0000","last_clean":"2024-01-31T05:00:51.676944+0000","last_became_active":"2024-01-31T05:00:46.572921+0000","last_became_peered":"2024-01-31T05:00:46.572921+0000","last_unstale":"2024-01-31T05:00:51.676944+0000","last_undegraded":"2024-01-31T05:00:51.676944+0000","last_fullsized":"2024-01-31T05:00:51.676944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:36.342238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916504+0000","last_change":"2024-01-31T05:00:45.587813+0000","last_active":"2024-01-31T05:00:51.916504+0000","last_peered":"2024-01-31T05:00:51.916504+0000","last_clean":"2024-01-31T05:00:51.916504+0000","last_became_active":"2024-01-31T05:00:45.587473+0000","last_became_peered":"2024-01-31T05:00:45.587473+0000","last_unstale":"2024-01-31T05:00:51.916504+0000","last_undegraded":"2024-01-31T05:00:51.916504+0000","last_fullsized":"2024-01-31T05:00:51.916504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:44:11.745313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673739+0000","last_change":"2024-01-31T05:00:45.574390+0000","last_active":"2024-01-31T05:00:51.673739+0000","last_peered":"2024-01-31T05:00:51.673739+0000","last_clean":"2024-01-31T05:00:51.673739+0000","last_became_active":"2024-01-31T05:00:45.574121+0000","last_became_peered":"2024-01-31T05:00:45.574121+0000","last_unstale":"2024-01-31T05:00:51.673739+0000","last_undegraded":"2024-01-31T05:00:51.673739+0000","last_fullsized":"2024-01-31T05:00:51.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:01.042942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677784+0000","last_change":"2024-01-31T05:00:46.582109+0000","last_active":"2024-01-31T05:00:51.677784+0000","last_peered":"2024-01-31T05:00:51.677784+0000","last_clean":"2024-01-31T05:00:51.677784+0000","last_became_active":"2024-01-31T05:00:46.581814+0000","last_became_peered":"2024-01-31T05:00:46.581814+0000","last_unstale":"2024-01-31T05:00:51.677784+0000","last_undegraded":"2024-01-31T05:00:51.677784+0000","last_fullsized":"2024-01-31T05:00:51.677784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:40.921562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.678543+0000","last_change":"2024-01-31T05:00:50.652235+0000","last_active":"2024-01-31T05:00:51.678543+0000","last_peered":"2024-01-31T05:00:51.678543+0000","last_clean":"2024-01-31T05:00:51.678543+0000","last_became_active":"2024-01-31T05:00:50.650199+0000","last_became_peered":"2024-01-31T05:00:50.650199+0000","last_unstale":"2024-01-31T05:00:51.678543+0000","last_undegraded":"2024-01-31T05:00:51.678543+0000","last_fullsized":"2024-01-31T05:00:51.678543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:15.584111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679821+0000","last_change":"2024-01-31T05:00:45.580589+0000","last_active":"2024-01-31T05:00:51.679821+0000","last_peered":"2024-01-31T05:00:51.679821+0000","last_clean":"2024-01-31T05:00:51.679821+0000","last_became_active":"2024-01-31T05:00:45.579288+0000","last_became_peered":"2024-01-31T05:00:45.579288+0000","last_unstale":"2024-01-31T05:00:51.679821+0000","last_undegraded":"2024-01-31T05:00:51.679821+0000","last_fullsized":"2024-01-31T05:00:51.679821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912730+0000","last_change":"2024-01-31T05:00:46.579010+0000","last_active":"2024-01-31T05:00:51.912730+0000","last_peered":"2024-01-31T05:00:51.912730+0000","last_clean":"2024-01-31T05:00:51.912730+0000","last_became_active":"2024-01-31T05:00:46.578792+0000","last_became_peered":"2024-01-31T05:00:46.578792+0000","last_unstale":"2024-01-31T05:00:51.912730+0000","last_undegraded":"2024-01-31T05:00:51.912730+0000","last_fullsized":"2024-01-31T05:00:51.912730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916113+0000","last_change":"2024-01-31T05:00:50.659322+0000","last_active":"2024-01-31T05:00:51.916113+0000","last_peered":"2024-01-31T05:00:51.916113+0000","last_clean":"2024-01-31T05:00:51.916113+0000","last_became_active":"2024-01-31T05:00:50.658743+0000","last_became_peered":"2024-01-31T05:00:50.658743+0000","last_unstale":"2024-01-31T05:00:51.916113+0000","last_undegraded":"2024-01-31T05:00:51.916113+0000","last_fullsized":"2024-01-31T05:00:51.916113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:20.923767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913064+0000","last_change":"2024-01-31T05:00:46.579418+0000","last_active":"2024-01-31T05:00:51.913064+0000","last_peered":"2024-01-31T05:00:51.913064+0000","last_clean":"2024-01-31T05:00:51.913064+0000","last_became_active":"2024-01-31T05:00:46.579076+0000","last_became_peered":"2024-01-31T05:00:46.579076+0000","last_unstale":"2024-01-31T05:00:51.913064+0000","last_undegraded":"2024-01-31T05:00:51.913064+0000","last_fullsized":"2024-01-31T05:00:51.913064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914823+0000","last_change":"2024-01-31T05:00:46.585461+0000","last_active":"2024-01-31T05:00:51.914823+0000","last_peered":"2024-01-31T05:00:51.914823+0000","last_clean":"2024-01-31T05:00:51.914823+0000","last_became_active":"2024-01-31T05:00:46.584818+0000","last_became_peered":"2024-01-31T05:00:46.584818+0000","last_unstale":"2024-01-31T05:00:51.914823+0000","last_undegraded":"2024-01-31T05:00:51.914823+0000","last_fullsized":"2024-01-31T05:00:51.914823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:52.011589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673116+0000","last_change":"2024-01-31T05:00:46.586264+0000","last_active":"2024-01-31T05:00:51.673116+0000","last_peered":"2024-01-31T05:00:51.673116+0000","last_clean":"2024-01-31T05:00:51.673116+0000","last_became_active":"2024-01-31T05:00:46.585995+0000","last_became_peered":"2024-01-31T05:00:46.585995+0000","last_unstale":"2024-01-31T05:00:51.673116+0000","last_undegraded":"2024-01-31T05:00:51.673116+0000","last_fullsized":"2024-01-31T05:00:51.673116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:11.523909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679572+0000","last_change":"2024-01-31T05:00:46.579199+0000","last_active":"2024-01-31T05:00:51.679572+0000","last_peered":"2024-01-31T05:00:51.679572+0000","last_clean":"2024-01-31T05:00:51.679572+0000","last_became_active":"2024-01-31T05:00:46.578257+0000","last_became_peered":"2024-01-31T05:00:46.578257+0000","last_unstale":"2024-01-31T05:00:51.679572+0000","last_undegraded":"2024-01-31T05:00:51.679572+0000","last_fullsized":"2024-01-31T05:00:51.679572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918235+0000","last_change":"2024-01-31T05:00:46.593320+0000","last_active":"2024-01-31T05:00:51.918235+0000","last_peered":"2024-01-31T05:00:51.918235+0000","last_clean":"2024-01-31T05:00:51.918235+0000","last_became_active":"2024-01-31T05:00:46.593053+0000","last_became_peered":"2024-01-31T05:00:46.593053+0000","last_unstale":"2024-01-31T05:00:51.918235+0000","last_undegraded":"2024-01-31T05:00:51.918235+0000","last_fullsized":"2024-01-31T05:00:51.918235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:21.326870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918207+0000","last_change":"2024-01-31T05:00:46.583943+0000","last_active":"2024-01-31T05:00:51.918207+0000","last_peered":"2024-01-31T05:00:51.918207+0000","last_clean":"2024-01-31T05:00:51.918207+0000","last_became_active":"2024-01-31T05:00:46.580249+0000","last_became_peered":"2024-01-31T05:00:46.580249+0000","last_unstale":"2024-01-31T05:00:51.918207+0000","last_undegraded":"2024-01-31T05:00:51.918207+0000","last_fullsized":"2024-01-31T05:00:51.918207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914742+0000","last_change":"2024-01-31T05:00:46.601110+0000","last_active":"2024-01-31T05:00:51.914742+0000","last_peered":"2024-01-31T05:00:51.914742+0000","last_clean":"2024-01-31T05:00:51.914742+0000","last_became_active":"2024-01-31T05:00:46.600844+0000","last_became_peered":"2024-01-31T05:00:46.600844+0000","last_unstale":"2024-01-31T05:00:51.914742+0000","last_undegraded":"2024-01-31T05:00:51.914742+0000","last_fullsized":"2024-01-31T05:00:51.914742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681192+0000","last_change":"2024-01-31T05:00:46.579792+0000","last_active":"2024-01-31T05:00:51.681192+0000","last_peered":"2024-01-31T05:00:51.681192+0000","last_clean":"2024-01-31T05:00:51.681192+0000","last_became_active":"2024-01-31T05:00:46.579357+0000","last_became_peered":"2024-01-31T05:00:46.579357+0000","last_unstale":"2024-01-31T05:00:51.681192+0000","last_undegraded":"2024-01-31T05:00:51.681192+0000","last_fullsized":"2024-01-31T05:00:51.681192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672904+0000","last_change":"2024-01-31T05:00:46.582507+0000","last_active":"2024-01-31T05:00:51.672904+0000","last_peered":"2024-01-31T05:00:51.672904+0000","last_clean":"2024-01-31T05:00:51.672904+0000","last_became_active":"2024-01-31T05:00:46.582243+0000","last_became_peered":"2024-01-31T05:00:46.582243+0000","last_unstale":"2024-01-31T05:00:51.672904+0000","last_undegraded":"2024-01-31T05:00:51.672904+0000","last_fullsized":"2024-01-31T05:00:51.672904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:07.689214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676563+0000","last_change":"2024-01-31T05:00:46.588900+0000","last_active":"2024-01-31T05:00:51.676563+0000","last_peered":"2024-01-31T05:00:51.676563+0000","last_clean":"2024-01-31T05:00:51.676563+0000","last_became_active":"2024-01-31T05:00:46.585994+0000","last_became_peered":"2024-01-31T05:00:46.585994+0000","last_unstale":"2024-01-31T05:00:51.676563+0000","last_undegraded":"2024-01-31T05:00:51.676563+0000","last_fullsized":"2024-01-31T05:00:51.676563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912637+0000","last_change":"2024-01-31T05:00:46.576223+0000","last_active":"2024-01-31T05:00:51.912637+0000","last_peered":"2024-01-31T05:00:51.912637+0000","last_clean":"2024-01-31T05:00:51.912637+0000","last_became_active":"2024-01-31T05:00:46.575962+0000","last_became_peered":"2024-01-31T05:00:46.575962+0000","last_unstale":"2024-01-31T05:00:51.912637+0000","last_undegraded":"2024-01-31T05:00:51.912637+0000","last_fullsized":"2024-01-31T05:00:51.912637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677638+0000","last_change":"2024-01-31T05:00:46.576001+0000","last_active":"2024-01-31T05:00:51.677638+0000","last_peered":"2024-01-31T05:00:51.677638+0000","last_clean":"2024-01-31T05:00:51.677638+0000","last_became_active":"2024-01-31T05:00:46.574721+0000","last_became_peered":"2024-01-31T05:00:46.574721+0000","last_unstale":"2024-01-31T05:00:51.677638+0000","last_undegraded":"2024-01-31T05:00:51.677638+0000","last_fullsized":"2024-01-31T05:00:51.677638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:31:53.337974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915731+0000","last_change":"2024-01-31T05:00:46.589503+0000","last_active":"2024-01-31T05:00:51.915731+0000","last_peered":"2024-01-31T05:00:51.915731+0000","last_clean":"2024-01-31T05:00:51.915731+0000","last_became_active":"2024-01-31T05:00:46.581750+0000","last_became_peered":"2024-01-31T05:00:46.581750+0000","last_unstale":"2024-01-31T05:00:51.915731+0000","last_undegraded":"2024-01-31T05:00:51.915731+0000","last_fullsized":"2024-01-31T05:00:51.915731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674812+0000","last_change":"2024-01-31T05:00:46.582051+0000","last_active":"2024-01-31T05:00:51.674812+0000","last_peered":"2024-01-31T05:00:51.674812+0000","last_clean":"2024-01-31T05:00:51.674812+0000","last_became_active":"2024-01-31T05:00:46.578134+0000","last_became_peered":"2024-01-31T05:00:46.578134+0000","last_unstale":"2024-01-31T05:00:51.674812+0000","last_undegraded":"2024-01-31T05:00:51.674812+0000","last_fullsized":"2024-01-31T05:00:51.674812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:31.255109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917009+0000","last_change":"2024-01-31T05:00:46.587070+0000","last_active":"2024-01-31T05:00:51.917009+0000","last_peered":"2024-01-31T05:00:51.917009+0000","last_clean":"2024-01-31T05:00:51.917009+0000","last_became_active":"2024-01-31T05:00:46.586707+0000","last_became_peered":"2024-01-31T05:00:46.586707+0000","last_unstale":"2024-01-31T05:00:51.917009+0000","last_undegraded":"2024-01-31T05:00:51.917009+0000","last_fullsized":"2024-01-31T05:00:51.917009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674901+0000","last_change":"2024-01-31T05:00:46.587042+0000","last_active":"2024-01-31T05:00:51.674901+0000","last_peered":"2024-01-31T05:00:51.674901+0000","last_clean":"2024-01-31T05:00:51.674901+0000","last_became_active":"2024-01-31T05:00:46.586518+0000","last_became_peered":"2024-01-31T05:00:46.586518+0000","last_unstale":"2024-01-31T05:00:51.674901+0000","last_undegraded":"2024-01-31T05:00:51.674901+0000","last_fullsized":"2024-01-31T05:00:51.674901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:54:17.597851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672833+0000","last_change":"2024-01-31T05:00:46.593111+0000","last_active":"2024-01-31T05:00:51.672833+0000","last_peered":"2024-01-31T05:00:51.672833+0000","last_clean":"2024-01-31T05:00:51.672833+0000","last_became_active":"2024-01-31T05:00:46.592828+0000","last_became_peered":"2024-01-31T05:00:46.592828+0000","last_unstale":"2024-01-31T05:00:51.672833+0000","last_undegraded":"2024-01-31T05:00:51.672833+0000","last_fullsized":"2024-01-31T05:00:51.672833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:59.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676004+0000","last_change":"2024-01-31T05:00:46.589807+0000","last_active":"2024-01-31T05:00:51.676004+0000","last_peered":"2024-01-31T05:00:51.676004+0000","last_clean":"2024-01-31T05:00:51.676004+0000","last_became_active":"2024-01-31T05:00:46.589518+0000","last_became_peered":"2024-01-31T05:00:46.589518+0000","last_unstale":"2024-01-31T05:00:51.676004+0000","last_undegraded":"2024-01-31T05:00:51.676004+0000","last_fullsized":"2024-01-31T05:00:51.676004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:06:03.428645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916672+0000","last_change":"2024-01-31T05:00:46.583014+0000","last_active":"2024-01-31T05:00:51.916672+0000","last_peered":"2024-01-31T05:00:51.916672+0000","last_clean":"2024-01-31T05:00:51.916672+0000","last_became_active":"2024-01-31T05:00:46.582607+0000","last_became_peered":"2024-01-31T05:00:46.582607+0000","last_unstale":"2024-01-31T05:00:51.916672+0000","last_undegraded":"2024-01-31T05:00:51.916672+0000","last_fullsized":"2024-01-31T05:00:51.916672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:21:09.066019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917869+0000","last_change":"2024-01-31T05:00:46.594160+0000","last_active":"2024-01-31T05:00:51.917869+0000","last_peered":"2024-01-31T05:00:51.917869+0000","last_clean":"2024-01-31T05:00:51.917869+0000","last_became_active":"2024-01-31T05:00:46.593838+0000","last_became_peered":"2024-01-31T05:00:46.593838+0000","last_unstale":"2024-01-31T05:00:51.917869+0000","last_undegraded":"2024-01-31T05:00:51.917869+0000","last_fullsized":"2024-01-31T05:00:51.917869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:58:27.273440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915637+0000","last_change":"2024-01-31T05:00:46.590079+0000","last_active":"2024-01-31T05:00:51.915637+0000","last_peered":"2024-01-31T05:00:51.915637+0000","last_clean":"2024-01-31T05:00:51.915637+0000","last_became_active":"2024-01-31T05:00:46.589828+0000","last_became_peered":"2024-01-31T05:00:46.589828+0000","last_unstale":"2024-01-31T05:00:51.915637+0000","last_undegraded":"2024-01-31T05:00:51.915637+0000","last_fullsized":"2024-01-31T05:00:51.915637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915410+0000","last_change":"2024-01-31T05:00:46.601762+0000","last_active":"2024-01-31T05:00:51.915410+0000","last_peered":"2024-01-31T05:00:51.915410+0000","last_clean":"2024-01-31T05:00:51.915410+0000","last_became_active":"2024-01-31T05:00:46.600161+0000","last_became_peered":"2024-01-31T05:00:46.600161+0000","last_unstale":"2024-01-31T05:00:51.915410+0000","last_undegraded":"2024-01-31T05:00:51.915410+0000","last_fullsized":"2024-01-31T05:00:51.915410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915285+0000","last_change":"2024-01-31T05:00:46.588560+0000","last_active":"2024-01-31T05:00:51.915285+0000","last_peered":"2024-01-31T05:00:51.915285+0000","last_clean":"2024-01-31T05:00:51.915285+0000","last_became_active":"2024-01-31T05:00:46.588291+0000","last_became_peered":"2024-01-31T05:00:46.588291+0000","last_unstale":"2024-01-31T05:00:51.915285+0000","last_undegraded":"2024-01-31T05:00:51.915285+0000","last_fullsized":"2024-01-31T05:00:51.915285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914247+0000","last_change":"2024-01-31T05:00:46.570493+0000","last_active":"2024-01-31T05:00:51.914247+0000","last_peered":"2024-01-31T05:00:51.914247+0000","last_clean":"2024-01-31T05:00:51.914247+0000","last_became_active":"2024-01-31T05:00:46.570196+0000","last_became_peered":"2024-01-31T05:00:46.570196+0000","last_unstale":"2024-01-31T05:00:51.914247+0000","last_undegraded":"2024-01-31T05:00:51.914247+0000","last_fullsized":"2024-01-31T05:00:51.914247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674510+0000","last_change":"2024-01-31T05:00:46.580076+0000","last_active":"2024-01-31T05:00:51.674510+0000","last_peered":"2024-01-31T05:00:51.674510+0000","last_clean":"2024-01-31T05:00:51.674510+0000","last_became_active":"2024-01-31T05:00:46.578312+0000","last_became_peered":"2024-01-31T05:00:46.578312+0000","last_unstale":"2024-01-31T05:00:51.674510+0000","last_undegraded":"2024-01-31T05:00:51.674510+0000","last_fullsized":"2024-01-31T05:00:51.674510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914636+0000","last_change":"2024-01-31T05:00:46.577353+0000","last_active":"2024-01-31T05:00:51.914636+0000","last_peered":"2024-01-31T05:00:51.914636+0000","last_clean":"2024-01-31T05:00:51.914636+0000","last_became_active":"2024-01-31T05:00:46.577019+0000","last_became_peered":"2024-01-31T05:00:46.577019+0000","last_unstale":"2024-01-31T05:00:51.914636+0000","last_undegraded":"2024-01-31T05:00:51.914636+0000","last_fullsized":"2024-01-31T05:00:51.914636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:09:13.929721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912937+0000","last_change":"2024-01-31T05:00:46.575406+0000","last_active":"2024-01-31T05:00:51.912937+0000","last_peered":"2024-01-31T05:00:51.912937+0000","last_clean":"2024-01-31T05:00:51.912937+0000","last_became_active":"2024-01-31T05:00:46.575056+0000","last_became_peered":"2024-01-31T05:00:46.575056+0000","last_unstale":"2024-01-31T05:00:51.912937+0000","last_undegraded":"2024-01-31T05:00:51.912937+0000","last_fullsized":"2024-01-31T05:00:51.912937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915174+0000","last_change":"2024-01-31T05:00:46.582321+0000","last_active":"2024-01-31T05:00:51.915174+0000","last_peered":"2024-01-31T05:00:51.915174+0000","last_clean":"2024-01-31T05:00:51.915174+0000","last_became_active":"2024-01-31T05:00:46.581547+0000","last_became_peered":"2024-01-31T05:00:46.581547+0000","last_unstale":"2024-01-31T05:00:51.915174+0000","last_undegraded":"2024-01-31T05:00:51.915174+0000","last_fullsized":"2024-01-31T05:00:51.915174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914144+0000","last_change":"2024-01-31T05:00:46.575296+0000","last_active":"2024-01-31T05:00:51.914144+0000","last_peered":"2024-01-31T05:00:51.914144+0000","last_clean":"2024-01-31T05:00:51.914144+0000","last_became_active":"2024-01-31T05:00:46.574983+0000","last_became_peered":"2024-01-31T05:00:46.574983+0000","last_unstale":"2024-01-31T05:00:51.914144+0000","last_undegraded":"2024-01-31T05:00:51.914144+0000","last_fullsized":"2024-01-31T05:00:51.914144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680670+0000","last_change":"2024-01-31T05:00:46.574733+0000","last_active":"2024-01-31T05:00:51.680670+0000","last_peered":"2024-01-31T05:00:51.680670+0000","last_clean":"2024-01-31T05:00:51.680670+0000","last_became_active":"2024-01-31T05:00:46.574335+0000","last_became_peered":"2024-01-31T05:00:46.574335+0000","last_unstale":"2024-01-31T05:00:51.680670+0000","last_undegraded":"2024-01-31T05:00:51.680670+0000","last_fullsized":"2024-01-31T05:00:51.680670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680984+0000","last_change":"2024-01-31T05:00:46.578081+0000","last_active":"2024-01-31T05:00:51.680984+0000","last_peered":"2024-01-31T05:00:51.680984+0000","last_clean":"2024-01-31T05:00:51.680984+0000","last_became_active":"2024-01-31T05:00:46.577576+0000","last_became_peered":"2024-01-31T05:00:46.577576+0000","last_unstale":"2024-01-31T05:00:51.680984+0000","last_undegraded":"2024-01-31T05:00:51.680984+0000","last_fullsized":"2024-01-31T05:00:51.680984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915608+0000","last_change":"2024-01-31T05:00:46.592072+0000","last_active":"2024-01-31T05:00:51.915608+0000","last_peered":"2024-01-31T05:00:51.915608+0000","last_clean":"2024-01-31T05:00:51.915608+0000","last_became_active":"2024-01-31T05:00:46.591682+0000","last_became_peered":"2024-01-31T05:00:46.591682+0000","last_unstale":"2024-01-31T05:00:51.915608+0000","last_undegraded":"2024-01-31T05:00:51.915608+0000","last_fullsized":"2024-01-31T05:00:51.915608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:13:21.174300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":106194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.728}]}]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":106652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6000000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":696508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4169999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8319999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":106126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.302}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9860000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.262}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4220000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:01:15.797 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:01:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.037+0000 7fed66000640 1 Processor -- start 2024-01-31T05:01:16.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.037+0000 7fed66000640 1 -- start start 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed66000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed6015d590 0x7fed6015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed66000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed600cfae0 0x7fed600cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed66000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600d0460 con 0x7fed60161080 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed66000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed600cded0 con 0x7fed600cfae0 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed66000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6015d970 con 0x7fed6015d590 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed56800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed600cfae0 0x7fed600cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed56800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed600cfae0 0x7fed600cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34348/0 (socket says 172.21.15.97:34348) 2024-01-31T05:01:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.039+0000 7fed56800640 1 -- 172.21.15.97:0/1586604242 learned_addr learned my addr 172.21.15.97:0/1586604242 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.040+0000 7fed5f400640 1 --2- 172.21.15.97:0/1586604242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed6015d590 0x7fed6015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.040+0000 7fed56800640 1 -- 172.21.15.97:0/1586604242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed6015d590 msgr2=0x7fed6015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.040+0000 7fed56800640 1 --2- 172.21.15.97:0/1586604242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed6015d590 0x7fed6015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.040+0000 7fed56800640 1 -- 172.21.15.97:0/1586604242 >> v1:172.21.15.188:6789/0 conn(0x7fed60161080 legacy=0x7fed60161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.040+0000 7fed56800640 1 -- 172.21.15.97:0/1586604242 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600cf210 con 0x7fed600cfae0 2024-01-31T05:01:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.041+0000 7fed56800640 1 --2- 172.21.15.97:0/1586604242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed600cfae0 0x7fed600cff20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed48009870 tx=0x7fed4802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a00b4ba60b11054 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.041+0000 7fed5ea00640 1 -- 172.21.15.97:0/1586604242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48030840 con 0x7fed600cfae0 2024-01-31T05:01:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.041+0000 7fed5ea00640 1 -- 172.21.15.97:0/1586604242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed480309a0 con 0x7fed600cfae0 2024-01-31T05:01:16.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.041+0000 7fed5ea00640 1 -- 172.21.15.97:0/1586604242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48030d30 con 0x7fed600cfae0 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 -- 172.21.15.97:0/1586604242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed600cfae0 msgr2=0x7fed600cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 --2- 172.21.15.97:0/1586604242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed600cfae0 0x7fed600cff20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed48009870 tx=0x7fed4802b980 comp rx=0 tx=0).stop 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 -- 172.21.15.97:0/1586604242 shutdown_connections 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 --2- 172.21.15.97:0/1586604242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed600cfae0 0x7fed600cff20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 --2- 172.21.15.97:0/1586604242 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed6015d590 0x7fed6015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 -- 172.21.15.97:0/1586604242 >> 172.21.15.97:0/1586604242 conn(0x7fed6009c560 msgr2=0x7fed600d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 -- 172.21.15.97:0/1586604242 shutdown_connections 2024-01-31T05:01:16.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.042+0000 7fed66000640 1 -- 172.21.15.97:0/1586604242 wait complete. 2024-01-31T05:01:16.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.044+0000 7fed66000640 1 Processor -- start 2024-01-31T05:01:16.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.044+0000 7fed66000640 1 -- start start 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed66000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed600cfae0 0x7fed60159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed66000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 0x7fed600b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed66000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed60161080 0x7fed600b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed66000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c7a40 con 0x7fed6015d590 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed66000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c7d20 con 0x7fed600cfae0 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed66000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c8000 con 0x7fed60161080 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 0x7fed600b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 0x7fed600b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34356/0 (socket says 172.21.15.97:34356) 2024-01-31T05:01:16.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 -- 172.21.15.97:0/2492708678 learned_addr learned my addr 172.21.15.97:0/2492708678 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:16.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 -- 172.21.15.97:0/2492708678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed60161080 msgr2=0x7fed600b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed60161080 0x7fed600b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 -- 172.21.15.97:0/2492708678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed600cfae0 msgr2=0x7fed60159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed600cfae0 0x7fed60159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.045+0000 7fed56800640 1 -- 172.21.15.97:0/2492708678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed480094d0 con 0x7fed6015d590 2024-01-31T05:01:16.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.046+0000 7fed56800640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 0x7fed600b0900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.046+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48030810 con 0x7fed6015d590 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.046+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed48043400 con 0x7fed6015d590 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.046+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed600c82e0 con 0x7fed6015d590 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.046+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed480437d0 con 0x7fed6015d590 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.046+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed600b5f70 con 0x7fed6015d590 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.047+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed600c84d0 con 0x7fed6015d590 2024-01-31T05:01:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.048+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fed48043930 con 0x7fed6015d590 2024-01-31T05:01:16.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.049+0000 7fed57e00640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed3806aed0 0x7fed3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.049+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed4803e070 con 0x7fed6015d590 2024-01-31T05:01:16.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.049+0000 7fed5f400640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed3806aed0 0x7fed3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.050+0000 7fed5f400640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed3806aed0 0x7fed3806d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed50006fd0 tx=0x7fed50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.053+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fed48077770 con 0x7fed6015d590 2024-01-31T05:01:16.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.229+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed600c7b80 con 0x7fed3806aed0 2024-01-31T05:01:16.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.254+0000 7fed57e00640 1 -- 172.21.15.97:0/2492708678 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345167 (secure 0 0 0) 0x7fed600c7b80 con 0x7fed3806aed0 2024-01-31T05:01:16.258 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:16.260 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:01:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.260+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed3806aed0 msgr2=0x7fed3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed3806aed0 0x7fed3806d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed50006fd0 tx=0x7fed50008040 comp rx=0 tx=0).stop 2024-01-31T05:01:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 msgr2=0x7fed600b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 0x7fed600b0900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed48000c00 tx=0x7fed48004730 comp rx=0 tx=0).stop 2024-01-31T05:01:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 shutdown_connections 2024-01-31T05:01:16.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed60161080 0x7fed600b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed3806aed0 0x7fed3806d390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed6015d590 0x7fed600b0900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 --2- 172.21.15.97:0/2492708678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed600cfae0 0x7fed60159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 >> 172.21.15.97:0/2492708678 conn(0x7fed6009c560 msgr2=0x7fed6015e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 shutdown_connections 2024-01-31T05:01:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.261+0000 7fed66000640 1 -- 172.21.15.97:0/2492708678 wait complete. 2024-01-31T05:01:16.277 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-01-31T05:01:15.560002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219496,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755224,"statfs":{"total":773094113280,"available":772869349376,"internally_reserved":0,"allocated":3416064,"data_stored":2038574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004784"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679907+0000","last_change":"2024-01-31T05:00:45.571880+0000","last_active":"2024-01-31T05:00:51.679907+0000","last_peered":"2024-01-31T05:00:51.679907+0000","last_clean":"2024-01-31T05:00:51.679907+0000","last_became_active":"2024-01-31T05:00:45.571633+0000","last_became_peered":"2024-01-31T05:00:45.571633+0000","last_unstale":"2024-01-31T05:00:51.679907+0000","last_undegraded":"2024-01-31T05:00:51.679907+0000","last_fullsized":"2024-01-31T05:00:51.679907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681114+0000","last_change":"2024-01-31T05:00:46.593889+0000","last_active":"2024-01-31T05:00:51.681114+0000","last_peered":"2024-01-31T05:00:51.681114+0000","last_clean":"2024-01-31T05:00:51.681114+0000","last_became_active":"2024-01-31T05:00:46.593583+0000","last_became_peered":"2024-01-31T05:00:46.593583+0000","last_unstale":"2024-01-31T05:00:51.681114+0000","last_undegraded":"2024-01-31T05:00:51.681114+0000","last_fullsized":"2024-01-31T05:00:51.681114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914795+0000","last_change":"2024-01-31T05:00:50.653143+0000","last_active":"2024-01-31T05:00:51.914795+0000","last_peered":"2024-01-31T05:00:51.914795+0000","last_clean":"2024-01-31T05:00:51.914795+0000","last_became_active":"2024-01-31T05:00:50.652831+0000","last_became_peered":"2024-01-31T05:00:50.652831+0000","last_unstale":"2024-01-31T05:00:51.914795+0000","last_undegraded":"2024-01-31T05:00:51.914795+0000","last_fullsized":"2024-01-31T05:00:51.914795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:44.981765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681288+0000","last_change":"2024-01-31T05:00:45.578379+0000","last_active":"2024-01-31T05:00:51.681288+0000","last_peered":"2024-01-31T05:00:51.681288+0000","last_clean":"2024-01-31T05:00:51.681288+0000","last_became_active":"2024-01-31T05:00:45.574393+0000","last_became_peered":"2024-01-31T05:00:45.574393+0000","last_unstale":"2024-01-31T05:00:51.681288+0000","last_undegraded":"2024-01-31T05:00:51.681288+0000","last_fullsized":"2024-01-31T05:00:51.681288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681650+0000","last_change":"2024-01-31T05:00:46.573630+0000","last_active":"2024-01-31T05:00:51.681650+0000","last_peered":"2024-01-31T05:00:51.681650+0000","last_clean":"2024-01-31T05:00:51.681650+0000","last_became_active":"2024-01-31T05:00:46.572199+0000","last_became_peered":"2024-01-31T05:00:46.572199+0000","last_unstale":"2024-01-31T05:00:51.681650+0000","last_undegraded":"2024-01-31T05:00:51.681650+0000","last_fullsized":"2024-01-31T05:00:51.681650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912863+0000","last_change":"2024-01-31T05:00:50.644804+0000","last_active":"2024-01-31T05:00:51.912863+0000","last_peered":"2024-01-31T05:00:51.912863+0000","last_clean":"2024-01-31T05:00:51.912863+0000","last_became_active":"2024-01-31T05:00:50.640815+0000","last_became_peered":"2024-01-31T05:00:50.640815+0000","last_unstale":"2024-01-31T05:00:51.912863+0000","last_undegraded":"2024-01-31T05:00:51.912863+0000","last_fullsized":"2024-01-31T05:00:51.912863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:03:57.881066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915593+0000","last_change":"2024-01-31T05:00:45.582233+0000","last_active":"2024-01-31T05:00:51.915593+0000","last_peered":"2024-01-31T05:00:51.915593+0000","last_clean":"2024-01-31T05:00:51.915593+0000","last_became_active":"2024-01-31T05:00:45.578094+0000","last_became_peered":"2024-01-31T05:00:45.578094+0000","last_unstale":"2024-01-31T05:00:51.915593+0000","last_undegraded":"2024-01-31T05:00:51.915593+0000","last_fullsized":"2024-01-31T05:00:51.915593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:21:00.930899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913671+0000","last_change":"2024-01-31T05:00:46.589171+0000","last_active":"2024-01-31T05:00:51.913671+0000","last_peered":"2024-01-31T05:00:51.913671+0000","last_clean":"2024-01-31T05:00:51.913671+0000","last_became_active":"2024-01-31T05:00:46.588749+0000","last_became_peered":"2024-01-31T05:00:46.588749+0000","last_unstale":"2024-01-31T05:00:51.913671+0000","last_undegraded":"2024-01-31T05:00:51.913671+0000","last_fullsized":"2024-01-31T05:00:51.913671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914061+0000","last_change":"2024-01-31T05:00:50.635774+0000","last_active":"2024-01-31T05:00:51.914061+0000","last_peered":"2024-01-31T05:00:51.914061+0000","last_clean":"2024-01-31T05:00:51.914061+0000","last_became_active":"2024-01-31T05:00:50.635404+0000","last_became_peered":"2024-01-31T05:00:50.635404+0000","last_unstale":"2024-01-31T05:00:51.914061+0000","last_undegraded":"2024-01-31T05:00:51.914061+0000","last_fullsized":"2024-01-31T05:00:51.914061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:01:05.575230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915820+0000","last_change":"2024-01-31T05:00:45.583051+0000","last_active":"2024-01-31T05:00:51.915820+0000","last_peered":"2024-01-31T05:00:51.915820+0000","last_clean":"2024-01-31T05:00:51.915820+0000","last_became_active":"2024-01-31T05:00:45.582464+0000","last_became_peered":"2024-01-31T05:00:45.582464+0000","last_unstale":"2024-01-31T05:00:51.915820+0000","last_undegraded":"2024-01-31T05:00:51.915820+0000","last_fullsized":"2024-01-31T05:00:51.915820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:46:05.159804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674284+0000","last_change":"2024-01-31T05:00:46.593796+0000","last_active":"2024-01-31T05:00:51.674284+0000","last_peered":"2024-01-31T05:00:51.674284+0000","last_clean":"2024-01-31T05:00:51.674284+0000","last_became_active":"2024-01-31T05:00:46.593485+0000","last_became_peered":"2024-01-31T05:00:46.593485+0000","last_unstale":"2024-01-31T05:00:51.674284+0000","last_undegraded":"2024-01-31T05:00:51.674284+0000","last_fullsized":"2024-01-31T05:00:51.674284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:51:45.113263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912620+0000","last_change":"2024-01-31T05:00:50.645582+0000","last_active":"2024-01-31T05:00:51.912620+0000","last_peered":"2024-01-31T05:00:51.912620+0000","last_clean":"2024-01-31T05:00:51.912620+0000","last_became_active":"2024-01-31T05:00:50.645198+0000","last_became_peered":"2024-01-31T05:00:50.645198+0000","last_unstale":"2024-01-31T05:00:51.912620+0000","last_undegraded":"2024-01-31T05:00:51.912620+0000","last_fullsized":"2024-01-31T05:00:51.912620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:07:24.039515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673371+0000","last_change":"2024-01-31T05:00:46.581785+0000","last_active":"2024-01-31T05:00:51.673371+0000","last_peered":"2024-01-31T05:00:51.673371+0000","last_clean":"2024-01-31T05:00:51.673371+0000","last_became_active":"2024-01-31T05:00:46.581463+0000","last_became_peered":"2024-01-31T05:00:46.581463+0000","last_unstale":"2024-01-31T05:00:51.673371+0000","last_undegraded":"2024-01-31T05:00:51.673371+0000","last_fullsized":"2024-01-31T05:00:51.673371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:24:26.658435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673762+0000","last_change":"2024-01-31T05:00:45.566782+0000","last_active":"2024-01-31T05:00:51.673762+0000","last_peered":"2024-01-31T05:00:51.673762+0000","last_clean":"2024-01-31T05:00:51.673762+0000","last_became_active":"2024-01-31T05:00:45.565360+0000","last_became_peered":"2024-01-31T05:00:45.565360+0000","last_unstale":"2024-01-31T05:00:51.673762+0000","last_undegraded":"2024-01-31T05:00:51.673762+0000","last_fullsized":"2024-01-31T05:00:51.673762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:45.457514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674294+0000","last_change":"2024-01-31T05:00:50.639468+0000","last_active":"2024-01-31T05:00:51.674294+0000","last_peered":"2024-01-31T05:00:51.674294+0000","last_clean":"2024-01-31T05:00:51.674294+0000","last_became_active":"2024-01-31T05:00:50.639035+0000","last_became_peered":"2024-01-31T05:00:50.639035+0000","last_unstale":"2024-01-31T05:00:51.674294+0000","last_undegraded":"2024-01-31T05:00:51.674294+0000","last_fullsized":"2024-01-31T05:00:51.674294+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:43:03.844139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915694+0000","last_change":"2024-01-31T05:00:46.583107+0000","last_active":"2024-01-31T05:00:51.915694+0000","last_peered":"2024-01-31T05:00:51.915694+0000","last_clean":"2024-01-31T05:00:51.915694+0000","last_became_active":"2024-01-31T05:00:46.582506+0000","last_became_peered":"2024-01-31T05:00:46.582506+0000","last_unstale":"2024-01-31T05:00:51.915694+0000","last_undegraded":"2024-01-31T05:00:51.915694+0000","last_fullsized":"2024-01-31T05:00:51.915694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917300+0000","last_change":"2024-01-31T05:00:45.585334+0000","last_active":"2024-01-31T05:00:51.917300+0000","last_peered":"2024-01-31T05:00:51.917300+0000","last_clean":"2024-01-31T05:00:51.917300+0000","last_became_active":"2024-01-31T05:00:45.585034+0000","last_became_peered":"2024-01-31T05:00:45.585034+0000","last_unstale":"2024-01-31T05:00:51.917300+0000","last_undegraded":"2024-01-31T05:00:51.917300+0000","last_fullsized":"2024-01-31T05:00:51.917300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915319+0000","last_change":"2024-01-31T05:00:50.642526+0000","last_active":"2024-01-31T05:00:51.915319+0000","last_peered":"2024-01-31T05:00:51.915319+0000","last_clean":"2024-01-31T05:00:51.915319+0000","last_became_active":"2024-01-31T05:00:50.642120+0000","last_became_peered":"2024-01-31T05:00:50.642120+0000","last_unstale":"2024-01-31T05:00:51.915319+0000","last_undegraded":"2024-01-31T05:00:51.915319+0000","last_fullsized":"2024-01-31T05:00:51.915319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674674+0000","last_change":"2024-01-31T05:00:46.585812+0000","last_active":"2024-01-31T05:00:51.674674+0000","last_peered":"2024-01-31T05:00:51.674674+0000","last_clean":"2024-01-31T05:00:51.674674+0000","last_became_active":"2024-01-31T05:00:46.585556+0000","last_became_peered":"2024-01-31T05:00:46.585556+0000","last_unstale":"2024-01-31T05:00:51.674674+0000","last_undegraded":"2024-01-31T05:00:51.674674+0000","last_fullsized":"2024-01-31T05:00:51.674674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:37.158256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676683+0000","last_change":"2024-01-31T05:00:45.580944+0000","last_active":"2024-01-31T05:00:51.676683+0000","last_peered":"2024-01-31T05:00:51.676683+0000","last_clean":"2024-01-31T05:00:51.676683+0000","last_became_active":"2024-01-31T05:00:45.580698+0000","last_became_peered":"2024-01-31T05:00:45.580698+0000","last_unstale":"2024-01-31T05:00:51.676683+0000","last_undegraded":"2024-01-31T05:00:51.676683+0000","last_fullsized":"2024-01-31T05:00:51.676683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:57.421278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916224+0000","last_change":"2024-01-31T05:00:50.659576+0000","last_active":"2024-01-31T05:00:51.916224+0000","last_peered":"2024-01-31T05:00:51.916224+0000","last_clean":"2024-01-31T05:00:51.916224+0000","last_became_active":"2024-01-31T05:00:50.659067+0000","last_became_peered":"2024-01-31T05:00:50.659067+0000","last_unstale":"2024-01-31T05:00:51.916224+0000","last_undegraded":"2024-01-31T05:00:51.916224+0000","last_fullsized":"2024-01-31T05:00:51.916224+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:11.777909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673016+0000","last_change":"2024-01-31T05:00:46.586716+0000","last_active":"2024-01-31T05:00:51.673016+0000","last_peered":"2024-01-31T05:00:51.673016+0000","last_clean":"2024-01-31T05:00:51.673016+0000","last_became_active":"2024-01-31T05:00:46.582889+0000","last_became_peered":"2024-01-31T05:00:46.582889+0000","last_unstale":"2024-01-31T05:00:51.673016+0000","last_undegraded":"2024-01-31T05:00:51.673016+0000","last_fullsized":"2024-01-31T05:00:51.673016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:07.424800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.572569+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.571841+0000","last_became_peered":"2024-01-31T05:00:45.571841+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914775+0000","last_change":"2024-01-31T05:00:50.637862+0000","last_active":"2024-01-31T05:00:51.914775+0000","last_peered":"2024-01-31T05:00:51.914775+0000","last_clean":"2024-01-31T05:00:51.914775+0000","last_became_active":"2024-01-31T05:00:50.637439+0000","last_became_peered":"2024-01-31T05:00:50.637439+0000","last_unstale":"2024-01-31T05:00:51.914775+0000","last_undegraded":"2024-01-31T05:00:51.914775+0000","last_fullsized":"2024-01-31T05:00:51.914775+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676808+0000","last_change":"2024-01-31T05:00:46.588199+0000","last_active":"2024-01-31T05:00:51.676808+0000","last_peered":"2024-01-31T05:00:51.676808+0000","last_clean":"2024-01-31T05:00:51.676808+0000","last_became_active":"2024-01-31T05:00:46.585236+0000","last_became_peered":"2024-01-31T05:00:46.585236+0000","last_unstale":"2024-01-31T05:00:51.676808+0000","last_undegraded":"2024-01-31T05:00:51.676808+0000","last_fullsized":"2024-01-31T05:00:51.676808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:16.863565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675061+0000","last_change":"2024-01-31T05:00:45.577787+0000","last_active":"2024-01-31T05:00:51.675061+0000","last_peered":"2024-01-31T05:00:51.675061+0000","last_clean":"2024-01-31T05:00:51.675061+0000","last_became_active":"2024-01-31T05:00:45.577538+0000","last_became_peered":"2024-01-31T05:00:45.577538+0000","last_unstale":"2024-01-31T05:00:51.675061+0000","last_undegraded":"2024-01-31T05:00:51.675061+0000","last_fullsized":"2024-01-31T05:00:51.675061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:15.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912746+0000","last_change":"2024-01-31T05:00:50.630730+0000","last_active":"2024-01-31T05:00:51.912746+0000","last_peered":"2024-01-31T05:00:51.912746+0000","last_clean":"2024-01-31T05:00:51.912746+0000","last_became_active":"2024-01-31T05:00:50.629780+0000","last_became_peered":"2024-01-31T05:00:50.629780+0000","last_unstale":"2024-01-31T05:00:51.912746+0000","last_undegraded":"2024-01-31T05:00:51.912746+0000","last_fullsized":"2024-01-31T05:00:51.912746+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:07:58.674398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679833+0000","last_change":"2024-01-31T05:00:46.583111+0000","last_active":"2024-01-31T05:00:51.679833+0000","last_peered":"2024-01-31T05:00:51.679833+0000","last_clean":"2024-01-31T05:00:51.679833+0000","last_became_active":"2024-01-31T05:00:46.582827+0000","last_became_peered":"2024-01-31T05:00:46.582827+0000","last_unstale":"2024-01-31T05:00:51.679833+0000","last_undegraded":"2024-01-31T05:00:51.679833+0000","last_fullsized":"2024-01-31T05:00:51.679833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:40:42.367021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679389+0000","last_change":"2024-01-31T05:00:45.580550+0000","last_active":"2024-01-31T05:00:51.679389+0000","last_peered":"2024-01-31T05:00:51.679389+0000","last_clean":"2024-01-31T05:00:51.679389+0000","last_became_active":"2024-01-31T05:00:45.579549+0000","last_became_peered":"2024-01-31T05:00:45.579549+0000","last_unstale":"2024-01-31T05:00:51.679389+0000","last_undegraded":"2024-01-31T05:00:51.679389+0000","last_fullsized":"2024-01-31T05:00:51.679389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:43.683478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674795+0000","last_change":"2024-01-31T05:00:50.649333+0000","last_active":"2024-01-31T05:00:51.674795+0000","last_peered":"2024-01-31T05:00:51.674795+0000","last_clean":"2024-01-31T05:00:51.674795+0000","last_became_active":"2024-01-31T05:00:50.648927+0000","last_became_peered":"2024-01-31T05:00:50.648927+0000","last_unstale":"2024-01-31T05:00:51.674795+0000","last_undegraded":"2024-01-31T05:00:51.674795+0000","last_fullsized":"2024-01-31T05:00:51.674795+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918351+0000","last_change":"2024-01-31T05:00:46.581800+0000","last_active":"2024-01-31T05:00:51.918351+0000","last_peered":"2024-01-31T05:00:51.918351+0000","last_clean":"2024-01-31T05:00:51.918351+0000","last_became_active":"2024-01-31T05:00:46.581435+0000","last_became_peered":"2024-01-31T05:00:46.581435+0000","last_unstale":"2024-01-31T05:00:51.918351+0000","last_undegraded":"2024-01-31T05:00:51.918351+0000","last_fullsized":"2024-01-31T05:00:51.918351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:55:29.033845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674998+0000","last_change":"2024-01-31T05:00:45.570866+0000","last_active":"2024-01-31T05:00:51.674998+0000","last_peered":"2024-01-31T05:00:51.674998+0000","last_clean":"2024-01-31T05:00:51.674998+0000","last_became_active":"2024-01-31T05:00:45.570644+0000","last_became_peered":"2024-01-31T05:00:45.570644+0000","last_unstale":"2024-01-31T05:00:51.674998+0000","last_undegraded":"2024-01-31T05:00:51.674998+0000","last_fullsized":"2024-01-31T05:00:51.674998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:54.101266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917676+0000","last_change":"2024-01-31T05:00:50.641145+0000","last_active":"2024-01-31T05:00:51.917676+0000","last_peered":"2024-01-31T05:00:51.917676+0000","last_clean":"2024-01-31T05:00:51.917676+0000","last_became_active":"2024-01-31T05:00:50.638626+0000","last_became_peered":"2024-01-31T05:00:50.638626+0000","last_unstale":"2024-01-31T05:00:51.917676+0000","last_undegraded":"2024-01-31T05:00:51.917676+0000","last_fullsized":"2024-01-31T05:00:51.917676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:46:01.447285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914572+0000","last_change":"2024-01-31T05:00:46.589437+0000","last_active":"2024-01-31T05:00:51.914572+0000","last_peered":"2024-01-31T05:00:51.914572+0000","last_clean":"2024-01-31T05:00:51.914572+0000","last_became_active":"2024-01-31T05:00:46.589134+0000","last_became_peered":"2024-01-31T05:00:46.589134+0000","last_unstale":"2024-01-31T05:00:51.914572+0000","last_undegraded":"2024-01-31T05:00:51.914572+0000","last_fullsized":"2024-01-31T05:00:51.914572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680322+0000","last_change":"2024-01-31T05:00:45.579033+0000","last_active":"2024-01-31T05:00:51.680322+0000","last_peered":"2024-01-31T05:00:51.680322+0000","last_clean":"2024-01-31T05:00:51.680322+0000","last_became_active":"2024-01-31T05:00:45.578730+0000","last_became_peered":"2024-01-31T05:00:45.578730+0000","last_unstale":"2024-01-31T05:00:51.680322+0000","last_undegraded":"2024-01-31T05:00:51.680322+0000","last_fullsized":"2024-01-31T05:00:51.680322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:25:07.240664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916714+0000","last_change":"2024-01-31T05:00:50.639581+0000","last_active":"2024-01-31T05:00:51.916714+0000","last_peered":"2024-01-31T05:00:51.916714+0000","last_clean":"2024-01-31T05:00:51.916714+0000","last_became_active":"2024-01-31T05:00:50.639192+0000","last_became_peered":"2024-01-31T05:00:50.639192+0000","last_unstale":"2024-01-31T05:00:51.916714+0000","last_undegraded":"2024-01-31T05:00:51.916714+0000","last_fullsized":"2024-01-31T05:00:51.916714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:28.385448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915081+0000","last_change":"2024-01-31T05:00:46.590496+0000","last_active":"2024-01-31T05:00:51.915081+0000","last_peered":"2024-01-31T05:00:51.915081+0000","last_clean":"2024-01-31T05:00:51.915081+0000","last_became_active":"2024-01-31T05:00:46.587386+0000","last_became_peered":"2024-01-31T05:00:46.587386+0000","last_unstale":"2024-01-31T05:00:51.915081+0000","last_undegraded":"2024-01-31T05:00:51.915081+0000","last_fullsized":"2024-01-31T05:00:51.915081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674054+0000","last_change":"2024-01-31T05:00:45.580484+0000","last_active":"2024-01-31T05:00:51.674054+0000","last_peered":"2024-01-31T05:00:51.674054+0000","last_clean":"2024-01-31T05:00:51.674054+0000","last_became_active":"2024-01-31T05:00:45.580179+0000","last_became_peered":"2024-01-31T05:00:45.580179+0000","last_unstale":"2024-01-31T05:00:51.674054+0000","last_undegraded":"2024-01-31T05:00:51.674054+0000","last_fullsized":"2024-01-31T05:00:51.674054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:54.780283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918220+0000","last_change":"2024-01-31T05:00:50.633651+0000","last_active":"2024-01-31T05:00:51.918220+0000","last_peered":"2024-01-31T05:00:51.918220+0000","last_clean":"2024-01-31T05:00:51.918220+0000","last_became_active":"2024-01-31T05:00:50.633176+0000","last_became_peered":"2024-01-31T05:00:50.633176+0000","last_unstale":"2024-01-31T05:00:51.918220+0000","last_undegraded":"2024-01-31T05:00:51.918220+0000","last_fullsized":"2024-01-31T05:00:51.918220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:41.984643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.675265+0000","last_change":"2024-01-31T05:00:46.578636+0000","last_active":"2024-01-31T05:00:51.675265+0000","last_peered":"2024-01-31T05:00:51.675265+0000","last_clean":"2024-01-31T05:00:51.675265+0000","last_became_active":"2024-01-31T05:00:46.577232+0000","last_became_peered":"2024-01-31T05:00:46.577232+0000","last_unstale":"2024-01-31T05:00:51.675265+0000","last_undegraded":"2024-01-31T05:00:51.675265+0000","last_fullsized":"2024-01-31T05:00:51.675265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:37:22.658839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915084+0000","last_change":"2024-01-31T05:00:45.574367+0000","last_active":"2024-01-31T05:00:51.915084+0000","last_peered":"2024-01-31T05:00:51.915084+0000","last_clean":"2024-01-31T05:00:51.915084+0000","last_became_active":"2024-01-31T05:00:45.574010+0000","last_became_peered":"2024-01-31T05:00:45.574010+0000","last_unstale":"2024-01-31T05:00:51.915084+0000","last_undegraded":"2024-01-31T05:00:51.915084+0000","last_fullsized":"2024-01-31T05:00:51.915084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:12:03.585718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918910+0000","last_change":"2024-01-31T05:00:50.642311+0000","last_active":"2024-01-31T05:00:51.918910+0000","last_peered":"2024-01-31T05:00:51.918910+0000","last_clean":"2024-01-31T05:00:51.918910+0000","last_became_active":"2024-01-31T05:00:50.641937+0000","last_became_peered":"2024-01-31T05:00:50.641937+0000","last_unstale":"2024-01-31T05:00:51.918910+0000","last_undegraded":"2024-01-31T05:00:51.918910+0000","last_fullsized":"2024-01-31T05:00:51.918910+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:57.841272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916940+0000","last_change":"2024-01-31T05:00:46.589298+0000","last_active":"2024-01-31T05:00:51.916940+0000","last_peered":"2024-01-31T05:00:51.916940+0000","last_clean":"2024-01-31T05:00:51.916940+0000","last_became_active":"2024-01-31T05:00:46.589007+0000","last_became_peered":"2024-01-31T05:00:46.589007+0000","last_unstale":"2024-01-31T05:00:51.916940+0000","last_undegraded":"2024-01-31T05:00:51.916940+0000","last_fullsized":"2024-01-31T05:00:51.916940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:53:18.216173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673208+0000","last_change":"2024-01-31T05:00:45.573415+0000","last_active":"2024-01-31T05:00:51.673208+0000","last_peered":"2024-01-31T05:00:51.673208+0000","last_clean":"2024-01-31T05:00:51.673208+0000","last_became_active":"2024-01-31T05:00:45.573080+0000","last_became_peered":"2024-01-31T05:00:45.573080+0000","last_unstale":"2024-01-31T05:00:51.673208+0000","last_undegraded":"2024-01-31T05:00:51.673208+0000","last_fullsized":"2024-01-31T05:00:51.673208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:22.222326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680907+0000","last_change":"2024-01-31T05:00:50.642960+0000","last_active":"2024-01-31T05:00:51.680907+0000","last_peered":"2024-01-31T05:00:51.680907+0000","last_clean":"2024-01-31T05:00:51.680907+0000","last_became_active":"2024-01-31T05:00:50.640025+0000","last_became_peered":"2024-01-31T05:00:50.640025+0000","last_unstale":"2024-01-31T05:00:51.680907+0000","last_undegraded":"2024-01-31T05:00:51.680907+0000","last_fullsized":"2024-01-31T05:00:51.680907+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:28:26.315820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677292+0000","last_change":"2024-01-31T05:00:46.592032+0000","last_active":"2024-01-31T05:00:51.677292+0000","last_peered":"2024-01-31T05:00:51.677292+0000","last_clean":"2024-01-31T05:00:51.677292+0000","last_became_active":"2024-01-31T05:00:46.588029+0000","last_became_peered":"2024-01-31T05:00:46.588029+0000","last_unstale":"2024-01-31T05:00:51.677292+0000","last_undegraded":"2024-01-31T05:00:51.677292+0000","last_fullsized":"2024-01-31T05:00:51.677292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:42.890080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673909+0000","last_change":"2024-01-31T05:00:45.562238+0000","last_active":"2024-01-31T05:00:51.673909+0000","last_peered":"2024-01-31T05:00:51.673909+0000","last_clean":"2024-01-31T05:00:51.673909+0000","last_became_active":"2024-01-31T05:00:45.561726+0000","last_became_peered":"2024-01-31T05:00:45.561726+0000","last_unstale":"2024-01-31T05:00:51.673909+0000","last_undegraded":"2024-01-31T05:00:51.673909+0000","last_fullsized":"2024-01-31T05:00:51.673909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:13.468671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912656+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:00:51.912656+0000","last_peered":"2024-01-31T05:00:51.912656+0000","last_clean":"2024-01-31T05:00:51.912656+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:00:51.912656+0000","last_undegraded":"2024-01-31T05:00:51.912656+0000","last_fullsized":"2024-01-31T05:00:51.912656+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674754+0000","last_change":"2024-01-31T05:00:50.629234+0000","last_active":"2024-01-31T05:00:51.674754+0000","last_peered":"2024-01-31T05:00:51.674754+0000","last_clean":"2024-01-31T05:00:51.674754+0000","last_became_active":"2024-01-31T05:00:50.628308+0000","last_became_peered":"2024-01-31T05:00:50.628308+0000","last_unstale":"2024-01-31T05:00:51.674754+0000","last_undegraded":"2024-01-31T05:00:51.674754+0000","last_fullsized":"2024-01-31T05:00:51.674754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:21:35.300095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674590+0000","last_change":"2024-01-31T05:00:46.583227+0000","last_active":"2024-01-31T05:00:51.674590+0000","last_peered":"2024-01-31T05:00:51.674590+0000","last_clean":"2024-01-31T05:00:51.674590+0000","last_became_active":"2024-01-31T05:00:46.582532+0000","last_became_peered":"2024-01-31T05:00:46.582532+0000","last_unstale":"2024-01-31T05:00:51.674590+0000","last_undegraded":"2024-01-31T05:00:51.674590+0000","last_fullsized":"2024-01-31T05:00:51.674590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:53:51.592799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918513+0000","last_change":"2024-01-31T05:00:45.583848+0000","last_active":"2024-01-31T05:00:51.918513+0000","last_peered":"2024-01-31T05:00:51.918513+0000","last_clean":"2024-01-31T05:00:51.918513+0000","last_became_active":"2024-01-31T05:00:45.583492+0000","last_became_peered":"2024-01-31T05:00:45.583492+0000","last_unstale":"2024-01-31T05:00:51.918513+0000","last_undegraded":"2024-01-31T05:00:51.918513+0000","last_fullsized":"2024-01-31T05:00:51.918513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912922+0000","last_change":"2024-01-31T05:00:50.642193+0000","last_active":"2024-01-31T05:00:51.912922+0000","last_peered":"2024-01-31T05:00:51.912922+0000","last_clean":"2024-01-31T05:00:51.912922+0000","last_became_active":"2024-01-31T05:00:50.639911+0000","last_became_peered":"2024-01-31T05:00:50.639911+0000","last_unstale":"2024-01-31T05:00:51.912922+0000","last_undegraded":"2024-01-31T05:00:51.912922+0000","last_fullsized":"2024-01-31T05:00:51.912922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:50:44.744734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915799+0000","last_change":"2024-01-31T05:00:46.580861+0000","last_active":"2024-01-31T05:00:51.915799+0000","last_peered":"2024-01-31T05:00:51.915799+0000","last_clean":"2024-01-31T05:00:51.915799+0000","last_became_active":"2024-01-31T05:00:46.580588+0000","last_became_peered":"2024-01-31T05:00:46.580588+0000","last_unstale":"2024-01-31T05:00:51.915799+0000","last_undegraded":"2024-01-31T05:00:51.915799+0000","last_fullsized":"2024-01-31T05:00:51.915799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681122+0000","last_change":"2024-01-31T05:00:45.567443+0000","last_active":"2024-01-31T05:00:51.681122+0000","last_peered":"2024-01-31T05:00:51.681122+0000","last_clean":"2024-01-31T05:00:51.681122+0000","last_became_active":"2024-01-31T05:00:45.567160+0000","last_became_peered":"2024-01-31T05:00:45.567160+0000","last_unstale":"2024-01-31T05:00:51.681122+0000","last_undegraded":"2024-01-31T05:00:51.681122+0000","last_fullsized":"2024-01-31T05:00:51.681122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:42.144047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679437+0000","last_change":"2024-01-31T05:00:50.647210+0000","last_active":"2024-01-31T05:00:51.679437+0000","last_peered":"2024-01-31T05:00:51.679437+0000","last_clean":"2024-01-31T05:00:51.679437+0000","last_became_active":"2024-01-31T05:00:50.646792+0000","last_became_peered":"2024-01-31T05:00:50.646792+0000","last_unstale":"2024-01-31T05:00:51.679437+0000","last_undegraded":"2024-01-31T05:00:51.679437+0000","last_fullsized":"2024-01-31T05:00:51.679437+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:37:48.123083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915671+0000","last_change":"2024-01-31T05:00:46.593737+0000","last_active":"2024-01-31T05:00:51.915671+0000","last_peered":"2024-01-31T05:00:51.915671+0000","last_clean":"2024-01-31T05:00:51.915671+0000","last_became_active":"2024-01-31T05:00:46.593484+0000","last_became_peered":"2024-01-31T05:00:46.593484+0000","last_unstale":"2024-01-31T05:00:51.915671+0000","last_undegraded":"2024-01-31T05:00:51.915671+0000","last_fullsized":"2024-01-31T05:00:51.915671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:38:14.536828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679315+0000","last_change":"2024-01-31T05:00:45.582469+0000","last_active":"2024-01-31T05:00:51.679315+0000","last_peered":"2024-01-31T05:00:51.679315+0000","last_clean":"2024-01-31T05:00:51.679315+0000","last_became_active":"2024-01-31T05:00:45.582151+0000","last_became_peered":"2024-01-31T05:00:45.582151+0000","last_unstale":"2024-01-31T05:00:51.679315+0000","last_undegraded":"2024-01-31T05:00:51.679315+0000","last_fullsized":"2024-01-31T05:00:51.679315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:36:17.941437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672556+0000","last_change":"2024-01-31T05:00:50.642993+0000","last_active":"2024-01-31T05:00:51.672556+0000","last_peered":"2024-01-31T05:00:51.672556+0000","last_clean":"2024-01-31T05:00:51.672556+0000","last_became_active":"2024-01-31T05:00:50.642558+0000","last_became_peered":"2024-01-31T05:00:50.642558+0000","last_unstale":"2024-01-31T05:00:51.672556+0000","last_undegraded":"2024-01-31T05:00:51.672556+0000","last_fullsized":"2024-01-31T05:00:51.672556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:07:54.060532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914870+0000","last_change":"2024-01-31T05:00:46.597235+0000","last_active":"2024-01-31T05:00:51.914870+0000","last_peered":"2024-01-31T05:00:51.914870+0000","last_clean":"2024-01-31T05:00:51.914870+0000","last_became_active":"2024-01-31T05:00:46.592597+0000","last_became_peered":"2024-01-31T05:00:46.592597+0000","last_unstale":"2024-01-31T05:00:51.914870+0000","last_undegraded":"2024-01-31T05:00:51.914870+0000","last_fullsized":"2024-01-31T05:00:51.914870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:51:55.281235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915788+0000","last_change":"2024-01-31T05:00:45.583433+0000","last_active":"2024-01-31T05:00:51.915788+0000","last_peered":"2024-01-31T05:00:51.915788+0000","last_clean":"2024-01-31T05:00:51.915788+0000","last_became_active":"2024-01-31T05:00:45.582873+0000","last_became_peered":"2024-01-31T05:00:45.582873+0000","last_unstale":"2024-01-31T05:00:51.915788+0000","last_undegraded":"2024-01-31T05:00:51.915788+0000","last_fullsized":"2024-01-31T05:00:51.915788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:44:30.759721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913367+0000","last_change":"2024-01-31T05:00:50.651478+0000","last_active":"2024-01-31T05:00:51.913367+0000","last_peered":"2024-01-31T05:00:51.913367+0000","last_clean":"2024-01-31T05:00:51.913367+0000","last_became_active":"2024-01-31T05:00:50.650775+0000","last_became_peered":"2024-01-31T05:00:50.650775+0000","last_unstale":"2024-01-31T05:00:51.913367+0000","last_undegraded":"2024-01-31T05:00:51.913367+0000","last_fullsized":"2024-01-31T05:00:51.913367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672742+0000","last_change":"2024-01-31T05:00:46.588386+0000","last_active":"2024-01-31T05:00:51.672742+0000","last_peered":"2024-01-31T05:00:51.672742+0000","last_clean":"2024-01-31T05:00:51.672742+0000","last_became_active":"2024-01-31T05:00:46.588150+0000","last_became_peered":"2024-01-31T05:00:46.588150+0000","last_unstale":"2024-01-31T05:00:51.672742+0000","last_undegraded":"2024-01-31T05:00:51.672742+0000","last_fullsized":"2024-01-31T05:00:51.672742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:38.932253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674904+0000","last_change":"2024-01-31T05:00:45.572910+0000","last_active":"2024-01-31T05:00:51.674904+0000","last_peered":"2024-01-31T05:00:51.674904+0000","last_clean":"2024-01-31T05:00:51.674904+0000","last_became_active":"2024-01-31T05:00:45.570281+0000","last_became_peered":"2024-01-31T05:00:45.570281+0000","last_unstale":"2024-01-31T05:00:51.674904+0000","last_undegraded":"2024-01-31T05:00:51.674904+0000","last_fullsized":"2024-01-31T05:00:51.674904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681309+0000","last_change":"2024-01-31T05:00:50.646215+0000","last_active":"2024-01-31T05:00:51.681309+0000","last_peered":"2024-01-31T05:00:51.681309+0000","last_clean":"2024-01-31T05:00:51.681309+0000","last_became_active":"2024-01-31T05:00:50.640453+0000","last_became_peered":"2024-01-31T05:00:50.640453+0000","last_unstale":"2024-01-31T05:00:51.681309+0000","last_undegraded":"2024-01-31T05:00:51.681309+0000","last_fullsized":"2024-01-31T05:00:51.681309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:03:20.595164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912715+0000","last_change":"2024-01-31T05:00:46.582233+0000","last_active":"2024-01-31T05:00:51.912715+0000","last_peered":"2024-01-31T05:00:51.912715+0000","last_clean":"2024-01-31T05:00:51.912715+0000","last_became_active":"2024-01-31T05:00:46.581941+0000","last_became_peered":"2024-01-31T05:00:46.581941+0000","last_unstale":"2024-01-31T05:00:51.912715+0000","last_undegraded":"2024-01-31T05:00:51.912715+0000","last_fullsized":"2024-01-31T05:00:51.912715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:39:23.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914727+0000","last_change":"2024-01-31T05:00:45.586099+0000","last_active":"2024-01-31T05:00:51.914727+0000","last_peered":"2024-01-31T05:00:51.914727+0000","last_clean":"2024-01-31T05:00:51.914727+0000","last_became_active":"2024-01-31T05:00:45.583149+0000","last_became_peered":"2024-01-31T05:00:45.583149+0000","last_unstale":"2024-01-31T05:00:51.914727+0000","last_undegraded":"2024-01-31T05:00:51.914727+0000","last_fullsized":"2024-01-31T05:00:51.914727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:54.278929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681581+0000","last_change":"2024-01-31T05:00:50.644931+0000","last_active":"2024-01-31T05:00:51.681581+0000","last_peered":"2024-01-31T05:00:51.681581+0000","last_clean":"2024-01-31T05:00:51.681581+0000","last_became_active":"2024-01-31T05:00:50.644554+0000","last_became_peered":"2024-01-31T05:00:50.644554+0000","last_unstale":"2024-01-31T05:00:51.681581+0000","last_undegraded":"2024-01-31T05:00:51.681581+0000","last_fullsized":"2024-01-31T05:00:51.681581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:23:08.930803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673424+0000","last_change":"2024-01-31T05:00:46.582885+0000","last_active":"2024-01-31T05:00:51.673424+0000","last_peered":"2024-01-31T05:00:51.673424+0000","last_clean":"2024-01-31T05:00:51.673424+0000","last_became_active":"2024-01-31T05:00:46.582585+0000","last_became_peered":"2024-01-31T05:00:46.582585+0000","last_unstale":"2024-01-31T05:00:51.673424+0000","last_undegraded":"2024-01-31T05:00:51.673424+0000","last_fullsized":"2024-01-31T05:00:51.673424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:15:39.727609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673905+0000","last_change":"2024-01-31T05:00:45.570263+0000","last_active":"2024-01-31T05:00:51.673905+0000","last_peered":"2024-01-31T05:00:51.673905+0000","last_clean":"2024-01-31T05:00:51.673905+0000","last_became_active":"2024-01-31T05:00:45.568414+0000","last_became_peered":"2024-01-31T05:00:45.568414+0000","last_unstale":"2024-01-31T05:00:51.673905+0000","last_undegraded":"2024-01-31T05:00:51.673905+0000","last_fullsized":"2024-01-31T05:00:51.673905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:02.658111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918156+0000","last_change":"2024-01-31T05:00:50.640753+0000","last_active":"2024-01-31T05:00:51.918156+0000","last_peered":"2024-01-31T05:00:51.918156+0000","last_clean":"2024-01-31T05:00:51.918156+0000","last_became_active":"2024-01-31T05:00:50.640179+0000","last_became_peered":"2024-01-31T05:00:50.640179+0000","last_unstale":"2024-01-31T05:00:51.918156+0000","last_undegraded":"2024-01-31T05:00:51.918156+0000","last_fullsized":"2024-01-31T05:00:51.918156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:39.555756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916848+0000","last_change":"2024-01-31T05:00:46.596210+0000","last_active":"2024-01-31T05:00:51.916848+0000","last_peered":"2024-01-31T05:00:51.916848+0000","last_clean":"2024-01-31T05:00:51.916848+0000","last_became_active":"2024-01-31T05:00:46.595941+0000","last_became_peered":"2024-01-31T05:00:46.595941+0000","last_unstale":"2024-01-31T05:00:51.916848+0000","last_undegraded":"2024-01-31T05:00:51.916848+0000","last_fullsized":"2024-01-31T05:00:51.916848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:34:09.854108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916254+0000","last_change":"2024-01-31T05:00:45.577952+0000","last_active":"2024-01-31T05:00:51.916254+0000","last_peered":"2024-01-31T05:00:51.916254+0000","last_clean":"2024-01-31T05:00:51.916254+0000","last_became_active":"2024-01-31T05:00:45.577632+0000","last_became_peered":"2024-01-31T05:00:45.577632+0000","last_unstale":"2024-01-31T05:00:51.916254+0000","last_undegraded":"2024-01-31T05:00:51.916254+0000","last_fullsized":"2024-01-31T05:00:51.916254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673947+0000","last_change":"2024-01-31T05:00:50.639630+0000","last_active":"2024-01-31T05:00:51.673947+0000","last_peered":"2024-01-31T05:00:51.673947+0000","last_clean":"2024-01-31T05:00:51.673947+0000","last_became_active":"2024-01-31T05:00:50.639187+0000","last_became_peered":"2024-01-31T05:00:50.639187+0000","last_unstale":"2024-01-31T05:00:51.673947+0000","last_undegraded":"2024-01-31T05:00:51.673947+0000","last_fullsized":"2024-01-31T05:00:51.673947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:41:38.465126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915506+0000","last_change":"2024-01-31T05:00:46.595225+0000","last_active":"2024-01-31T05:00:51.915506+0000","last_peered":"2024-01-31T05:00:51.915506+0000","last_clean":"2024-01-31T05:00:51.915506+0000","last_became_active":"2024-01-31T05:00:46.594892+0000","last_became_peered":"2024-01-31T05:00:46.594892+0000","last_unstale":"2024-01-31T05:00:51.915506+0000","last_undegraded":"2024-01-31T05:00:51.915506+0000","last_fullsized":"2024-01-31T05:00:51.915506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:01:05.347110+0000","last_change":"2024-01-31T05:00:45.575073+0000","last_active":"2024-01-31T05:01:05.347110+0000","last_peered":"2024-01-31T05:01:05.347110+0000","last_clean":"2024-01-31T05:01:05.347110+0000","last_became_active":"2024-01-31T05:00:45.574725+0000","last_became_peered":"2024-01-31T05:00:45.574725+0000","last_unstale":"2024-01-31T05:01:05.347110+0000","last_undegraded":"2024-01-31T05:01:05.347110+0000","last_fullsized":"2024-01-31T05:01:05.347110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.343544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674916+0000","last_change":"2024-01-31T05:00:50.640744+0000","last_active":"2024-01-31T05:00:51.674916+0000","last_peered":"2024-01-31T05:00:51.674916+0000","last_clean":"2024-01-31T05:00:51.674916+0000","last_became_active":"2024-01-31T05:00:50.640181+0000","last_became_peered":"2024-01-31T05:00:50.640181+0000","last_unstale":"2024-01-31T05:00:51.674916+0000","last_undegraded":"2024-01-31T05:00:51.674916+0000","last_fullsized":"2024-01-31T05:00:51.674916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:11.749416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673600+0000","last_change":"2024-01-31T05:00:46.587774+0000","last_active":"2024-01-31T05:00:51.673600+0000","last_peered":"2024-01-31T05:00:51.673600+0000","last_clean":"2024-01-31T05:00:51.673600+0000","last_became_active":"2024-01-31T05:00:46.587495+0000","last_became_peered":"2024-01-31T05:00:46.587495+0000","last_unstale":"2024-01-31T05:00:51.673600+0000","last_undegraded":"2024-01-31T05:00:51.673600+0000","last_fullsized":"2024-01-31T05:00:51.673600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:19:31.923714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917086+0000","last_change":"2024-01-31T05:00:45.577387+0000","last_active":"2024-01-31T05:00:51.917086+0000","last_peered":"2024-01-31T05:00:51.917086+0000","last_clean":"2024-01-31T05:00:51.917086+0000","last_became_active":"2024-01-31T05:00:45.577035+0000","last_became_peered":"2024-01-31T05:00:45.577035+0000","last_unstale":"2024-01-31T05:00:51.917086+0000","last_undegraded":"2024-01-31T05:00:51.917086+0000","last_fullsized":"2024-01-31T05:00:51.917086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914130+0000","last_change":"2024-01-31T05:00:50.641531+0000","last_active":"2024-01-31T05:00:51.914130+0000","last_peered":"2024-01-31T05:00:51.914130+0000","last_clean":"2024-01-31T05:00:51.914130+0000","last_became_active":"2024-01-31T05:00:50.641164+0000","last_became_peered":"2024-01-31T05:00:50.641164+0000","last_unstale":"2024-01-31T05:00:51.914130+0000","last_undegraded":"2024-01-31T05:00:51.914130+0000","last_fullsized":"2024-01-31T05:00:51.914130+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:03.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681764+0000","last_change":"2024-01-31T05:00:46.584219+0000","last_active":"2024-01-31T05:00:51.681764+0000","last_peered":"2024-01-31T05:00:51.681764+0000","last_clean":"2024-01-31T05:00:51.681764+0000","last_became_active":"2024-01-31T05:00:46.583942+0000","last_became_peered":"2024-01-31T05:00:46.583942+0000","last_unstale":"2024-01-31T05:00:51.681764+0000","last_undegraded":"2024-01-31T05:00:51.681764+0000","last_fullsized":"2024-01-31T05:00:51.681764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:29:53.781070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915433+0000","last_change":"2024-01-31T05:00:45.576533+0000","last_active":"2024-01-31T05:00:51.915433+0000","last_peered":"2024-01-31T05:00:51.915433+0000","last_clean":"2024-01-31T05:00:51.915433+0000","last_became_active":"2024-01-31T05:00:45.576327+0000","last_became_peered":"2024-01-31T05:00:45.576327+0000","last_unstale":"2024-01-31T05:00:51.915433+0000","last_undegraded":"2024-01-31T05:00:51.915433+0000","last_fullsized":"2024-01-31T05:00:51.915433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:09:58.811691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914596+0000","last_change":"2024-01-31T05:00:50.654987+0000","last_active":"2024-01-31T05:00:51.914596+0000","last_peered":"2024-01-31T05:00:51.914596+0000","last_clean":"2024-01-31T05:00:51.914596+0000","last_became_active":"2024-01-31T05:00:50.654461+0000","last_became_peered":"2024-01-31T05:00:50.654461+0000","last_unstale":"2024-01-31T05:00:51.914596+0000","last_undegraded":"2024-01-31T05:00:51.914596+0000","last_fullsized":"2024-01-31T05:00:51.914596+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:22:41.218458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914829+0000","last_change":"2024-01-31T05:00:46.597875+0000","last_active":"2024-01-31T05:00:51.914829+0000","last_peered":"2024-01-31T05:00:51.914829+0000","last_clean":"2024-01-31T05:00:51.914829+0000","last_became_active":"2024-01-31T05:00:46.597494+0000","last_became_peered":"2024-01-31T05:00:46.597494+0000","last_unstale":"2024-01-31T05:00:51.914829+0000","last_undegraded":"2024-01-31T05:00:51.914829+0000","last_fullsized":"2024-01-31T05:00:51.914829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:26:14.648640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914275+0000","last_change":"2024-01-31T05:00:45.574121+0000","last_active":"2024-01-31T05:00:51.914275+0000","last_peered":"2024-01-31T05:00:51.914275+0000","last_clean":"2024-01-31T05:00:51.914275+0000","last_became_active":"2024-01-31T05:00:45.573715+0000","last_became_peered":"2024-01-31T05:00:45.573715+0000","last_unstale":"2024-01-31T05:00:51.914275+0000","last_undegraded":"2024-01-31T05:00:51.914275+0000","last_fullsized":"2024-01-31T05:00:51.914275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:52:48.186799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676164+0000","last_change":"2024-01-31T05:00:50.642176+0000","last_active":"2024-01-31T05:00:51.676164+0000","last_peered":"2024-01-31T05:00:51.676164+0000","last_clean":"2024-01-31T05:00:51.676164+0000","last_became_active":"2024-01-31T05:00:50.639134+0000","last_became_peered":"2024-01-31T05:00:50.639134+0000","last_unstale":"2024-01-31T05:00:51.676164+0000","last_undegraded":"2024-01-31T05:00:51.676164+0000","last_fullsized":"2024-01-31T05:00:51.676164+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:13:11.485002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676564+0000","last_change":"2024-01-31T05:00:50.649633+0000","last_active":"2024-01-31T05:00:51.676564+0000","last_peered":"2024-01-31T05:00:51.676564+0000","last_clean":"2024-01-31T05:00:51.676564+0000","last_became_active":"2024-01-31T05:00:50.647001+0000","last_became_peered":"2024-01-31T05:00:50.647001+0000","last_unstale":"2024-01-31T05:00:51.676564+0000","last_undegraded":"2024-01-31T05:00:51.676564+0000","last_fullsized":"2024-01-31T05:00:51.676564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:16:42.992455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676393+0000","last_change":"2024-01-31T05:00:46.567858+0000","last_active":"2024-01-31T05:00:51.676393+0000","last_peered":"2024-01-31T05:00:51.676393+0000","last_clean":"2024-01-31T05:00:51.676393+0000","last_became_active":"2024-01-31T05:00:46.567351+0000","last_became_peered":"2024-01-31T05:00:46.567351+0000","last_unstale":"2024-01-31T05:00:51.676393+0000","last_undegraded":"2024-01-31T05:00:51.676393+0000","last_fullsized":"2024-01-31T05:00:51.676393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:52:07.717707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673099+0000","last_change":"2024-01-31T05:00:45.562694+0000","last_active":"2024-01-31T05:00:51.673099+0000","last_peered":"2024-01-31T05:00:51.673099+0000","last_clean":"2024-01-31T05:00:51.673099+0000","last_became_active":"2024-01-31T05:00:45.560991+0000","last_became_peered":"2024-01-31T05:00:45.560991+0000","last_unstale":"2024-01-31T05:00:51.673099+0000","last_undegraded":"2024-01-31T05:00:51.673099+0000","last_fullsized":"2024-01-31T05:00:51.673099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:50.831619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674525+0000","last_change":"2024-01-31T05:00:50.638454+0000","last_active":"2024-01-31T05:00:51.674525+0000","last_peered":"2024-01-31T05:00:51.674525+0000","last_clean":"2024-01-31T05:00:51.674525+0000","last_became_active":"2024-01-31T05:00:50.637600+0000","last_became_peered":"2024-01-31T05:00:50.637600+0000","last_unstale":"2024-01-31T05:00:51.674525+0000","last_undegraded":"2024-01-31T05:00:51.674525+0000","last_fullsized":"2024-01-31T05:00:51.674525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:59:16.246206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676944+0000","last_change":"2024-01-31T05:00:46.574103+0000","last_active":"2024-01-31T05:00:51.676944+0000","last_peered":"2024-01-31T05:00:51.676944+0000","last_clean":"2024-01-31T05:00:51.676944+0000","last_became_active":"2024-01-31T05:00:46.572921+0000","last_became_peered":"2024-01-31T05:00:46.572921+0000","last_unstale":"2024-01-31T05:00:51.676944+0000","last_undegraded":"2024-01-31T05:00:51.676944+0000","last_fullsized":"2024-01-31T05:00:51.676944+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:36.342238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916504+0000","last_change":"2024-01-31T05:00:45.587813+0000","last_active":"2024-01-31T05:00:51.916504+0000","last_peered":"2024-01-31T05:00:51.916504+0000","last_clean":"2024-01-31T05:00:51.916504+0000","last_became_active":"2024-01-31T05:00:45.587473+0000","last_became_peered":"2024-01-31T05:00:45.587473+0000","last_unstale":"2024-01-31T05:00:51.916504+0000","last_undegraded":"2024-01-31T05:00:51.916504+0000","last_fullsized":"2024-01-31T05:00:51.916504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:44:11.745313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673739+0000","last_change":"2024-01-31T05:00:45.574390+0000","last_active":"2024-01-31T05:00:51.673739+0000","last_peered":"2024-01-31T05:00:51.673739+0000","last_clean":"2024-01-31T05:00:51.673739+0000","last_became_active":"2024-01-31T05:00:45.574121+0000","last_became_peered":"2024-01-31T05:00:45.574121+0000","last_unstale":"2024-01-31T05:00:51.673739+0000","last_undegraded":"2024-01-31T05:00:51.673739+0000","last_fullsized":"2024-01-31T05:00:51.673739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:01.042942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677784+0000","last_change":"2024-01-31T05:00:46.582109+0000","last_active":"2024-01-31T05:00:51.677784+0000","last_peered":"2024-01-31T05:00:51.677784+0000","last_clean":"2024-01-31T05:00:51.677784+0000","last_became_active":"2024-01-31T05:00:46.581814+0000","last_became_peered":"2024-01-31T05:00:46.581814+0000","last_unstale":"2024-01-31T05:00:51.677784+0000","last_undegraded":"2024-01-31T05:00:51.677784+0000","last_fullsized":"2024-01-31T05:00:51.677784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:40.921562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.678543+0000","last_change":"2024-01-31T05:00:50.652235+0000","last_active":"2024-01-31T05:00:51.678543+0000","last_peered":"2024-01-31T05:00:51.678543+0000","last_clean":"2024-01-31T05:00:51.678543+0000","last_became_active":"2024-01-31T05:00:50.650199+0000","last_became_peered":"2024-01-31T05:00:50.650199+0000","last_unstale":"2024-01-31T05:00:51.678543+0000","last_undegraded":"2024-01-31T05:00:51.678543+0000","last_fullsized":"2024-01-31T05:00:51.678543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:15.584111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679821+0000","last_change":"2024-01-31T05:00:45.580589+0000","last_active":"2024-01-31T05:00:51.679821+0000","last_peered":"2024-01-31T05:00:51.679821+0000","last_clean":"2024-01-31T05:00:51.679821+0000","last_became_active":"2024-01-31T05:00:45.579288+0000","last_became_peered":"2024-01-31T05:00:45.579288+0000","last_unstale":"2024-01-31T05:00:51.679821+0000","last_undegraded":"2024-01-31T05:00:51.679821+0000","last_fullsized":"2024-01-31T05:00:51.679821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:44.352083+0000","last_clean_scrub_stamp":"2024-01-31T05:00:44.352083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:11:00.549687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912730+0000","last_change":"2024-01-31T05:00:46.579010+0000","last_active":"2024-01-31T05:00:51.912730+0000","last_peered":"2024-01-31T05:00:51.912730+0000","last_clean":"2024-01-31T05:00:51.912730+0000","last_became_active":"2024-01-31T05:00:46.578792+0000","last_became_peered":"2024-01-31T05:00:46.578792+0000","last_unstale":"2024-01-31T05:00:51.912730+0000","last_undegraded":"2024-01-31T05:00:51.912730+0000","last_fullsized":"2024-01-31T05:00:51.912730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916113+0000","last_change":"2024-01-31T05:00:50.659322+0000","last_active":"2024-01-31T05:00:51.916113+0000","last_peered":"2024-01-31T05:00:51.916113+0000","last_clean":"2024-01-31T05:00:51.916113+0000","last_became_active":"2024-01-31T05:00:50.658743+0000","last_became_peered":"2024-01-31T05:00:50.658743+0000","last_unstale":"2024-01-31T05:00:51.916113+0000","last_undegraded":"2024-01-31T05:00:51.916113+0000","last_fullsized":"2024-01-31T05:00:51.916113+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:49.592255+0000","last_clean_scrub_stamp":"2024-01-31T05:00:49.592255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:42:20.923767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.913064+0000","last_change":"2024-01-31T05:00:46.579418+0000","last_active":"2024-01-31T05:00:51.913064+0000","last_peered":"2024-01-31T05:00:51.913064+0000","last_clean":"2024-01-31T05:00:51.913064+0000","last_became_active":"2024-01-31T05:00:46.579076+0000","last_became_peered":"2024-01-31T05:00:46.579076+0000","last_unstale":"2024-01-31T05:00:51.913064+0000","last_undegraded":"2024-01-31T05:00:51.913064+0000","last_fullsized":"2024-01-31T05:00:51.913064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914823+0000","last_change":"2024-01-31T05:00:46.585461+0000","last_active":"2024-01-31T05:00:51.914823+0000","last_peered":"2024-01-31T05:00:51.914823+0000","last_clean":"2024-01-31T05:00:51.914823+0000","last_became_active":"2024-01-31T05:00:46.584818+0000","last_became_peered":"2024-01-31T05:00:46.584818+0000","last_unstale":"2024-01-31T05:00:51.914823+0000","last_undegraded":"2024-01-31T05:00:51.914823+0000","last_fullsized":"2024-01-31T05:00:51.914823+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:02:52.011589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.673116+0000","last_change":"2024-01-31T05:00:46.586264+0000","last_active":"2024-01-31T05:00:51.673116+0000","last_peered":"2024-01-31T05:00:51.673116+0000","last_clean":"2024-01-31T05:00:51.673116+0000","last_became_active":"2024-01-31T05:00:46.585995+0000","last_became_peered":"2024-01-31T05:00:46.585995+0000","last_unstale":"2024-01-31T05:00:51.673116+0000","last_undegraded":"2024-01-31T05:00:51.673116+0000","last_fullsized":"2024-01-31T05:00:51.673116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:16:11.523909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.679572+0000","last_change":"2024-01-31T05:00:46.579199+0000","last_active":"2024-01-31T05:00:51.679572+0000","last_peered":"2024-01-31T05:00:51.679572+0000","last_clean":"2024-01-31T05:00:51.679572+0000","last_became_active":"2024-01-31T05:00:46.578257+0000","last_became_peered":"2024-01-31T05:00:46.578257+0000","last_unstale":"2024-01-31T05:00:51.679572+0000","last_undegraded":"2024-01-31T05:00:51.679572+0000","last_fullsized":"2024-01-31T05:00:51.679572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918235+0000","last_change":"2024-01-31T05:00:46.593320+0000","last_active":"2024-01-31T05:00:51.918235+0000","last_peered":"2024-01-31T05:00:51.918235+0000","last_clean":"2024-01-31T05:00:51.918235+0000","last_became_active":"2024-01-31T05:00:46.593053+0000","last_became_peered":"2024-01-31T05:00:46.593053+0000","last_unstale":"2024-01-31T05:00:51.918235+0000","last_undegraded":"2024-01-31T05:00:51.918235+0000","last_fullsized":"2024-01-31T05:00:51.918235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:50:21.326870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.918207+0000","last_change":"2024-01-31T05:00:46.583943+0000","last_active":"2024-01-31T05:00:51.918207+0000","last_peered":"2024-01-31T05:00:51.918207+0000","last_clean":"2024-01-31T05:00:51.918207+0000","last_became_active":"2024-01-31T05:00:46.580249+0000","last_became_peered":"2024-01-31T05:00:46.580249+0000","last_unstale":"2024-01-31T05:00:51.918207+0000","last_undegraded":"2024-01-31T05:00:51.918207+0000","last_fullsized":"2024-01-31T05:00:51.918207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914742+0000","last_change":"2024-01-31T05:00:46.601110+0000","last_active":"2024-01-31T05:00:51.914742+0000","last_peered":"2024-01-31T05:00:51.914742+0000","last_clean":"2024-01-31T05:00:51.914742+0000","last_became_active":"2024-01-31T05:00:46.600844+0000","last_became_peered":"2024-01-31T05:00:46.600844+0000","last_unstale":"2024-01-31T05:00:51.914742+0000","last_undegraded":"2024-01-31T05:00:51.914742+0000","last_fullsized":"2024-01-31T05:00:51.914742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:31:55.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.681192+0000","last_change":"2024-01-31T05:00:46.579792+0000","last_active":"2024-01-31T05:00:51.681192+0000","last_peered":"2024-01-31T05:00:51.681192+0000","last_clean":"2024-01-31T05:00:51.681192+0000","last_became_active":"2024-01-31T05:00:46.579357+0000","last_became_peered":"2024-01-31T05:00:46.579357+0000","last_unstale":"2024-01-31T05:00:51.681192+0000","last_undegraded":"2024-01-31T05:00:51.681192+0000","last_fullsized":"2024-01-31T05:00:51.681192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:49:35.495554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672904+0000","last_change":"2024-01-31T05:00:46.582507+0000","last_active":"2024-01-31T05:00:51.672904+0000","last_peered":"2024-01-31T05:00:51.672904+0000","last_clean":"2024-01-31T05:00:51.672904+0000","last_became_active":"2024-01-31T05:00:46.582243+0000","last_became_peered":"2024-01-31T05:00:46.582243+0000","last_unstale":"2024-01-31T05:00:51.672904+0000","last_undegraded":"2024-01-31T05:00:51.672904+0000","last_fullsized":"2024-01-31T05:00:51.672904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:07.689214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676563+0000","last_change":"2024-01-31T05:00:46.588900+0000","last_active":"2024-01-31T05:00:51.676563+0000","last_peered":"2024-01-31T05:00:51.676563+0000","last_clean":"2024-01-31T05:00:51.676563+0000","last_became_active":"2024-01-31T05:00:46.585994+0000","last_became_peered":"2024-01-31T05:00:46.585994+0000","last_unstale":"2024-01-31T05:00:51.676563+0000","last_undegraded":"2024-01-31T05:00:51.676563+0000","last_fullsized":"2024-01-31T05:00:51.676563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:45:47.058114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912637+0000","last_change":"2024-01-31T05:00:46.576223+0000","last_active":"2024-01-31T05:00:51.912637+0000","last_peered":"2024-01-31T05:00:51.912637+0000","last_clean":"2024-01-31T05:00:51.912637+0000","last_became_active":"2024-01-31T05:00:46.575962+0000","last_became_peered":"2024-01-31T05:00:46.575962+0000","last_unstale":"2024-01-31T05:00:51.912637+0000","last_undegraded":"2024-01-31T05:00:51.912637+0000","last_fullsized":"2024-01-31T05:00:51.912637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:16:46.637483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.677638+0000","last_change":"2024-01-31T05:00:46.576001+0000","last_active":"2024-01-31T05:00:51.677638+0000","last_peered":"2024-01-31T05:00:51.677638+0000","last_clean":"2024-01-31T05:00:51.677638+0000","last_became_active":"2024-01-31T05:00:46.574721+0000","last_became_peered":"2024-01-31T05:00:46.574721+0000","last_unstale":"2024-01-31T05:00:51.677638+0000","last_undegraded":"2024-01-31T05:00:51.677638+0000","last_fullsized":"2024-01-31T05:00:51.677638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:31:53.337974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915731+0000","last_change":"2024-01-31T05:00:46.589503+0000","last_active":"2024-01-31T05:00:51.915731+0000","last_peered":"2024-01-31T05:00:51.915731+0000","last_clean":"2024-01-31T05:00:51.915731+0000","last_became_active":"2024-01-31T05:00:46.581750+0000","last_became_peered":"2024-01-31T05:00:46.581750+0000","last_unstale":"2024-01-31T05:00:51.915731+0000","last_undegraded":"2024-01-31T05:00:51.915731+0000","last_fullsized":"2024-01-31T05:00:51.915731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:20:37.924441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674812+0000","last_change":"2024-01-31T05:00:46.582051+0000","last_active":"2024-01-31T05:00:51.674812+0000","last_peered":"2024-01-31T05:00:51.674812+0000","last_clean":"2024-01-31T05:00:51.674812+0000","last_became_active":"2024-01-31T05:00:46.578134+0000","last_became_peered":"2024-01-31T05:00:46.578134+0000","last_unstale":"2024-01-31T05:00:51.674812+0000","last_undegraded":"2024-01-31T05:00:51.674812+0000","last_fullsized":"2024-01-31T05:00:51.674812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:31.255109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917009+0000","last_change":"2024-01-31T05:00:46.587070+0000","last_active":"2024-01-31T05:00:51.917009+0000","last_peered":"2024-01-31T05:00:51.917009+0000","last_clean":"2024-01-31T05:00:51.917009+0000","last_became_active":"2024-01-31T05:00:46.586707+0000","last_became_peered":"2024-01-31T05:00:46.586707+0000","last_unstale":"2024-01-31T05:00:51.917009+0000","last_undegraded":"2024-01-31T05:00:51.917009+0000","last_fullsized":"2024-01-31T05:00:51.917009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674901+0000","last_change":"2024-01-31T05:00:46.587042+0000","last_active":"2024-01-31T05:00:51.674901+0000","last_peered":"2024-01-31T05:00:51.674901+0000","last_clean":"2024-01-31T05:00:51.674901+0000","last_became_active":"2024-01-31T05:00:46.586518+0000","last_became_peered":"2024-01-31T05:00:46.586518+0000","last_unstale":"2024-01-31T05:00:51.674901+0000","last_undegraded":"2024-01-31T05:00:51.674901+0000","last_fullsized":"2024-01-31T05:00:51.674901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:54:17.597851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.672833+0000","last_change":"2024-01-31T05:00:46.593111+0000","last_active":"2024-01-31T05:00:51.672833+0000","last_peered":"2024-01-31T05:00:51.672833+0000","last_clean":"2024-01-31T05:00:51.672833+0000","last_became_active":"2024-01-31T05:00:46.592828+0000","last_became_peered":"2024-01-31T05:00:46.592828+0000","last_unstale":"2024-01-31T05:00:51.672833+0000","last_undegraded":"2024-01-31T05:00:51.672833+0000","last_fullsized":"2024-01-31T05:00:51.672833+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:59.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.676004+0000","last_change":"2024-01-31T05:00:46.589807+0000","last_active":"2024-01-31T05:00:51.676004+0000","last_peered":"2024-01-31T05:00:51.676004+0000","last_clean":"2024-01-31T05:00:51.676004+0000","last_became_active":"2024-01-31T05:00:46.589518+0000","last_became_peered":"2024-01-31T05:00:46.589518+0000","last_unstale":"2024-01-31T05:00:51.676004+0000","last_undegraded":"2024-01-31T05:00:51.676004+0000","last_fullsized":"2024-01-31T05:00:51.676004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:06:03.428645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.916672+0000","last_change":"2024-01-31T05:00:46.583014+0000","last_active":"2024-01-31T05:00:51.916672+0000","last_peered":"2024-01-31T05:00:51.916672+0000","last_clean":"2024-01-31T05:00:51.916672+0000","last_became_active":"2024-01-31T05:00:46.582607+0000","last_became_peered":"2024-01-31T05:00:46.582607+0000","last_unstale":"2024-01-31T05:00:51.916672+0000","last_undegraded":"2024-01-31T05:00:51.916672+0000","last_fullsized":"2024-01-31T05:00:51.916672+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:21:09.066019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.917869+0000","last_change":"2024-01-31T05:00:46.594160+0000","last_active":"2024-01-31T05:00:51.917869+0000","last_peered":"2024-01-31T05:00:51.917869+0000","last_clean":"2024-01-31T05:00:51.917869+0000","last_became_active":"2024-01-31T05:00:46.593838+0000","last_became_peered":"2024-01-31T05:00:46.593838+0000","last_unstale":"2024-01-31T05:00:51.917869+0000","last_undegraded":"2024-01-31T05:00:51.917869+0000","last_fullsized":"2024-01-31T05:00:51.917869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:58:27.273440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915637+0000","last_change":"2024-01-31T05:00:46.590079+0000","last_active":"2024-01-31T05:00:51.915637+0000","last_peered":"2024-01-31T05:00:51.915637+0000","last_clean":"2024-01-31T05:00:51.915637+0000","last_became_active":"2024-01-31T05:00:46.589828+0000","last_became_peered":"2024-01-31T05:00:46.589828+0000","last_unstale":"2024-01-31T05:00:51.915637+0000","last_undegraded":"2024-01-31T05:00:51.915637+0000","last_fullsized":"2024-01-31T05:00:51.915637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:57.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915410+0000","last_change":"2024-01-31T05:00:46.601762+0000","last_active":"2024-01-31T05:00:51.915410+0000","last_peered":"2024-01-31T05:00:51.915410+0000","last_clean":"2024-01-31T05:00:51.915410+0000","last_became_active":"2024-01-31T05:00:46.600161+0000","last_became_peered":"2024-01-31T05:00:46.600161+0000","last_unstale":"2024-01-31T05:00:51.915410+0000","last_undegraded":"2024-01-31T05:00:51.915410+0000","last_fullsized":"2024-01-31T05:00:51.915410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:32:40.045263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915285+0000","last_change":"2024-01-31T05:00:46.588560+0000","last_active":"2024-01-31T05:00:51.915285+0000","last_peered":"2024-01-31T05:00:51.915285+0000","last_clean":"2024-01-31T05:00:51.915285+0000","last_became_active":"2024-01-31T05:00:46.588291+0000","last_became_peered":"2024-01-31T05:00:46.588291+0000","last_unstale":"2024-01-31T05:00:51.915285+0000","last_undegraded":"2024-01-31T05:00:51.915285+0000","last_fullsized":"2024-01-31T05:00:51.915285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914247+0000","last_change":"2024-01-31T05:00:46.570493+0000","last_active":"2024-01-31T05:00:51.914247+0000","last_peered":"2024-01-31T05:00:51.914247+0000","last_clean":"2024-01-31T05:00:51.914247+0000","last_became_active":"2024-01-31T05:00:46.570196+0000","last_became_peered":"2024-01-31T05:00:46.570196+0000","last_unstale":"2024-01-31T05:00:51.914247+0000","last_undegraded":"2024-01-31T05:00:51.914247+0000","last_fullsized":"2024-01-31T05:00:51.914247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.674510+0000","last_change":"2024-01-31T05:00:46.580076+0000","last_active":"2024-01-31T05:00:51.674510+0000","last_peered":"2024-01-31T05:00:51.674510+0000","last_clean":"2024-01-31T05:00:51.674510+0000","last_became_active":"2024-01-31T05:00:46.578312+0000","last_became_peered":"2024-01-31T05:00:46.578312+0000","last_unstale":"2024-01-31T05:00:51.674510+0000","last_undegraded":"2024-01-31T05:00:51.674510+0000","last_fullsized":"2024-01-31T05:00:51.674510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:39.690990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914636+0000","last_change":"2024-01-31T05:00:46.577353+0000","last_active":"2024-01-31T05:00:51.914636+0000","last_peered":"2024-01-31T05:00:51.914636+0000","last_clean":"2024-01-31T05:00:51.914636+0000","last_became_active":"2024-01-31T05:00:46.577019+0000","last_became_peered":"2024-01-31T05:00:46.577019+0000","last_unstale":"2024-01-31T05:00:51.914636+0000","last_undegraded":"2024-01-31T05:00:51.914636+0000","last_fullsized":"2024-01-31T05:00:51.914636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:09:13.929721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.912937+0000","last_change":"2024-01-31T05:00:46.575406+0000","last_active":"2024-01-31T05:00:51.912937+0000","last_peered":"2024-01-31T05:00:51.912937+0000","last_clean":"2024-01-31T05:00:51.912937+0000","last_became_active":"2024-01-31T05:00:46.575056+0000","last_became_peered":"2024-01-31T05:00:46.575056+0000","last_unstale":"2024-01-31T05:00:51.912937+0000","last_undegraded":"2024-01-31T05:00:51.912937+0000","last_fullsized":"2024-01-31T05:00:51.912937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:02.222912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915174+0000","last_change":"2024-01-31T05:00:46.582321+0000","last_active":"2024-01-31T05:00:51.915174+0000","last_peered":"2024-01-31T05:00:51.915174+0000","last_clean":"2024-01-31T05:00:51.915174+0000","last_became_active":"2024-01-31T05:00:46.581547+0000","last_became_peered":"2024-01-31T05:00:46.581547+0000","last_unstale":"2024-01-31T05:00:51.915174+0000","last_undegraded":"2024-01-31T05:00:51.915174+0000","last_fullsized":"2024-01-31T05:00:51.915174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.914144+0000","last_change":"2024-01-31T05:00:46.575296+0000","last_active":"2024-01-31T05:00:51.914144+0000","last_peered":"2024-01-31T05:00:51.914144+0000","last_clean":"2024-01-31T05:00:51.914144+0000","last_became_active":"2024-01-31T05:00:46.574983+0000","last_became_peered":"2024-01-31T05:00:46.574983+0000","last_unstale":"2024-01-31T05:00:51.914144+0000","last_undegraded":"2024-01-31T05:00:51.914144+0000","last_fullsized":"2024-01-31T05:00:51.914144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:09:03.838081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680670+0000","last_change":"2024-01-31T05:00:46.574733+0000","last_active":"2024-01-31T05:00:51.680670+0000","last_peered":"2024-01-31T05:00:51.680670+0000","last_clean":"2024-01-31T05:00:51.680670+0000","last_became_active":"2024-01-31T05:00:46.574335+0000","last_became_peered":"2024-01-31T05:00:46.574335+0000","last_unstale":"2024-01-31T05:00:51.680670+0000","last_undegraded":"2024-01-31T05:00:51.680670+0000","last_fullsized":"2024-01-31T05:00:51.680670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:24.325246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.680984+0000","last_change":"2024-01-31T05:00:46.578081+0000","last_active":"2024-01-31T05:00:51.680984+0000","last_peered":"2024-01-31T05:00:51.680984+0000","last_clean":"2024-01-31T05:00:51.680984+0000","last_became_active":"2024-01-31T05:00:46.577576+0000","last_became_peered":"2024-01-31T05:00:46.577576+0000","last_unstale":"2024-01-31T05:00:51.680984+0000","last_undegraded":"2024-01-31T05:00:51.680984+0000","last_fullsized":"2024-01-31T05:00:51.680984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:15.514999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-01-31T05:00:51.915608+0000","last_change":"2024-01-31T05:00:46.592072+0000","last_active":"2024-01-31T05:00:51.915608+0000","last_peered":"2024-01-31T05:00:51.915608+0000","last_clean":"2024-01-31T05:00:51.915608+0000","last_became_active":"2024-01-31T05:00:46.591682+0000","last_became_peered":"2024-01-31T05:00:46.591682+0000","last_unstale":"2024-01-31T05:00:51.915608+0000","last_undegraded":"2024-01-31T05:00:51.915608+0000","last_fullsized":"2024-01-31T05:00:51.915608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:45.532053+0000","last_clean_scrub_stamp":"2024-01-31T05:00:45.532053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:13:21.174300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":106194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2789999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.728}]}]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":106652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":282624,"data_stored":109911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6000000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":696508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0379999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4169999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5270000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8319999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.264}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3859999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":106126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.302}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9860000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.262}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4220000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:01:16.283 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-31T05:01:16.283 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-31T05:01:16.283 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-31T05:01:16.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-31T05:01:16.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.522+0000 7f2bf7000640 1 Processor -- start 2024-01-31T05:01:16.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.522+0000 7f2bf7000640 1 -- start start 2024-01-31T05:01:16.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.523+0000 7f2bf7000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bf015a600 0x7f2bf015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.523+0000 7f2bf7000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf006c8b0 con 0x7f2bf00b1d10 2024-01-31T05:01:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.523+0000 7f2bf7000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf015b120 con 0x7f2bf006d090 2024-01-31T05:01:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.523+0000 7f2bf7000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf015b400 con 0x7f2bf015a600 2024-01-31T05:01:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.523+0000 7f2befe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2bf00b1d10 0x7f2bf00b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33132/0 (socket says 172.21.15.97:33132) 2024-01-31T05:01:16.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.523+0000 7f2befe00640 1 -- 172.21.15.97:0/2145272463 learned_addr learned my addr 172.21.15.97:0/2145272463 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334747634 0 0) 0x7f2bf006c8b0 con 0x7f2bf00b1d10 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd0003520 con 0x7f2bf00b1d10 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2be7200640 1 --2- 172.21.15.97:0/2145272463 >> v2:172.21.15.188:3300/0 conn(0x7f2bf015a600 0x7f2bf015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2785411522 0 0) 0x7f2bd0003520 con 0x7f2bf00b1d10 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 >> v2:172.21.15.188:3300/0 conn(0x7f2bf015a600 msgr2=0x7f2bf015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 --2- 172.21.15.97:0/2145272463 >> v2:172.21.15.188:3300/0 conn(0x7f2bf015a600 0x7f2bf015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 >> v1:172.21.15.188:6789/0 conn(0x7f2bf006d090 legacy=0x7f2bf006d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf006cb70 con 0x7f2bf00b1d10 2024-01-31T05:01:16.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f2be0001670 con 0x7f2bf00b1d10 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 >> v1:172.21.15.97:6789/0 conn(0x7f2bf00b1d10 legacy=0x7f2bf00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bd00047c0 0x7f2bd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bd0005180 0x7f2bd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf015b400 con 0x7f2bd0003970 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000c0d0 con 0x7f2bd0005180 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.524+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000c290 con 0x7f2bd00047c0 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2befe00640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2befe00640 1 -- 172.21.15.97:0/2145272463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bd00047c0 msgr2=0x7f2bd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2befe00640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bd00047c0 0x7f2bd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2befe00640 1 -- 172.21.15.97:0/2145272463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bd0005180 msgr2=0x7f2bd000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2befe00640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bd0005180 0x7f2bd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2befe00640 1 -- 172.21.15.97:0/2145272463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0003520 con 0x7f2bd0003970 2024-01-31T05:01:16.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.525+0000 7f2bf5000640 1 -- 172.21.15.97:0/2145272463 reap_dead start 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.526+0000 7f2befe00640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bd0003d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2be000c240 tx=0x7f2bd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd84e6116a28d8dc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.526+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0005ad0 con 0x7f2bd0003970 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.526+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be0005c50 con 0x7f2bd0003970 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.526+0000 7f2bef400640 1 -- 172.21.15.97:0/2145272463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be002c8b0 con 0x7f2bd0003970 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 -- 172.21.15.97:0/2145272463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 msgr2=0x7f2bd0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bd0003d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2be000c240 tx=0x7f2bd00057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 -- 172.21.15.97:0/2145272463 shutdown_connections 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 --2- 172.21.15.97:0/2145272463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bd0003d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 -- 172.21.15.97:0/2145272463 >> 172.21.15.97:0/2145272463 conn(0x7f2bf00b0a10 msgr2=0x7f2bf00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 -- 172.21.15.97:0/2145272463 shutdown_connections 2024-01-31T05:01:16.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.527+0000 7f2bf7000640 1 -- 172.21.15.97:0/2145272463 wait complete. 2024-01-31T05:01:16.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.529+0000 7f2bf7000640 1 Processor -- start 2024-01-31T05:01:16.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.529+0000 7f2bf7000640 1 -- start start 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2bf7000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bf014fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2bf7000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf00723b0 0x7f2bf00cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2bf7000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf015b6e0 0x7f2bf00af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2bf7000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf00cbbd0 con 0x7f2bd0003970 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2bf7000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0159660 con 0x7f2bf015b6e0 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2bf7000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0159940 con 0x7f2bf00723b0 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2befe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bf014fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2befe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bf014fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34370/0 (socket says 172.21.15.97:34370) 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2befe00640 1 -- 172.21.15.97:0/718186253 learned_addr learned my addr 172.21.15.97:0/718186253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2be7200640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf00723b0 0x7f2bf00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2befe00640 1 -- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf00723b0 msgr2=0x7f2bf00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.530+0000 7f2befe00640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf00723b0 0x7f2bf00cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2befe00640 1 -- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf015b6e0 msgr2=0x7f2bf00af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2befe00640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf015b6e0 0x7f2bf00af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2befe00640 1 -- 172.21.15.97:0/718186253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0159c20 con 0x7f2bd0003970 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2befe00640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bf014fcd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2be0001930 tx=0x7f2be002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be003c040 con 0x7f2bd0003970 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be002cd80 con 0x7f2bd0003970 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0007290 con 0x7f2bd0003970 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf0150210 con 0x7f2bd0003970 2024-01-31T05:01:16.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.531+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf00aff10 con 0x7f2bd0003970 2024-01-31T05:01:16.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.532+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf00677c0 con 0x7f2bd0003970 2024-01-31T05:01:16.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.533+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2be0032070 con 0x7f2bd0003970 2024-01-31T05:01:16.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.534+0000 7f2becc00640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2bc806aed0 0x7f2bc806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.534+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2be00aaf20 con 0x7f2bd0003970 2024-01-31T05:01:16.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.534+0000 7f2be7200640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2bc806aed0 0x7f2bc806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.535+0000 7f2be7200640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2bc806aed0 0x7f2bc806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2bf009d010 tx=0x7f2bd800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.538+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2be0075af0 con 0x7f2bd0003970 2024-01-31T05:01:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.750+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2bf006e740 con 0x7f2bd0003970 2024-01-31T05:01:16.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.751+0000 7f2becc00640 1 -- 172.21.15.97:0/718186253 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2be0046080 con 0x7f2bd0003970 2024-01-31T05:01:16.751 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:16.751 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-31T05:01:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.753+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2bc806aed0 msgr2=0x7f2bc806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.753+0000 7f2bf7000640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2bc806aed0 0x7f2bc806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2bf009d010 tx=0x7f2bd800b040 comp rx=0 tx=0).stop 2024-01-31T05:01:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.753+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 msgr2=0x7f2bf014fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.753+0000 7f2bf7000640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bf014fcd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2be0001930 tx=0x7f2be002e070 comp rx=0 tx=0).stop 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 shutdown_connections 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf015b6e0 0x7f2bf00af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf00723b0 0x7f2bf00cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2bc806aed0 0x7f2bc806d390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 --2- 172.21.15.97:0/718186253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bd0003970 0x7f2bf014fcd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 >> 172.21.15.97:0/718186253 conn(0x7f2bf00b0a10 msgr2=0x7f2bf00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 shutdown_connections 2024-01-31T05:01:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.754+0000 7f2bf7000640 1 -- 172.21.15.97:0/718186253 wait complete. 2024-01-31T05:01:16.771 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-31T05:01:16.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.978+0000 7f3846200640 1 Processor -- start 2024-01-31T05:01:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.978+0000 7f3846200640 1 -- start start 2024-01-31T05:01:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.979+0000 7f3846200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38400cb810 0x7f38400cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.979+0000 7f3846200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400ccd00 0x7f38400cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.979+0000 7f3846200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384006d1a0 con 0x7f384015b400 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.979+0000 7f3846200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38400cc960 con 0x7f38400cb810 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.979+0000 7f3846200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400cd6f0 con 0x7f38400ccd00 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38400cb810 0x7f38400cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38400cb810 0x7f38400cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34386/0 (socket says 172.21.15.97:34386) 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 -- 172.21.15.97:0/449193112 learned_addr learned my addr 172.21.15.97:0/449193112 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 -- 172.21.15.97:0/449193112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400ccd00 msgr2=0x7f38400cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 --2- 172.21.15.97:0/449193112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400ccd00 0x7f38400cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 -- 172.21.15.97:0/449193112 >> v1:172.21.15.188:6789/0 conn(0x7f384015b400 legacy=0x7f384006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.980+0000 7f383f400640 1 -- 172.21.15.97:0/449193112 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384006c600 con 0x7f38400cb810 2024-01-31T05:01:16.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.981+0000 7f383f400640 1 --2- 172.21.15.97:0/449193112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400cb810 0x7f38400cbbf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3830009870 tx=0x7f383002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1940c89573963b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f383ea00640 1 -- 172.21.15.97:0/449193112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830031640 con 0x7f38400cb810 2024-01-31T05:01:16.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f383ea00640 1 -- 172.21.15.97:0/449193112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38300317a0 con 0x7f38400cb810 2024-01-31T05:01:16.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f383ea00640 1 -- 172.21.15.97:0/449193112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830031b30 con 0x7f38400cb810 2024-01-31T05:01:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 -- 172.21.15.97:0/449193112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400cb810 msgr2=0x7f38400cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 --2- 172.21.15.97:0/449193112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400cb810 0x7f38400cbbf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3830009870 tx=0x7f383002b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 -- 172.21.15.97:0/449193112 shutdown_connections 2024-01-31T05:01:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 --2- 172.21.15.97:0/449193112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38400ccd00 0x7f38400cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 --2- 172.21.15.97:0/449193112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400cb810 0x7f38400cbbf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 -- 172.21.15.97:0/449193112 >> 172.21.15.97:0/449193112 conn(0x7f38400b0a10 msgr2=0x7f38400b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.982+0000 7f3846200640 1 -- 172.21.15.97:0/449193112 shutdown_connections 2024-01-31T05:01:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.983+0000 7f3846200640 1 -- 172.21.15.97:0/449193112 wait complete. 2024-01-31T05:01:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.984+0000 7f3846200640 1 Processor -- start 2024-01-31T05:01:16.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.984+0000 7f3846200640 1 -- start start 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.985+0000 7f3846200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400cb810 0x7f38400c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.985+0000 7f3846200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 0x7f384014e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.985+0000 7f3846200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384015b400 0x7f3840071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.985+0000 7f3846200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384014edc0 con 0x7f38400ccd00 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.985+0000 7f3846200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384015b8b0 con 0x7f38400cb810 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.985+0000 7f3846200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400cace0 con 0x7f384015b400 2024-01-31T05:01:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 0x7f384014e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 0x7f384014e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34398/0 (socket says 172.21.15.97:34398) 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 -- 172.21.15.97:0/1902828995 learned_addr learned my addr 172.21.15.97:0/1902828995 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f383f400640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400cb810 0x7f38400c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f383fe00640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384015b400 0x7f3840071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 -- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384015b400 msgr2=0x7f3840071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384015b400 0x7f3840071260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 -- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400cb810 msgr2=0x7f38400c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400cb810 0x7f38400c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 -- 172.21.15.97:0/1902828995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300094d0 con 0x7f38400ccd00 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.986+0000 7f3837e00640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 0x7f384014e880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f382800b3c0 tx=0x7f382800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.987+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828011830 con 0x7f38400ccd00 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.987+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3828011990 con 0x7f38400ccd00 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.987+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828011d80 con 0x7f38400ccd00 2024-01-31T05:01:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.987+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384015e2e0 con 0x7f38400ccd00 2024-01-31T05:01:16.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.987+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3840156770 con 0x7f38400ccd00 2024-01-31T05:01:16.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.988+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384015e140 con 0x7f38400ccd00 2024-01-31T05:01:16.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.989+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f382801b460 con 0x7f38400ccd00 2024-01-31T05:01:16.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.990+0000 7f3837400640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f381806aed0 0x7f381806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:16.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.990+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3828089560 con 0x7f38400ccd00 2024-01-31T05:01:16.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.990+0000 7f383f400640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f381806aed0 0x7f381806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:16.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.991+0000 7f383f400640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f381806aed0 0x7f381806d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3830036000 tx=0x7f383003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:16.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:16.994+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f384015e140 con 0x7f38400ccd00 2024-01-31T05:01:17.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.188+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f384015e140 con 0x7f38400ccd00 2024-01-31T05:01:17.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.189+0000 7f3837400640 1 -- 172.21.15.97:0/1902828995 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f384015e140 con 0x7f38400ccd00 2024-01-31T05:01:17.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:17.189 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f381806aed0 msgr2=0x7f381806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f381806aed0 0x7f381806d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3830036000 tx=0x7f383003c040 comp rx=0 tx=0).stop 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 msgr2=0x7f384014e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 0x7f384014e880 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f382800b3c0 tx=0x7f382800b880 comp rx=0 tx=0).stop 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 shutdown_connections 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f384015b400 0x7f3840071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f381806aed0 0x7f381806d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38400ccd00 0x7f384014e880 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 --2- 172.21.15.97:0/1902828995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38400cb810 0x7f38400c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 >> 172.21.15.97:0/1902828995 conn(0x7f38400b0a10 msgr2=0x7f38400b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:17.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 shutdown_connections 2024-01-31T05:01:17.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.192+0000 7f3846200640 1 -- 172.21.15.97:0/1902828995 wait complete. 2024-01-31T05:01:17.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:17.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:17.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.417+0000 7fed8fc00640 1 Processor -- start 2024-01-31T05:01:17.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.417+0000 7fed8fc00640 1 -- start start 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed8fc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed8815a730 0x7fed880abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed8fc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed880cde10 0x7fed880ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed8fc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8815abc0 con 0x7fed8815a350 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed8fc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880ce7e0 con 0x7fed8815a730 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed8fc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880b2c90 con 0x7fed880cde10 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed87e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed8815a730 0x7fed880abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed87e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed8815a730 0x7fed880abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34400/0 (socket says 172.21.15.97:34400) 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed87e00640 1 -- 172.21.15.97:0/2711306630 learned_addr learned my addr 172.21.15.97:0/2711306630 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:17.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.419+0000 7fed8dc00640 1 --2- 172.21.15.97:0/2711306630 >> v2:172.21.15.188:3300/0 conn(0x7fed880cde10 0x7fed880ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.420+0000 7fed87e00640 1 -- 172.21.15.97:0/2711306630 >> v2:172.21.15.188:3300/0 conn(0x7fed880cde10 msgr2=0x7fed880ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:17.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.420+0000 7fed87e00640 1 --2- 172.21.15.97:0/2711306630 >> v2:172.21.15.188:3300/0 conn(0x7fed880cde10 0x7fed880ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.420+0000 7fed87e00640 1 -- 172.21.15.97:0/2711306630 >> v1:172.21.15.188:6789/0 conn(0x7fed8815a350 legacy=0x7fed8806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.420+0000 7fed87e00640 1 -- 172.21.15.97:0/2711306630 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8815aea0 con 0x7fed8815a730 2024-01-31T05:01:17.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.421+0000 7fed87e00640 1 --2- 172.21.15.97:0/2711306630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8815a730 0x7fed880abcf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed880ac260 tx=0x7fed7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78a8ce8f42bc667 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:17.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.421+0000 7fed87400640 1 -- 172.21.15.97:0/2711306630 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70031870 con 0x7fed8815a730 2024-01-31T05:01:17.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.421+0000 7fed87400640 1 -- 172.21.15.97:0/2711306630 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed700319d0 con 0x7fed8815a730 2024-01-31T05:01:17.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.421+0000 7fed87400640 1 -- 172.21.15.97:0/2711306630 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70031d60 con 0x7fed8815a730 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 -- 172.21.15.97:0/2711306630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8815a730 msgr2=0x7fed880abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 --2- 172.21.15.97:0/2711306630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8815a730 0x7fed880abcf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fed880ac260 tx=0x7fed7002b910 comp rx=0 tx=0).stop 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 -- 172.21.15.97:0/2711306630 shutdown_connections 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 --2- 172.21.15.97:0/2711306630 >> v2:172.21.15.188:3300/0 conn(0x7fed880cde10 0x7fed880ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 --2- 172.21.15.97:0/2711306630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8815a730 0x7fed880abcf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 -- 172.21.15.97:0/2711306630 >> 172.21.15.97:0/2711306630 conn(0x7fed880b0a10 msgr2=0x7fed880b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 -- 172.21.15.97:0/2711306630 shutdown_connections 2024-01-31T05:01:17.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.422+0000 7fed8fc00640 1 -- 172.21.15.97:0/2711306630 wait complete. 2024-01-31T05:01:17.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.424+0000 7fed8fc00640 1 Processor -- start 2024-01-31T05:01:17.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.424+0000 7fed8fc00640 1 -- start start 2024-01-31T05:01:17.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed880cde10 0x7fed8814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8fc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed8815a350 0x7fed880715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 0x7fed88071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed88158b30 con 0x7fed8809bfa0 2024-01-31T05:01:17.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8fc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8809c510 con 0x7fed8815a350 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed88159690 con 0x7fed880cde10 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 0x7fed88071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 0x7fed88071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34410/0 (socket says 172.21.15.97:34410) 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8dc00640 1 -- 172.21.15.97:0/1957065210 learned_addr learned my addr 172.21.15.97:0/1957065210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed8d200640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed880cde10 0x7fed8814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.425+0000 7fed87e00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed8815a350 0x7fed880715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8dc00640 1 -- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed880cde10 msgr2=0x7fed8814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8dc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed880cde10 0x7fed8814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8dc00640 1 -- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed8815a350 msgr2=0x7fed880715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8dc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed8815a350 0x7fed880715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8dc00640 1 -- 172.21.15.97:0/1957065210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed700094d0 con 0x7fed8809bfa0 2024-01-31T05:01:17.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8dc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 0x7fed88071ae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed80009dd0 tx=0x7fed8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:17.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000ced0 con 0x7fed8809bfa0 2024-01-31T05:01:17.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8000f040 con 0x7fed8809bfa0 2024-01-31T05:01:17.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800146a0 con 0x7fed8809bfa0 2024-01-31T05:01:17.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8814ff60 con 0x7fed8809bfa0 2024-01-31T05:01:17.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.426+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed88159040 con 0x7fed8809bfa0 2024-01-31T05:01:17.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.428+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed88067830 con 0x7fed8809bfa0 2024-01-31T05:01:17.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.429+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fed80014800 con 0x7fed8809bfa0 2024-01-31T05:01:17.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.429+0000 7fed84c00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed6006aed0 0x7fed6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.429+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed8008a5a0 con 0x7fed8809bfa0 2024-01-31T05:01:17.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.430+0000 7fed8d200640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed6006aed0 0x7fed6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.430+0000 7fed8d200640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed6006aed0 0x7fed6006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed780016d0 tx=0x7fed78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:17.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.433+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fed80055050 con 0x7fed8809bfa0 2024-01-31T05:01:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.630+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed881597d0 con 0x7fed8809bfa0 2024-01-31T05:01:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.631+0000 7fed84c00640 1 -- 172.21.15.97:0/1957065210 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed80018020 con 0x7fed8809bfa0 2024-01-31T05:01:17.630 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed6006aed0 msgr2=0x7fed6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed6006aed0 0x7fed6006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fed780016d0 tx=0x7fed78009040 comp rx=0 tx=0).stop 2024-01-31T05:01:17.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 msgr2=0x7fed88071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 0x7fed88071ae0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed80009dd0 tx=0x7fed8000c490 comp rx=0 tx=0).stop 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 shutdown_connections 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fed6006aed0 0x7fed6006d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed8809bfa0 0x7fed88071ae0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed8815a350 0x7fed880715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 --2- 172.21.15.97:0/1957065210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed880cde10 0x7fed8814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 >> 172.21.15.97:0/1957065210 conn(0x7fed880b0a10 msgr2=0x7fed880ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 shutdown_connections 2024-01-31T05:01:17.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.634+0000 7fed8fc00640 1 -- 172.21.15.97:0/1957065210 wait complete. 2024-01-31T05:01:17.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:17.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.853+0000 7fb04be00640 1 Processor -- start 2024-01-31T05:01:17.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.853+0000 7fb04be00640 1 -- start start 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.855+0000 7fb04be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04415a730 0x7fb0440abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.855+0000 7fb04be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0440cde10 0x7fb0440ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.855+0000 7fb04be00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04415abc0 con 0x7fb04415a350 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.855+0000 7fb04be00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0440b2c90 con 0x7fb0440cde10 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.855+0000 7fb04be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440ce7c0 con 0x7fb04415a730 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0440cde10 0x7fb0440ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0440cde10 0x7fb0440ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34416/0 (socket says 172.21.15.97:34416) 2024-01-31T05:01:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 -- 172.21.15.97:0/2417919426 learned_addr learned my addr 172.21.15.97:0/2417919426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb048a00640 1 --2- 172.21.15.97:0/2417919426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04415a730 0x7fb0440abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 -- 172.21.15.97:0/2417919426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04415a730 msgr2=0x7fb0440abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 --2- 172.21.15.97:0/2417919426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04415a730 0x7fb0440abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 -- 172.21.15.97:0/2417919426 >> v1:172.21.15.188:6789/0 conn(0x7fb04415a350 legacy=0x7fb04406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.856+0000 7fb049e00640 1 -- 172.21.15.97:0/2417919426 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04415aea0 con 0x7fb0440cde10 2024-01-31T05:01:17.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.858+0000 7fb049e00640 1 --2- 172.21.15.97:0/2417919426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440ce280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009e80 tx=0x7fb03c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be33d89f02d8cb40 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:17.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.858+0000 7fb043e00640 1 -- 172.21.15.97:0/2417919426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c02ca50 con 0x7fb0440cde10 2024-01-31T05:01:17.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.858+0000 7fb043e00640 1 -- 172.21.15.97:0/2417919426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c02cbb0 con 0x7fb0440cde10 2024-01-31T05:01:17.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.858+0000 7fb043e00640 1 -- 172.21.15.97:0/2417919426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0343f0 con 0x7fb0440cde10 2024-01-31T05:01:17.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.858+0000 7fb04be00640 1 -- 172.21.15.97:0/2417919426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 msgr2=0x7fb0440ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.858+0000 7fb04be00640 1 --2- 172.21.15.97:0/2417919426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440ce280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009e80 tx=0x7fb03c02beb0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.859+0000 7fb04be00640 1 -- 172.21.15.97:0/2417919426 shutdown_connections 2024-01-31T05:01:17.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.859+0000 7fb04be00640 1 --2- 172.21.15.97:0/2417919426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440ce280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.859+0000 7fb04be00640 1 --2- 172.21.15.97:0/2417919426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04415a730 0x7fb0440abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.859+0000 7fb04be00640 1 -- 172.21.15.97:0/2417919426 >> 172.21.15.97:0/2417919426 conn(0x7fb0440b0a10 msgr2=0x7fb0440b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:17.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.859+0000 7fb04be00640 1 -- 172.21.15.97:0/2417919426 shutdown_connections 2024-01-31T05:01:17.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.859+0000 7fb04be00640 1 -- 172.21.15.97:0/2417919426 wait complete. 2024-01-31T05:01:17.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.861+0000 7fb04be00640 1 Processor -- start 2024-01-31T05:01:17.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.861+0000 7fb04be00640 1 -- start start 2024-01-31T05:01:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb04be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb04be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04415a350 0x7fb0440ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb04be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440cf290 0x7fb04413b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb04be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04413c2a0 con 0x7fb0440cde10 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb04be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440cf720 con 0x7fb04415a350 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb04be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440ca130 con 0x7fb0440cf290 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34428/0 (socket says 172.21.15.97:34428) 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 -- 172.21.15.97:0/1315765930 learned_addr learned my addr 172.21.15.97:0/1315765930 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 -- 172.21.15.97:0/1315765930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440cf290 msgr2=0x7fb04413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440cf290 0x7fb04413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 -- 172.21.15.97:0/1315765930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04415a350 msgr2=0x7fb0440ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04415a350 0x7fb0440ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.862+0000 7fb049400640 1 -- 172.21.15.97:0/1315765930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009b30 con 0x7fb0440cde10 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.863+0000 7fb049400640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440c8860 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c400 tx=0x7fb03400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.863+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340052d0 con 0x7fb0440cde10 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.863+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034005470 con 0x7fb0440cde10 2024-01-31T05:01:17.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.863+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440ca3d0 con 0x7fb0440cde10 2024-01-31T05:01:17.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.863+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034006a60 con 0x7fb0440cde10 2024-01-31T05:01:17.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.863+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0440ca990 con 0x7fb0440cde10 2024-01-31T05:01:17.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.864+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044067830 con 0x7fb0440cde10 2024-01-31T05:01:17.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.865+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb034006bc0 con 0x7fb0440cde10 2024-01-31T05:01:17.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.866+0000 7fb041600640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb01c06aed0 0x7fb01c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:17.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.866+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb034088d10 con 0x7fb0440cde10 2024-01-31T05:01:17.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.866+0000 7fb048a00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb01c06aed0 0x7fb01c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:17.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.867+0000 7fb048a00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb01c06aed0 0x7fb01c06d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb0441f6430 tx=0x7fb02c009500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:17.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:17.871+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb0340538f0 con 0x7fb0440cde10 2024-01-31T05:01:18.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.061+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb044156650 con 0x7fb0440cde10 2024-01-31T05:01:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.062+0000 7fb041600640 1 -- 172.21.15.97:0/1315765930 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb034056f10 con 0x7fb0440cde10 2024-01-31T05:01:18.062 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.064+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb01c06aed0 msgr2=0x7fb01c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.064+0000 7fb04be00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb01c06aed0 0x7fb01c06d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb0441f6430 tx=0x7fb02c009500 comp rx=0 tx=0).stop 2024-01-31T05:01:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.064+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 msgr2=0x7fb0440c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.064+0000 7fb04be00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440c8860 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c400 tx=0x7fb03400c8c0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 shutdown_connections 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440cf290 0x7fb04413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04415a350 0x7fb0440ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb01c06aed0 0x7fb01c06d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 --2- 172.21.15.97:0/1315765930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0440cde10 0x7fb0440c8860 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 >> 172.21.15.97:0/1315765930 conn(0x7fb0440b0a10 msgr2=0x7fb0440ac600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 shutdown_connections 2024-01-31T05:01:18.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.065+0000 7fb04be00640 1 -- 172.21.15.97:0/1315765930 wait complete. 2024-01-31T05:01:18.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.299+0000 7f8074c00640 1 Processor -- start 2024-01-31T05:01:18.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.299+0000 7f8074c00640 1 -- start start 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f8074c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d0040 0x7f80700b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f8074c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80700d39f0 0x7f80700b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f8074c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80700d2190 con 0x7f807015d450 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f8074c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80700b3060 con 0x7f80700d39f0 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f8074c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700b2a30 con 0x7f80700d0040 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f806fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80700d39f0 0x7f80700b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f806fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80700d39f0 0x7f80700b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34438/0 (socket says 172.21.15.97:34438) 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f806fe00640 1 -- 172.21.15.97:0/1113548540 learned_addr learned my addr 172.21.15.97:0/1113548540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.301+0000 7f8067e00640 1 --2- 172.21.15.97:0/1113548540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d0040 0x7f80700b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.302+0000 7f806fe00640 1 -- 172.21.15.97:0/1113548540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d0040 msgr2=0x7f80700b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.302+0000 7f806fe00640 1 --2- 172.21.15.97:0/1113548540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d0040 0x7f80700b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.302+0000 7f806fe00640 1 -- 172.21.15.97:0/1113548540 >> v1:172.21.15.188:6789/0 conn(0x7f807015d450 legacy=0x7f80700cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.302+0000 7f806fe00640 1 -- 172.21.15.97:0/1113548540 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700d1c30 con 0x7f80700d39f0 2024-01-31T05:01:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.303+0000 7f806fe00640 1 --2- 172.21.15.97:0/1113548540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d39f0 0x7f80700b39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8068009e50 tx=0x7f806802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ee4a3066dbbfdc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.303+0000 7f806ea00640 1 -- 172.21.15.97:0/1113548540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806802ca70 con 0x7f80700d39f0 2024-01-31T05:01:18.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.303+0000 7f806ea00640 1 -- 172.21.15.97:0/1113548540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806802cbd0 con 0x7f80700d39f0 2024-01-31T05:01:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.303+0000 7f806ea00640 1 -- 172.21.15.97:0/1113548540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680343f0 con 0x7f80700d39f0 2024-01-31T05:01:18.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 -- 172.21.15.97:0/1113548540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d39f0 msgr2=0x7f80700b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 --2- 172.21.15.97:0/1113548540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d39f0 0x7f80700b39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8068009e50 tx=0x7f806802bed0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 -- 172.21.15.97:0/1113548540 shutdown_connections 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 --2- 172.21.15.97:0/1113548540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d39f0 0x7f80700b39c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 --2- 172.21.15.97:0/1113548540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d0040 0x7f80700b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 -- 172.21.15.97:0/1113548540 >> 172.21.15.97:0/1113548540 conn(0x7f807009c400 msgr2=0x7f80700d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 -- 172.21.15.97:0/1113548540 shutdown_connections 2024-01-31T05:01:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.304+0000 7f8074c00640 1 -- 172.21.15.97:0/1113548540 wait complete. 2024-01-31T05:01:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.305+0000 7f8074c00640 1 Processor -- start 2024-01-31T05:01:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.306+0000 7f8074c00640 1 -- start start 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.306+0000 7f8074c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 0x7f807015c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.306+0000 7f8074c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d39f0 0x7f80700b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f8074c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807015d450 0x7f80700c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f8074c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80700b8d10 con 0x7f80700d0040 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f8074c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80700c6a60 con 0x7f807015d450 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f8074c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700c6d40 con 0x7f80700d39f0 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 0x7f807015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 0x7f807015c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34440/0 (socket says 172.21.15.97:34440) 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 -- 172.21.15.97:0/3368289957 learned_addr learned my addr 172.21.15.97:0/3368289957 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f8067e00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d39f0 0x7f80700b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806fe00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807015d450 0x7f80700c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 -- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d39f0 msgr2=0x7f80700b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d39f0 0x7f80700b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 -- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807015d450 msgr2=0x7f80700c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807015d450 0x7f80700c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f806f400640 1 -- 172.21.15.97:0/3368289957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8068009b30 con 0x7f80700d0040 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.307+0000 7f8067e00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d39f0 0x7f80700b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.308+0000 7f806f400640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 0x7f807015c100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f806000b3f0 tx=0x7f806000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.308+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060011830 con 0x7f80700d0040 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.308+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060011990 con 0x7f80700d0040 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.308+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060011d40 con 0x7f80700d0040 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.308+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80700c7020 con 0x7f80700d0040 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.308+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80700c75e0 con 0x7f80700d0040 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.309+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80700c7800 con 0x7f80700d0040 2024-01-31T05:01:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.310+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f806001a400 con 0x7f80700d0040 2024-01-31T05:01:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.311+0000 7f8067400640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f804806aed0 0x7f804806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.311+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8060089790 con 0x7f80700d0040 2024-01-31T05:01:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.311+0000 7f8067e00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f804806aed0 0x7f804806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.312+0000 7f8067e00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f804806aed0 0x7f804806d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80700c8610 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:18.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.315+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8060054360 con 0x7f80700d0040 2024-01-31T05:01:18.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.508+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80700676b0 con 0x7f80700d0040 2024-01-31T05:01:18.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.509+0000 7f8067400640 1 -- 172.21.15.97:0/3368289957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8060057980 con 0x7f80700d0040 2024-01-31T05:01:18.508 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f804806aed0 msgr2=0x7f804806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f804806aed0 0x7f804806d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80700c8610 tx=0x7f8058008040 comp rx=0 tx=0).stop 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 msgr2=0x7f807015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 0x7f807015c100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f806000b3f0 tx=0x7f806000b8b0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 shutdown_connections 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f807015d450 0x7f80700c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80700d39f0 0x7f80700b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f804806aed0 0x7f804806d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 --2- 172.21.15.97:0/3368289957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80700d0040 0x7f807015c100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 >> 172.21.15.97:0/3368289957 conn(0x7f807009c400 msgr2=0x7f80700cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.511+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 shutdown_connections 2024-01-31T05:01:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.512+0000 7f8074c00640 1 -- 172.21.15.97:0/3368289957 wait complete. 2024-01-31T05:01:18.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:18.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.742+0000 7fb625200640 1 Processor -- start 2024-01-31T05:01:18.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.742+0000 7fb625200640 1 -- start start 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.743+0000 7fb625200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb62015d590 0x7fb62015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb625200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb620161ea0 con 0x7fb6200cff20 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb625200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6200cded0 con 0x7fb6200cfae0 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb625200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62015d970 con 0x7fb62015d590 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61de00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb62015d590 0x7fb62015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61de00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb62015d590 0x7fb62015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34454/0 (socket says 172.21.15.97:34454) 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61de00640 1 -- 172.21.15.97:0/2915566256 learned_addr learned my addr 172.21.15.97:0/2915566256 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61e800640 1 --1- 172.21.15.97:0/2915566256 >> v1:172.21.15.97:6789/0 conn(0x7fb6200cff20 0x7fb620161790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33138/0 (socket says 172.21.15.97:33138) 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61de00640 1 -- 172.21.15.97:0/2915566256 >> v1:172.21.15.188:6789/0 conn(0x7fb6200cfae0 legacy=0x7fb620161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:18.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61de00640 1 -- 172.21.15.97:0/2915566256 >> v1:172.21.15.97:6789/0 conn(0x7fb6200cff20 legacy=0x7fb620161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.744+0000 7fb61de00640 1 -- 172.21.15.97:0/2915566256 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6200cf210 con 0x7fb62015d590 2024-01-31T05:01:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.745+0000 7fb61de00640 1 --2- 172.21.15.97:0/2915566256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb62015d590 0x7fb62015e540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb608009870 tx=0x7fb60802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4584451ec79ba2a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb61ca00640 1 -- 172.21.15.97:0/2915566256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802f850 con 0x7fb62015d590 2024-01-31T05:01:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb61ca00640 1 -- 172.21.15.97:0/2915566256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60802f9b0 con 0x7fb62015d590 2024-01-31T05:01:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb61ca00640 1 -- 172.21.15.97:0/2915566256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802fd40 con 0x7fb62015d590 2024-01-31T05:01:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb625200640 1 -- 172.21.15.97:0/2915566256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb62015d590 msgr2=0x7fb62015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb625200640 1 --2- 172.21.15.97:0/2915566256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb62015d590 0x7fb62015e540 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb608009870 tx=0x7fb60802ba30 comp rx=0 tx=0).stop 2024-01-31T05:01:18.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb625200640 1 -- 172.21.15.97:0/2915566256 shutdown_connections 2024-01-31T05:01:18.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb625200640 1 --2- 172.21.15.97:0/2915566256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb62015d590 0x7fb62015e540 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.746+0000 7fb625200640 1 -- 172.21.15.97:0/2915566256 >> 172.21.15.97:0/2915566256 conn(0x7fb62009c560 msgr2=0x7fb6200d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:18.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.747+0000 7fb625200640 1 -- 172.21.15.97:0/2915566256 shutdown_connections 2024-01-31T05:01:18.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.747+0000 7fb625200640 1 -- 172.21.15.97:0/2915566256 wait complete. 2024-01-31T05:01:18.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.748+0000 7fb625200640 1 Processor -- start 2024-01-31T05:01:18.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.748+0000 7fb625200640 1 -- start start 2024-01-31T05:01:18.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.749+0000 7fb625200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 0x7fb62014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.749+0000 7fb625200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6200cff20 0x7fb6200c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.749+0000 7fb625200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb62015d590 0x7fb6200d1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.749+0000 7fb625200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c9900 con 0x7fb6200cfae0 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.749+0000 7fb625200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200c9ea0 con 0x7fb62015d590 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.749+0000 7fb625200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6200b5fd0 con 0x7fb6200cff20 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 0x7fb62014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 0x7fb62014e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34470/0 (socket says 172.21.15.97:34470) 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 -- 172.21.15.97:0/4153637503 learned_addr learned my addr 172.21.15.97:0/4153637503 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61d400640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6200cff20 0x7fb6200c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61e800640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb62015d590 0x7fb6200d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 -- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6200cff20 msgr2=0x7fb6200c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6200cff20 0x7fb6200c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 -- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb62015d590 msgr2=0x7fb6200d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb62015d590 0x7fb6200d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 -- 172.21.15.97:0/4153637503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6080094d0 con 0x7fb6200cfae0 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61e800640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb62015d590 0x7fb6200d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:18.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61d400640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6200cff20 0x7fb6200c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.750+0000 7fb61de00640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 0x7fb62014e7f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb608000c00 tx=0x7fb608004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.751+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60802f800 con 0x7fb6200cfae0 2024-01-31T05:01:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.751+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608041400 con 0x7fb6200cfae0 2024-01-31T05:01:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.751+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6080417b0 con 0x7fb6200cfae0 2024-01-31T05:01:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.751+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6200b6270 con 0x7fb6200cfae0 2024-01-31T05:01:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.751+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6200b6830 con 0x7fb6200cfae0 2024-01-31T05:01:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.752+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb620067830 con 0x7fb6200cfae0 2024-01-31T05:01:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.753+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb608041910 con 0x7fb6200cfae0 2024-01-31T05:01:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.754+0000 7fb616000640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5f806aed0 0x7fb5f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.754+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb60803c070 con 0x7fb6200cfae0 2024-01-31T05:01:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.754+0000 7fb61d400640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5f806aed0 0x7fb5f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:18.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.755+0000 7fb61d400640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5f806aed0 0x7fb5f806d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6200b91e0 tx=0x7fb610009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.758+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb6080773e0 con 0x7fb6200cfae0 2024-01-31T05:01:18.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.952+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb6200d04e0 con 0x7fb6200cfae0 2024-01-31T05:01:18.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.953+0000 7fb616000640 1 -- 172.21.15.97:0/4153637503 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb6080aae60 con 0x7fb6200cfae0 2024-01-31T05:01:18.953 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5f806aed0 msgr2=0x7fb5f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5f806aed0 0x7fb5f806d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb6200b91e0 tx=0x7fb610009040 comp rx=0 tx=0).stop 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 msgr2=0x7fb62014e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 0x7fb62014e7f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb608000c00 tx=0x7fb608004730 comp rx=0 tx=0).stop 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 shutdown_connections 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb62015d590 0x7fb6200d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6200cff20 0x7fb6200c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5f806aed0 0x7fb5f806d390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 --2- 172.21.15.97:0/4153637503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6200cfae0 0x7fb62014e7f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 >> 172.21.15.97:0/4153637503 conn(0x7fb62009c560 msgr2=0x7fb62015b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 shutdown_connections 2024-01-31T05:01:18.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:18.956+0000 7fb625200640 1 -- 172.21.15.97:0/4153637503 wait complete. 2024-01-31T05:01:18.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:01:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.178+0000 7f6531a00640 1 Processor -- start 2024-01-31T05:01:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.178+0000 7f6531a00640 1 -- start start 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f6531a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f652c0cb7d0 0x7f652c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f6531a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f6531a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652c06e0d0 con 0x7f652c06dc40 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f6531a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652c06cba0 con 0x7f652c0cb7d0 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f6531a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c06ce80 con 0x7f652c06c350 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f652aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f652c0cb7d0 0x7f652c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f652aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f652c0cb7d0 0x7f652c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34474/0 (socket says 172.21.15.97:34474) 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f652aa00640 1 -- 172.21.15.97:0/2103988870 learned_addr learned my addr 172.21.15.97:0/2103988870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.180+0000 7f652b400640 1 --2- 172.21.15.97:0/2103988870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.181+0000 7f652aa00640 1 -- 172.21.15.97:0/2103988870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 msgr2=0x7f652c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.181+0000 7f652aa00640 1 --2- 172.21.15.97:0/2103988870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.181+0000 7f652aa00640 1 -- 172.21.15.97:0/2103988870 >> v1:172.21.15.188:6789/0 conn(0x7f652c06dc40 legacy=0x7f652c0cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.181+0000 7f652aa00640 1 -- 172.21.15.97:0/2103988870 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c06d160 con 0x7f652c0cb7d0 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.182+0000 7f652aa00640 1 --2- 172.21.15.97:0/2103988870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c0cb7d0 0x7f652c0b2e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f651c009870 tx=0x7f651c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97257081d9677229 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.182+0000 7f6529600640 1 -- 172.21.15.97:0/2103988870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c031640 con 0x7f652c0cb7d0 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.182+0000 7f6529600640 1 -- 172.21.15.97:0/2103988870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c0317a0 con 0x7f652c0cb7d0 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.182+0000 7f6529600640 1 -- 172.21.15.97:0/2103988870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c031b30 con 0x7f652c0cb7d0 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.182+0000 7f6531a00640 1 -- 172.21.15.97:0/2103988870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c0cb7d0 msgr2=0x7f652c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.182+0000 7f6531a00640 1 --2- 172.21.15.97:0/2103988870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c0cb7d0 0x7f652c0b2e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f651c009870 tx=0x7f651c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.183+0000 7f6531a00640 1 -- 172.21.15.97:0/2103988870 shutdown_connections 2024-01-31T05:01:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.183+0000 7f6531a00640 1 --2- 172.21.15.97:0/2103988870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.183+0000 7f6531a00640 1 --2- 172.21.15.97:0/2103988870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c0cb7d0 0x7f652c0b2e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.183+0000 7f6531a00640 1 -- 172.21.15.97:0/2103988870 >> 172.21.15.97:0/2103988870 conn(0x7f652c0b0a10 msgr2=0x7f652c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.183+0000 7f6531a00640 1 -- 172.21.15.97:0/2103988870 shutdown_connections 2024-01-31T05:01:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.183+0000 7f6531a00640 1 -- 172.21.15.97:0/2103988870 wait complete. 2024-01-31T05:01:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.185+0000 7f6531a00640 1 Processor -- start 2024-01-31T05:01:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.185+0000 7f6531a00640 1 -- start start 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f6531a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f6531a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 0x7f652c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f6531a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f652c0cb7d0 0x7f652c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f6531a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c15fc50 con 0x7f652c06dc40 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f6531a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652c09a920 con 0x7f652c0cb7d0 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f6531a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652c099ff0 con 0x7f652c06c350 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f652a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 0x7f652c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f652a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 0x7f652c15f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34490/0 (socket says 172.21.15.97:34490) 2024-01-31T05:01:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f652a000640 1 -- 172.21.15.97:0/929085971 learned_addr learned my addr 172.21.15.97:0/929085971 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f652aa00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f652b400640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f652c0cb7d0 0x7f652c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.186+0000 7f652a000640 1 -- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 msgr2=0x7f652c15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f652a000640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f652a000640 1 -- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f652c0cb7d0 msgr2=0x7f652c0af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f652a000640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f652c0cb7d0 0x7f652c0af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f652a000640 1 -- 172.21.15.97:0/929085971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0094d0 con 0x7f652c06dc40 2024-01-31T05:01:19.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f652a000640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 0x7f652c15f710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f651400c430 tx=0x7f651400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514011790 con 0x7f652c06dc40 2024-01-31T05:01:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65140118f0 con 0x7f652c06dc40 2024-01-31T05:01:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514011ce0 con 0x7f652c06dc40 2024-01-31T05:01:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652c09a2d0 con 0x7f652c06dc40 2024-01-31T05:01:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.187+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c09adb0 con 0x7f652c06dc40 2024-01-31T05:01:19.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.188+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652c15d7d0 con 0x7f652c06dc40 2024-01-31T05:01:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.189+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f651401c460 con 0x7f652c06dc40 2024-01-31T05:01:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.190+0000 7f6522a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f650406af00 0x7f650406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.190+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6514089850 con 0x7f652c06dc40 2024-01-31T05:01:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.190+0000 7f652aa00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f650406af00 0x7f650406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.191+0000 7f652aa00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f650406af00 0x7f650406d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f651c036000 tx=0x7f651c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:19.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.194+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6514054430 con 0x7f652c06dc40 2024-01-31T05:01:19.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.426+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f652c09aa90 con 0x7f652c06dc40 2024-01-31T05:01:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.427+0000 7f6522a00640 1 -- 172.21.15.97:0/929085971 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f6514057a50 con 0x7f652c06dc40 2024-01-31T05:01:19.427 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:19.427 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:01:19.428 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:01:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f650406af00 msgr2=0x7f650406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f650406af00 0x7f650406d3c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f651c036000 tx=0x7f651c03c040 comp rx=0 tx=0).stop 2024-01-31T05:01:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 msgr2=0x7f652c15f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 0x7f652c15f710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f651400c430 tx=0x7f651400c8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 shutdown_connections 2024-01-31T05:01:19.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f652c0cb7d0 0x7f652c0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f650406af00 0x7f650406d3c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f652c06dc40 0x7f652c15f710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 --2- 172.21.15.97:0/929085971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f652c06c350 0x7f652c15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.430+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 >> 172.21.15.97:0/929085971 conn(0x7f652c0b0a10 msgr2=0x7f652c071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.431+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 shutdown_connections 2024-01-31T05:01:19.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.431+0000 7f6531a00640 1 -- 172.21.15.97:0/929085971 wait complete. 2024-01-31T05:01:19.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:19.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.653+0000 7fd876800640 1 Processor -- start 2024-01-31T05:01:19.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.653+0000 7fd876800640 1 -- start start 2024-01-31T05:01:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.654+0000 7fd876800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd87015ec70 0x7fd870071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.654+0000 7fd876800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd870070400 con 0x7fd8700ae3b0 2024-01-31T05:01:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.654+0000 7fd876800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8700ca580 con 0x7fd8700723a0 2024-01-31T05:01:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.654+0000 7fd876800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8700ad8c0 con 0x7fd87015ec70 2024-01-31T05:01:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd875400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd87015ec70 0x7fd870071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd874a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8700ae3b0 0x7fd87006cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33140/0 (socket says 172.21.15.97:33140) 2024-01-31T05:01:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd874a00640 1 -- 172.21.15.97:0/3061885370 learned_addr learned my addr 172.21.15.97:0/3061885370 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd875400640 1 -- 172.21.15.97:0/3061885370 >> v1:172.21.15.188:6789/0 conn(0x7fd8700723a0 legacy=0x7fd87015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd875400640 1 -- 172.21.15.97:0/3061885370 >> v1:172.21.15.97:6789/0 conn(0x7fd8700ae3b0 legacy=0x7fd87006cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd86fe00640 1 -- 172.21.15.97:0/3061885370 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60338517 0 0) 0x7fd870070400 con 0x7fd8700ae3b0 2024-01-31T05:01:19.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.655+0000 7fd875400640 1 -- 172.21.15.97:0/3061885370 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870070650 con 0x7fd87015ec70 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.656+0000 7fd875400640 1 --2- 172.21.15.97:0/3061885370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87015ec70 0x7fd870071e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd8600097a0 tx=0x7fd86002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c2c7c20a865e1a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd86fe00640 1 -- 172.21.15.97:0/3061885370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86002f050 con 0x7fd87015ec70 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd86fe00640 1 -- 172.21.15.97:0/3061885370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd860031700 con 0x7fd87015ec70 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd86fe00640 1 -- 172.21.15.97:0/3061885370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860031a90 con 0x7fd87015ec70 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 -- 172.21.15.97:0/3061885370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87015ec70 msgr2=0x7fd870071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 --2- 172.21.15.97:0/3061885370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87015ec70 0x7fd870071e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd8600097a0 tx=0x7fd86002b7a0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 -- 172.21.15.97:0/3061885370 shutdown_connections 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 --2- 172.21.15.97:0/3061885370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd87015ec70 0x7fd870071e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 -- 172.21.15.97:0/3061885370 >> 172.21.15.97:0/3061885370 conn(0x7fd8700cec40 msgr2=0x7fd8700cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 -- 172.21.15.97:0/3061885370 shutdown_connections 2024-01-31T05:01:19.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.657+0000 7fd876800640 1 -- 172.21.15.97:0/3061885370 wait complete. 2024-01-31T05:01:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.659+0000 7fd876800640 1 Processor -- start 2024-01-31T05:01:19.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.659+0000 7fd876800640 1 -- start start 2024-01-31T05:01:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd876800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 0x7fd8701485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd876800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700ae3b0 0x7fd8700999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd876800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd87015ec70 0x7fd870099f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd876800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd870157cb0 con 0x7fd8700723a0 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd876800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd870157f90 con 0x7fd8700ae3b0 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd876800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd870158270 con 0x7fd87015ec70 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd875400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 0x7fd8701485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd875400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 0x7fd8701485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34512/0 (socket says 172.21.15.97:34512) 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.660+0000 7fd875400640 1 -- 172.21.15.97:0/3222701310 learned_addr learned my addr 172.21.15.97:0/3222701310 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd874a00640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700ae3b0 0x7fd8700999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875400640 1 -- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd87015ec70 msgr2=0x7fd870099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875e00640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd87015ec70 0x7fd870099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875400640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd87015ec70 0x7fd870099f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875400640 1 -- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700ae3b0 msgr2=0x7fd8700999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875400640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700ae3b0 0x7fd8700999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875400640 1 -- 172.21.15.97:0/3222701310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860009400 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd874a00640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700ae3b0 0x7fd8700999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd875400640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 0x7fd8701485f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd8600097a0 tx=0x7fd8600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860033c30 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd860033d90 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86003d6b0 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.661+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd870158550 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.662+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd870148b30 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.663+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8700676b0 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.663+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd86003a040 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.664+0000 7fd86d600640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd84806af00 0x7fd84806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.664+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8600ab920 con 0x7fd8700723a0 2024-01-31T05:01:19.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.665+0000 7fd874a00640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd84806af00 0x7fd84806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:19.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.665+0000 7fd874a00640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd84806af00 0x7fd84806d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8700b4eb0 tx=0x7fd858009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:19.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.669+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd860076460 con 0x7fd8700723a0 2024-01-31T05:01:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.862+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8700cfa70 con 0x7fd8700723a0 2024-01-31T05:01:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.865+0000 7fd86d600640 1 -- 172.21.15.97:0/3222701310 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fd860079a80 con 0x7fd8700723a0 2024-01-31T05:01:19.869 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:19.869 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd84806af00 msgr2=0x7fd84806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd84806af00 0x7fd84806d3c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8700b4eb0 tx=0x7fd858009040 comp rx=0 tx=0).stop 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 msgr2=0x7fd8701485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 0x7fd8701485f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd8600097a0 tx=0x7fd8600046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 shutdown_connections 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd87015ec70 0x7fd870099f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8700ae3b0 0x7fd8700999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd84806af00 0x7fd84806d3c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 --2- 172.21.15.97:0/3222701310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8700723a0 0x7fd8701485f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 >> 172.21.15.97:0/3222701310 conn(0x7fd8700cec40 msgr2=0x7fd87015aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 shutdown_connections 2024-01-31T05:01:19.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:19.868+0000 7fd876800640 1 -- 172.21.15.97:0/3222701310 wait complete. 2024-01-31T05:01:19.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:19.887 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:20.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.103+0000 7fdef0200640 1 Processor -- start 2024-01-31T05:01:20.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.104+0000 7fdef0200640 1 -- start start 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdef0200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdee815d590 0x7fdee815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdef0200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdee80cff20 0x7fdee8161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdef0200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80cded0 con 0x7fdee80cfae0 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdef0200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee815d970 con 0x7fdee80cff20 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdef0200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee80cf210 con 0x7fdee815d590 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdeece00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdee80cfae0 0x7fdee8161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33144/0 (socket says 172.21.15.97:33144) 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.105+0000 7fdeece00640 1 -- 172.21.15.97:0/3771032002 learned_addr learned my addr 172.21.15.97:0/3771032002 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdeee200640 1 --2- 172.21.15.97:0/3771032002 >> v2:172.21.15.97:3300/0 conn(0x7fdee80cff20 0x7fdee8161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdeed800640 1 --2- 172.21.15.97:0/3771032002 >> v2:172.21.15.188:3300/0 conn(0x7fdee815d590 0x7fdee815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdee7e00640 1 -- 172.21.15.97:0/3771032002 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309520475 0 0) 0x7fdee80cded0 con 0x7fdee80cfae0 2024-01-31T05:01:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdee7e00640 1 -- 172.21.15.97:0/3771032002 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec8003520 con 0x7fdee80cfae0 2024-01-31T05:01:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdeee200640 1 -- 172.21.15.97:0/3771032002 >> v2:172.21.15.188:3300/0 conn(0x7fdee815d590 msgr2=0x7fdee815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdeee200640 1 --2- 172.21.15.97:0/3771032002 >> v2:172.21.15.188:3300/0 conn(0x7fdee815d590 0x7fdee815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdeee200640 1 -- 172.21.15.97:0/3771032002 >> v1:172.21.15.97:6789/0 conn(0x7fdee80cfae0 legacy=0x7fdee8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.106+0000 7fdeee200640 1 -- 172.21.15.97:0/3771032002 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80ce170 con 0x7fdee80cff20 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.107+0000 7fdeee200640 1 --2- 172.21.15.97:0/3771032002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee80cff20 0x7fdee8161790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdee808b830 tx=0x7fdee002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ee5b82dbbcf090 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.107+0000 7fdee7e00640 1 -- 172.21.15.97:0/3771032002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0030610 con 0x7fdee80cff20 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdee7e00640 1 -- 172.21.15.97:0/3771032002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0030770 con 0x7fdee80cff20 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdee7e00640 1 -- 172.21.15.97:0/3771032002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0030b00 con 0x7fdee80cff20 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 -- 172.21.15.97:0/3771032002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee80cff20 msgr2=0x7fdee8161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 --2- 172.21.15.97:0/3771032002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee80cff20 0x7fdee8161790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdee808b830 tx=0x7fdee002b960 comp rx=0 tx=0).stop 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 -- 172.21.15.97:0/3771032002 shutdown_connections 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 --2- 172.21.15.97:0/3771032002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee80cff20 0x7fdee8161790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 --2- 172.21.15.97:0/3771032002 >> v2:172.21.15.188:3300/0 conn(0x7fdee815d590 0x7fdee815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 -- 172.21.15.97:0/3771032002 >> 172.21.15.97:0/3771032002 conn(0x7fdee809c560 msgr2=0x7fdee80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 -- 172.21.15.97:0/3771032002 shutdown_connections 2024-01-31T05:01:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.108+0000 7fdef0200640 1 -- 172.21.15.97:0/3771032002 wait complete. 2024-01-31T05:01:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.110+0000 7fdef0200640 1 Processor -- start 2024-01-31T05:01:20.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.110+0000 7fdef0200640 1 -- start start 2024-01-31T05:01:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdef0200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdee80cfae0 0x7fdee814e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdef0200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdee80cff20 0x7fdee80b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdef0200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 0x7fdee80b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdef0200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80cbdb0 con 0x7fdee815d590 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdef0200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80cb130 con 0x7fdee80cff20 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdef0200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80cb410 con 0x7fdee80cfae0 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.111+0000 7fdeee200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 0x7fdee80b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 0x7fdee80b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34520/0 (socket says 172.21.15.97:34520) 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 -- 172.21.15.97:0/1266486925 learned_addr learned my addr 172.21.15.97:0/1266486925 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeed800640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdee80cfae0 0x7fdee814e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeece00640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdee80cff20 0x7fdee80b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 -- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdee80cfae0 msgr2=0x7fdee814e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdee80cfae0 0x7fdee814e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 -- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdee80cff20 msgr2=0x7fdee80b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdee80cff20 0x7fdee80b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 -- 172.21.15.97:0/1266486925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0009400 con 0x7fdee815d590 2024-01-31T05:01:20.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.112+0000 7fdeee200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 0x7fdee80b8470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009890 tx=0x7fdee0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:20.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.113+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee003e040 con 0x7fdee815d590 2024-01-31T05:01:20.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.113+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0039900 con 0x7fdee815d590 2024-01-31T05:01:20.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.113+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0039cb0 con 0x7fdee815d590 2024-01-31T05:01:20.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.113+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80cb6f0 con 0x7fdee815d590 2024-01-31T05:01:20.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.113+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee814eb30 con 0x7fdee815d590 2024-01-31T05:01:20.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.114+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee80cbef0 con 0x7fdee815d590 2024-01-31T05:01:20.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.115+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fdee004d3b0 con 0x7fdee815d590 2024-01-31T05:01:20.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.116+0000 7fdee5600640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdec006aed0 0x7fdec006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.116+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdee00abe40 con 0x7fdee815d590 2024-01-31T05:01:20.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.116+0000 7fdeed800640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdec006aed0 0x7fdec006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.117+0000 7fdeed800640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdec006aed0 0x7fdec006d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fded0009a30 tx=0x7fded0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.120+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fdee80cbef0 con 0x7fdee815d590 2024-01-31T05:01:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.313+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdee80cbef0 con 0x7fdee815d590 2024-01-31T05:01:20.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.314+0000 7fdee5600640 1 -- 172.21.15.97:0/1266486925 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdee80cbef0 con 0x7fdee815d590 2024-01-31T05:01:20.313 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdec006aed0 msgr2=0x7fdec006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdec006aed0 0x7fdec006d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fded0009a30 tx=0x7fded0008040 comp rx=0 tx=0).stop 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 msgr2=0x7fdee80b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 0x7fdee80b8470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009890 tx=0x7fdee0004730 comp rx=0 tx=0).stop 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 shutdown_connections 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdec006aed0 0x7fdec006d390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdee815d590 0x7fdee80b8470 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdee80cff20 0x7fdee80b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 --2- 172.21.15.97:0/1266486925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdee80cfae0 0x7fdee814e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.316+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 >> 172.21.15.97:0/1266486925 conn(0x7fdee809c560 msgr2=0x7fdee806cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.317+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 shutdown_connections 2024-01-31T05:01:20.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.317+0000 7fdef0200640 1 -- 172.21.15.97:0/1266486925 wait complete. 2024-01-31T05:01:20.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:20.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.547+0000 7f5b23600640 1 Processor -- start 2024-01-31T05:01:20.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.547+0000 7f5b23600640 1 -- start start 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b23600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b23600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b1c15d690 0x7f5b1c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b23600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5b1c06c350 0x7f5b1c06c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b23600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c15e7c0 con 0x7f5b1c15d690 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b23600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c06dea0 con 0x7f5b1c06c350 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b23600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c15db40 con 0x7f5b1c06d580 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b1be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b1c15d690 0x7f5b1c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b1be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b1c15d690 0x7f5b1c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34522/0 (socket says 172.21.15.97:34522) 2024-01-31T05:01:20.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.549+0000 7f5b1be00640 1 -- 172.21.15.97:0/386573631 learned_addr learned my addr 172.21.15.97:0/386573631 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:20.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b20c00640 1 --2- 172.21.15.97:0/386573631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b21600640 1 --2- 172.21.15.97:0/386573631 >> v2:172.21.15.188:3300/0 conn(0x7f5b1c06c350 0x7f5b1c06c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b1be00640 1 -- 172.21.15.97:0/386573631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 msgr2=0x7f5b1c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:20.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b1be00640 1 --2- 172.21.15.97:0/386573631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c06d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b1be00640 1 -- 172.21.15.97:0/386573631 >> v2:172.21.15.188:3300/0 conn(0x7f5b1c06c350 msgr2=0x7f5b1c06c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:20.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b1be00640 1 --2- 172.21.15.97:0/386573631 >> v2:172.21.15.188:3300/0 conn(0x7f5b1c06c350 0x7f5b1c06c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b1be00640 1 -- 172.21.15.97:0/386573631 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c15de20 con 0x7f5b1c15d690 2024-01-31T05:01:20.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.550+0000 7f5b21600640 1 --2- 172.21.15.97:0/386573631 >> v2:172.21.15.188:3300/0 conn(0x7f5b1c06c350 0x7f5b1c06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:20.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.551+0000 7f5b1be00640 1 --2- 172.21.15.97:0/386573631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c15d690 0x7f5b1c0ce430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0cc690 tx=0x7f5b0402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42ef4a6dcc79769 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.551+0000 7f5b1b400640 1 -- 172.21.15.97:0/386573631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04031860 con 0x7f5b1c15d690 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b1b400640 1 -- 172.21.15.97:0/386573631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040319c0 con 0x7f5b1c15d690 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b1b400640 1 -- 172.21.15.97:0/386573631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04031d50 con 0x7f5b1c15d690 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 -- 172.21.15.97:0/386573631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c15d690 msgr2=0x7f5b1c0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 --2- 172.21.15.97:0/386573631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c15d690 0x7f5b1c0ce430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0cc690 tx=0x7f5b0402b900 comp rx=0 tx=0).stop 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 -- 172.21.15.97:0/386573631 shutdown_connections 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 --2- 172.21.15.97:0/386573631 >> v2:172.21.15.188:3300/0 conn(0x7f5b1c06c350 0x7f5b1c06c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 --2- 172.21.15.97:0/386573631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c15d690 0x7f5b1c0ce430 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 --2- 172.21.15.97:0/386573631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c06d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 -- 172.21.15.97:0/386573631 >> 172.21.15.97:0/386573631 conn(0x7f5b1c0b0a10 msgr2=0x7f5b1c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:20.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 -- 172.21.15.97:0/386573631 shutdown_connections 2024-01-31T05:01:20.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.552+0000 7f5b23600640 1 -- 172.21.15.97:0/386573631 wait complete. 2024-01-31T05:01:20.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.554+0000 7f5b23600640 1 Processor -- start 2024-01-31T05:01:20.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.554+0000 7f5b23600640 1 -- start start 2024-01-31T05:01:20.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b23600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 0x7f5b1c1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b23600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c09be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b23600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c15d690 0x7f5b1c09c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b23600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c151ad0 con 0x7f5b1c06c350 2024-01-31T05:01:20.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b23600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c14ffa0 con 0x7f5b1c15d690 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b23600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c09cd80 con 0x7f5b1c06d580 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b20c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 0x7f5b1c1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b20c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 0x7f5b1c1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34536/0 (socket says 172.21.15.97:34536) 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.555+0000 7f5b20c00640 1 -- 172.21.15.97:0/484971118 learned_addr learned my addr 172.21.15.97:0/484971118 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b1be00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c09be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b21600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c15d690 0x7f5b1c09c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b20c00640 1 -- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 msgr2=0x7f5b1c09be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b20c00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c09be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b20c00640 1 -- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c15d690 msgr2=0x7f5b1c09c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b20c00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c15d690 0x7f5b1c09c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b20c00640 1 -- 172.21.15.97:0/484971118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040094d0 con 0x7f5b1c06c350 2024-01-31T05:01:20.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b20c00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 0x7f5b1c1565a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e900 tx=0x7f5b0c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c009660 con 0x7f5b1c06c350 2024-01-31T05:01:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00ca00 con 0x7f5b1c06c350 2024-01-31T05:01:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c09caa0 con 0x7f5b1c06c350 2024-01-31T05:01:20.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.556+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00cd70 con 0x7f5b1c06c350 2024-01-31T05:01:20.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.557+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c151760 con 0x7f5b1c06c350 2024-01-31T05:01:20.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.558+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c067830 con 0x7f5b1c06c350 2024-01-31T05:01:20.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.559+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5b0c013470 con 0x7f5b1c06c350 2024-01-31T05:01:20.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.560+0000 7f5b18c00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5af406aed0 0x7f5af406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.560+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5b0c08a080 con 0x7f5b1c06c350 2024-01-31T05:01:20.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.560+0000 7f5b1be00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5af406aed0 0x7f5af406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:20.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.561+0000 7f5b1be00640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5af406aed0 0x7f5af406d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b04036000 tx=0x7f5b0403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:20.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.564+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5b0c0bd190 con 0x7f5b1c06c350 2024-01-31T05:01:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.759+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b1c0cc6c0 con 0x7f5b1c06c350 2024-01-31T05:01:20.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.760+0000 7f5b18c00640 1 -- 172.21.15.97:0/484971118 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b0c021020 con 0x7f5b1c06c350 2024-01-31T05:01:20.760 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5af406aed0 msgr2=0x7f5af406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5af406aed0 0x7f5af406d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b04036000 tx=0x7f5b0403c040 comp rx=0 tx=0).stop 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 msgr2=0x7f5b1c1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 0x7f5b1c1565a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e900 tx=0x7f5b0c00edc0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 shutdown_connections 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b1c15d690 0x7f5b1c09c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b1c06d580 0x7f5b1c09be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5af406aed0 0x7f5af406d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 --2- 172.21.15.97:0/484971118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b1c06c350 0x7f5b1c1565a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 >> 172.21.15.97:0/484971118 conn(0x7f5b1c0b0a10 msgr2=0x7f5b1c157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 shutdown_connections 2024-01-31T05:01:20.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.763+0000 7f5b23600640 1 -- 172.21.15.97:0/484971118 wait complete. 2024-01-31T05:01:20.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:20.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.996+0000 7fd83ec00640 1 Processor -- start 2024-01-31T05:01:20.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.997+0000 7fd83ec00640 1 -- start start 2024-01-31T05:01:20.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83ec00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd83815d590 0x7fd83815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83ec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd8380cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83ec00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8380d0460 con 0x7fd838161080 2024-01-31T05:01:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83ec00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380cded0 con 0x7fd83815d590 2024-01-31T05:01:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83ec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83815d970 con 0x7fd8380cfae0 2024-01-31T05:01:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83cc00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd838161080 0x7fd838161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33150/0 (socket says 172.21.15.97:33150) 2024-01-31T05:01:20.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.998+0000 7fd83cc00640 1 -- 172.21.15.97:0/2769658201 learned_addr learned my addr 172.21.15.97:0/2769658201 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd837e00640 1 --2- 172.21.15.97:0/2769658201 >> v2:172.21.15.188:3300/0 conn(0x7fd83815d590 0x7fd83815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd837400640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd8380cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396093208 0 0) 0x7fd8380d0460 con 0x7fd838161080 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd818003520 con 0x7fd838161080 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034070083 0 0) 0x7fd818003520 con 0x7fd838161080 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 msgr2=0x7fd8380cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd8380cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 >> v2:172.21.15.188:3300/0 conn(0x7fd83815d590 msgr2=0x7fd83815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 --2- 172.21.15.97:0/2769658201 >> v2:172.21.15.188:3300/0 conn(0x7fd83815d590 0x7fd83815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380cf210 con 0x7fd838161080 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fd828001670 con 0x7fd838161080 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd837400640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd8380cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 >> v1:172.21.15.97:6789/0 conn(0x7fd838161080 legacy=0x7fd838161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd818003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd818003eb0 0x7fd8180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd818004d00 0x7fd818005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cded0 con 0x7fd818003eb0 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83815d970 con 0x7fd818004d00 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:20.999+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81800ba30 con 0x7fd8380cfae0 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837400640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd818003eb0 0x7fd8180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837e00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd818003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd83cc00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd818004d00 0x7fd818005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837400640 1 -- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 msgr2=0x7fd818003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837400640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8380cfae0 0x7fd818003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837400640 1 -- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd818004d00 msgr2=0x7fd818005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837400640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd818004d00 0x7fd818005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.000+0000 7fd837400640 1 -- 172.21.15.97:0/2769658201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818003520 con 0x7fd818003eb0 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.001+0000 7fd837400640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd818003eb0 0x7fd8180047c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd8180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95130b67d68331f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.001+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820004230 con 0x7fd818003eb0 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.001+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820004390 con 0x7fd818003eb0 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.001+0000 7fd836a00640 1 -- 172.21.15.97:0/2769658201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820003830 con 0x7fd818003eb0 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83ec00640 1 -- 172.21.15.97:0/2769658201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd818003eb0 msgr2=0x7fd8180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83ec00640 1 --2- 172.21.15.97:0/2769658201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd818003eb0 0x7fd8180047c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd8180057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83cc00640 1 -- 172.21.15.97:0/2769658201 reap_dead start 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83ec00640 1 -- 172.21.15.97:0/2769658201 shutdown_connections 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83ec00640 1 -- 172.21.15.97:0/2769658201 >> 172.21.15.97:0/2769658201 conn(0x7fd83809c560 msgr2=0x7fd8380d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83ec00640 1 -- 172.21.15.97:0/2769658201 shutdown_connections 2024-01-31T05:01:21.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.002+0000 7fd83ec00640 1 -- 172.21.15.97:0/2769658201 wait complete. 2024-01-31T05:01:21.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.004+0000 7fd83ec00640 1 Processor -- start 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.004+0000 7fd83ec00640 1 -- start start 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd83ec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd818003eb0 0x7fd8381f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd83ec00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 0x7fd8380c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd83ec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83809dbe0 0x7fd83809e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd83ec00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380c9500 con 0x7fd8380c8b40 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd83ec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cba70 con 0x7fd83809dbe0 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd83ec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83809e5a0 con 0x7fd818003eb0 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd837400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 0x7fd8380c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd837400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 0x7fd8380c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34546/0 (socket says 172.21.15.97:34546) 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd837400640 1 -- 172.21.15.97:0/1676471510 learned_addr learned my addr 172.21.15.97:0/1676471510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:21.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.005+0000 7fd837e00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd818003eb0 0x7fd8381f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd83cc00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83809dbe0 0x7fd83809e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd837400640 1 -- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd818003eb0 msgr2=0x7fd8381f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd837400640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd818003eb0 0x7fd8381f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd837400640 1 -- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83809dbe0 msgr2=0x7fd83809e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd837400640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83809dbe0 0x7fd83809e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd837400640 1 -- 172.21.15.97:0/1676471510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8381f68a0 con 0x7fd8380c8b40 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd837400640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 0x7fd8380c8fc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8200097f0 tx=0x7fd820004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820030630 con 0x7fd8380c8b40 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820030790 con 0x7fd8380c8b40 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8381f6b80 con 0x7fd8380c8b40 2024-01-31T05:01:21.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.006+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820030b40 con 0x7fd8380c8b40 2024-01-31T05:01:21.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.007+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8381f7140 con 0x7fd8380c8b40 2024-01-31T05:01:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.008+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838067830 con 0x7fd8380c8b40 2024-01-31T05:01:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.008+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd820030ca0 con 0x7fd8380c8b40 2024-01-31T05:01:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.009+0000 7fd82fe00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd81006aed0 0x7fd81006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.009+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd8200ab500 con 0x7fd8380c8b40 2024-01-31T05:01:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.010+0000 7fd837e00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd81006aed0 0x7fd81006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.010+0000 7fd837e00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd81006aed0 0x7fd81006d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd830007520 tx=0x7fd83000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.014+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd8200760d0 con 0x7fd8380c8b40 2024-01-31T05:01:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.204+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd83815f470 con 0x7fd8380c8b40 2024-01-31T05:01:21.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.205+0000 7fd82fe00640 1 -- 172.21.15.97:0/1676471510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd82000f040 con 0x7fd8380c8b40 2024-01-31T05:01:21.205 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd81006aed0 msgr2=0x7fd81006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd81006aed0 0x7fd81006d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd830007520 tx=0x7fd83000b040 comp rx=0 tx=0).stop 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 msgr2=0x7fd8380c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 0x7fd8380c8fc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd8200097f0 tx=0x7fd820004730 comp rx=0 tx=0).stop 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 shutdown_connections 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83809dbe0 0x7fd83809e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd81006aed0 0x7fd81006d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8380c8b40 0x7fd8380c8fc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 --2- 172.21.15.97:0/1676471510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd818003eb0 0x7fd8381f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 >> 172.21.15.97:0/1676471510 conn(0x7fd83809c560 msgr2=0x7fd83815de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 shutdown_connections 2024-01-31T05:01:21.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.208+0000 7fd83ec00640 1 -- 172.21.15.97:0/1676471510 wait complete. 2024-01-31T05:01:21.228 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:21.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.441+0000 7f3311600640 1 Processor -- start 2024-01-31T05:01:21.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.441+0000 7f3311600640 1 -- start start 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3311600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c15b470 0x7f330c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3311600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f330c0abcf0 0x7f330c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3311600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c06cd20 con 0x7f330c0cb810 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3311600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c0ce970 con 0x7f330c0abcf0 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3311600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c15b850 con 0x7f330c15b470 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3303e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f330c0abcf0 0x7f330c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3303e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f330c0abcf0 0x7f330c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34560/0 (socket says 172.21.15.97:34560) 2024-01-31T05:01:21.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f3303e00640 1 -- 172.21.15.97:0/3930995551 learned_addr learned my addr 172.21.15.97:0/3930995551 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.443+0000 7f330aa00640 1 --2- 172.21.15.97:0/3930995551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c15b470 0x7f330c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.444+0000 7f3303e00640 1 -- 172.21.15.97:0/3930995551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c15b470 msgr2=0x7f330c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.444+0000 7f3303e00640 1 --2- 172.21.15.97:0/3930995551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c15b470 0x7f330c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.444+0000 7f3303e00640 1 -- 172.21.15.97:0/3930995551 >> v1:172.21.15.188:6789/0 conn(0x7f330c0cb810 legacy=0x7f330c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.444+0000 7f3303e00640 1 -- 172.21.15.97:0/3930995551 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0ceb60 con 0x7f330c0abcf0 2024-01-31T05:01:21.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.445+0000 7f3303e00640 1 --2- 172.21.15.97:0/3930995551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0abcf0 0x7f330c0ce430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f32f40098a0 tx=0x7f32f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62702916d9741138 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.445+0000 7f330a000640 1 -- 172.21.15.97:0/3930995551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4031640 con 0x7f330c0abcf0 2024-01-31T05:01:21.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.445+0000 7f330a000640 1 -- 172.21.15.97:0/3930995551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f40317a0 con 0x7f330c0abcf0 2024-01-31T05:01:21.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.445+0000 7f330a000640 1 -- 172.21.15.97:0/3930995551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4031b30 con 0x7f330c0abcf0 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 -- 172.21.15.97:0/3930995551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0abcf0 msgr2=0x7f330c0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 --2- 172.21.15.97:0/3930995551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0abcf0 0x7f330c0ce430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f32f40098a0 tx=0x7f32f402b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 -- 172.21.15.97:0/3930995551 shutdown_connections 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 --2- 172.21.15.97:0/3930995551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c0abcf0 0x7f330c0ce430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 --2- 172.21.15.97:0/3930995551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c15b470 0x7f330c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 -- 172.21.15.97:0/3930995551 >> 172.21.15.97:0/3930995551 conn(0x7f330c0b0a10 msgr2=0x7f330c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 -- 172.21.15.97:0/3930995551 shutdown_connections 2024-01-31T05:01:21.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.446+0000 7f3311600640 1 -- 172.21.15.97:0/3930995551 wait complete. 2024-01-31T05:01:21.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.448+0000 7f3311600640 1 Processor -- start 2024-01-31T05:01:21.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.448+0000 7f3311600640 1 -- start start 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3311600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c0abcf0 0x7f330c140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3311600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f330c0cb810 0x7f330c09c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3311600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 0x7f330c144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3311600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c1536a0 con 0x7f330c15b470 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3311600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c140f90 con 0x7f330c0cb810 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3311600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c152a90 con 0x7f330c0abcf0 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f330b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 0x7f330c144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f330b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 0x7f330c144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34568/0 (socket says 172.21.15.97:34568) 2024-01-31T05:01:21.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f330b400640 1 -- 172.21.15.97:0/3278171034 learned_addr learned my addr 172.21.15.97:0/3278171034 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f3303e00640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f330c0cb810 0x7f330c09c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.449+0000 7f330aa00640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c0abcf0 0x7f330c140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f330b400640 1 -- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c0abcf0 msgr2=0x7f330c140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f330b400640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c0abcf0 0x7f330c140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f330b400640 1 -- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f330c0cb810 msgr2=0x7f330c09c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f330b400640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f330c0cb810 0x7f330c09c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f330b400640 1 -- 172.21.15.97:0/3278171034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f40094d0 con 0x7f330c15b470 2024-01-31T05:01:21.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f3303e00640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f330c0cb810 0x7f330c09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f330b400640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 0x7f330c144700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f330400c780 tx=0x7f330400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304004b60 con 0x7f330c15b470 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330400ce90 con 0x7f330c15b470 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330400a850 con 0x7f330c15b470 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.450+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c152d70 con 0x7f330c15b470 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.451+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c153330 con 0x7f330c15b470 2024-01-31T05:01:21.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.452+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c06e350 con 0x7f330c15b470 2024-01-31T05:01:21.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.453+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f330400a9b0 con 0x7f330c15b470 2024-01-31T05:01:21.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.453+0000 7f3302a00640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f32e406aed0 0x7f32e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.454+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3304089ac0 con 0x7f330c15b470 2024-01-31T05:01:21.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.454+0000 7f330aa00640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f32e406aed0 0x7f32e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.454+0000 7f330aa00640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f32e406aed0 0x7f32e406d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b9d0 tx=0x7f32fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.458+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f33040546a0 con 0x7f330c15b470 2024-01-31T05:01:21.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.653+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f330c155960 con 0x7f330c15b470 2024-01-31T05:01:21.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.654+0000 7f3302a00640 1 -- 172.21.15.97:0/3278171034 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3304089e60 con 0x7f330c15b470 2024-01-31T05:01:21.653 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f32e406aed0 msgr2=0x7f32e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f32e406aed0 0x7f32e406d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b9d0 tx=0x7f32fc008040 comp rx=0 tx=0).stop 2024-01-31T05:01:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 msgr2=0x7f330c144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 0x7f330c144700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f330400c780 tx=0x7f330400cc40 comp rx=0 tx=0).stop 2024-01-31T05:01:21.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 shutdown_connections 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f32e406aed0 0x7f32e406d390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f330c15b470 0x7f330c144700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f330c0cb810 0x7f330c09c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 --2- 172.21.15.97:0/3278171034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f330c0abcf0 0x7f330c140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 >> 172.21.15.97:0/3278171034 conn(0x7f330c0b0a10 msgr2=0x7f330c159490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.656+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 shutdown_connections 2024-01-31T05:01:21.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.657+0000 7f3311600640 1 -- 172.21.15.97:0/3278171034 wait complete. 2024-01-31T05:01:21.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:01:21.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.882+0000 7fc86e800640 1 Processor -- start 2024-01-31T05:01:21.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.883+0000 7fc86e800640 1 -- start start 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc86e800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86815d590 0x7fc86815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc86e800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8680cff20 0x7fc868161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc86e800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680cded0 con 0x7fc8680cfae0 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc86e800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86815d970 con 0x7fc86815d590 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc86e800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680cf210 con 0x7fc8680cff20 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc867400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86815d590 0x7fc86815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc867400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86815d590 0x7fc86815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34572/0 (socket says 172.21.15.97:34572) 2024-01-31T05:01:21.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.884+0000 7fc867400640 1 -- 172.21.15.97:0/1206050219 learned_addr learned my addr 172.21.15.97:0/1206050219 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:21.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.885+0000 7fc867e00640 1 --2- 172.21.15.97:0/1206050219 >> v2:172.21.15.188:3300/0 conn(0x7fc8680cff20 0x7fc868161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.885+0000 7fc867400640 1 -- 172.21.15.97:0/1206050219 >> v2:172.21.15.188:3300/0 conn(0x7fc8680cff20 msgr2=0x7fc868161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:21.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.885+0000 7fc867400640 1 --2- 172.21.15.97:0/1206050219 >> v2:172.21.15.188:3300/0 conn(0x7fc8680cff20 0x7fc868161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.885+0000 7fc867400640 1 -- 172.21.15.97:0/1206050219 >> v1:172.21.15.97:6789/0 conn(0x7fc8680cfae0 legacy=0x7fc868161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.885+0000 7fc867400640 1 -- 172.21.15.97:0/1206050219 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680ce170 con 0x7fc86815d590 2024-01-31T05:01:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.886+0000 7fc867400640 1 --2- 172.21.15.97:0/1206050219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815d590 0x7fc86815e540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cdbd0 tx=0x7fc85802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9848313af9027a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.886+0000 7fc866000640 1 -- 172.21.15.97:0/1206050219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85802f850 con 0x7fc86815d590 2024-01-31T05:01:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.886+0000 7fc866000640 1 -- 172.21.15.97:0/1206050219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85802f9b0 con 0x7fc86815d590 2024-01-31T05:01:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.886+0000 7fc866000640 1 -- 172.21.15.97:0/1206050219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85802fd40 con 0x7fc86815d590 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 -- 172.21.15.97:0/1206050219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815d590 msgr2=0x7fc86815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 --2- 172.21.15.97:0/1206050219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815d590 0x7fc86815e540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cdbd0 tx=0x7fc85802ba30 comp rx=0 tx=0).stop 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 -- 172.21.15.97:0/1206050219 shutdown_connections 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 --2- 172.21.15.97:0/1206050219 >> v2:172.21.15.188:3300/0 conn(0x7fc8680cff20 0x7fc868161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 --2- 172.21.15.97:0/1206050219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86815d590 0x7fc86815e540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 -- 172.21.15.97:0/1206050219 >> 172.21.15.97:0/1206050219 conn(0x7fc86809c560 msgr2=0x7fc8680d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 -- 172.21.15.97:0/1206050219 shutdown_connections 2024-01-31T05:01:21.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.887+0000 7fc86e800640 1 -- 172.21.15.97:0/1206050219 wait complete. 2024-01-31T05:01:21.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.889+0000 7fc86e800640 1 Processor -- start 2024-01-31T05:01:21.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.889+0000 7fc86e800640 1 -- start start 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.890+0000 7fc86e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680cfae0 0x7fc8680c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc86e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 0x7fc8680d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc86e800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815d590 0x7fc8680d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc86e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680c7390 con 0x7fc8680cff20 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc86e800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680cd0e0 con 0x7fc86815d590 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc86e800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680cbf10 con 0x7fc8680cfae0 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 0x7fc8680d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 0x7fc8680d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34574/0 (socket says 172.21.15.97:34574) 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 -- 172.21.15.97:0/3776715004 learned_addr learned my addr 172.21.15.97:0/3776715004 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc867e00640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815d590 0x7fc8680d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc867400640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680cfae0 0x7fc8680c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 -- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680cfae0 msgr2=0x7fc8680c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680cfae0 0x7fc8680c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 -- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815d590 msgr2=0x7fc8680d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:21.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815d590 0x7fc8680d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:21.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.891+0000 7fc866a00640 1 -- 172.21.15.97:0/3776715004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580094d0 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.892+0000 7fc866a00640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 0x7fc8680d0dc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c790 tx=0x7fc85000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.892+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000cea0 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.892+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850006d50 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.892+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850003bb0 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.892+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680cc1f0 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.892+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8680c7020 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.893+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8680c7500 con 0x7fc8680cff20 2024-01-31T05:01:21.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.894+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc850003d10 con 0x7fc8680cff20 2024-01-31T05:01:21.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.895+0000 7fc85f400640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc84006aed0 0x7fc84006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:21.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.895+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc850089610 con 0x7fc8680cff20 2024-01-31T05:01:21.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.895+0000 7fc867400640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc84006aed0 0x7fc84006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:21.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.896+0000 7fc867400640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc84006aed0 0x7fc84006d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc858036000 tx=0x7fc858008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:21.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:21.901+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc850057bf0 con 0x7fc8680cff20 2024-01-31T05:01:22.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.132+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8680c6b20 con 0x7fc8680cff20 2024-01-31T05:01:22.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.134+0000 7fc85f400640 1 -- 172.21.15.97:0/3776715004 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fc850057780 con 0x7fc8680cff20 2024-01-31T05:01:22.133 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:22.133 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:01:22.134 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:01:22.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc84006aed0 msgr2=0x7fc84006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc84006aed0 0x7fc84006d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc858036000 tx=0x7fc858008030 comp rx=0 tx=0).stop 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 msgr2=0x7fc8680d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 0x7fc8680d0dc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc85000c790 tx=0x7fc85000cc50 comp rx=0 tx=0).stop 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 shutdown_connections 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815d590 0x7fc8680d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc84006aed0 0x7fc84006d390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8680cff20 0x7fc8680d0dc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 --2- 172.21.15.97:0/3776715004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680cfae0 0x7fc8680c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 >> 172.21.15.97:0/3776715004 conn(0x7fc86809c560 msgr2=0x7fc8680b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 shutdown_connections 2024-01-31T05:01:22.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.137+0000 7fc86e800640 1 -- 172.21.15.97:0/3776715004 wait complete. 2024-01-31T05:01:22.154 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:22.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.351+0000 7f32baa00640 1 Processor -- start 2024-01-31T05:01:22.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.351+0000 7f32baa00640 1 -- start start 2024-01-31T05:01:22.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.353+0000 7f32baa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 0x7f32b40cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.353+0000 7f32baa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b40abcf0 con 0x7f32b40ce430 2024-01-31T05:01:22.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.353+0000 7f32baa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b406ca40 con 0x7f32b41616d0 2024-01-31T05:01:22.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.353+0000 7f32baa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b415d900 con 0x7f32b415d520 2024-01-31T05:01:22.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.353+0000 7f32b3400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32b40ce430 0x7f32b4160fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33170/0 (socket says 172.21.15.97:33170) 2024-01-31T05:01:22.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.353+0000 7f32b3400640 1 -- 172.21.15.97:0/2667054519 learned_addr learned my addr 172.21.15.97:0/2667054519 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b3e00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 0x7f32b40cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665307799 0 0) 0x7f32b40abcf0 con 0x7f32b40ce430 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3294003520 con 0x7f32b40ce430 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 523155411 0 0) 0x7f3294003520 con 0x7f32b40ce430 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 msgr2=0x7f32b40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 0x7f32b40cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 >> v1:172.21.15.188:6789/0 conn(0x7f32b41616d0 legacy=0x7f32b406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40ce890 con 0x7f32b40ce430 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f329c001670 con 0x7f32b40ce430 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 >> v1:172.21.15.97:6789/0 conn(0x7f32b40ce430 legacy=0x7f32b4160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32940039a0 0x7f3294003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.354+0000 7f32b2a00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 0x7f32940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b2a00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3294004d00 0x7f3294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b415d900 con 0x7f32940039a0 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329400ba30 con 0x7f3294004d00 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329400bbf0 con 0x7f32b415d520 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3e00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32940039a0 0x7f3294003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3400640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 0x7f32940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3e00640 1 -- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 msgr2=0x7f32940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3e00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b415d520 0x7f32940047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3e00640 1 -- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3294004d00 msgr2=0x7f3294005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3e00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3294004d00 0x7f3294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.355+0000 7f32b3e00640 1 -- 172.21.15.97:0/2667054519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294003520 con 0x7f32940039a0 2024-01-31T05:01:22.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.356+0000 7f32b3e00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32940039a0 0x7f3294003dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009a90 tx=0x7f32940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fcc2c972c0329c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.356+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a402c680 con 0x7f32940039a0 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.356+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a402c7e0 con 0x7f32940039a0 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.356+0000 7f32b2a00640 1 -- 172.21.15.97:0/2667054519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a402cb70 con 0x7f32940039a0 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32baa00640 1 -- 172.21.15.97:0/2667054519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32940039a0 msgr2=0x7f3294003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32baa00640 1 --2- 172.21.15.97:0/2667054519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32940039a0 0x7f3294003dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009a90 tx=0x7f32940057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32b8a00640 1 -- 172.21.15.97:0/2667054519 reap_dead start 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32baa00640 1 -- 172.21.15.97:0/2667054519 shutdown_connections 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32baa00640 1 -- 172.21.15.97:0/2667054519 >> 172.21.15.97:0/2667054519 conn(0x7f32b40b0a10 msgr2=0x7f32b40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32baa00640 1 -- 172.21.15.97:0/2667054519 shutdown_connections 2024-01-31T05:01:22.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.357+0000 7f32baa00640 1 -- 172.21.15.97:0/2667054519 wait complete. 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.359+0000 7f32baa00640 1 Processor -- start 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.359+0000 7f32baa00640 1 -- start start 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32baa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32940039a0 0x7f32b41603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32baa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 0x7f32b41f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32baa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b41f6a50 0x7f32b409bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32baa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40acb80 con 0x7f32b413e2e0 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32baa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41f6f00 con 0x7f32940039a0 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32baa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40abcf0 con 0x7f32b41f6a50 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 0x7f32b41f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 0x7f32b41f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34580/0 (socket says 172.21.15.97:34580) 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b3400640 1 -- 172.21.15.97:0/3927867478 learned_addr learned my addr 172.21.15.97:0/3927867478 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:22.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b3400640 1 -- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b41f6a50 msgr2=0x7f32b409bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b8a00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b41f6a50 0x7f32b409bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b3400640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b41f6a50 0x7f32b409bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.360+0000 7f32b3e00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32940039a0 0x7f32b41603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32b3400640 1 -- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32940039a0 msgr2=0x7f32b41603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32b3400640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32940039a0 0x7f32b41603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32b3400640 1 -- 172.21.15.97:0/3927867478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b40abf00 con 0x7f32b413e2e0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32b3e00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32940039a0 0x7f32b41603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32b3400640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 0x7f32b41f6510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d820 tx=0x7f329c00dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0074e0 con 0x7f32b413e2e0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c007640 con 0x7f32b413e2e0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0049e0 con 0x7f32b413e2e0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b40ac1e0 con 0x7f32b413e2e0 2024-01-31T05:01:22.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.361+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b40ac7a0 con 0x7f32b413e2e0 2024-01-31T05:01:22.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.362+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b40accc0 con 0x7f32b413e2e0 2024-01-31T05:01:22.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.363+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f329c004b40 con 0x7f32b413e2e0 2024-01-31T05:01:22.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.364+0000 7f32abe00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f328c06af00 0x7f328c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.364+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f329c089320 con 0x7f32b413e2e0 2024-01-31T05:01:22.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.364+0000 7f32b3e00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f328c06af00 0x7f328c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.365+0000 7f32b3e00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f328c06af00 0x7f328c06d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f32a4036000 tx=0x7f32a4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:22.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.368+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f32b40accc0 con 0x7f32b413e2e0 2024-01-31T05:01:22.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.563+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b40accc0 con 0x7f32b413e2e0 2024-01-31T05:01:22.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.564+0000 7f32abe00640 1 -- 172.21.15.97:0/3927867478 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18351 (secure 0 0 0) 0x7f32b40accc0 con 0x7f32b413e2e0 2024-01-31T05:01:22.564 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:22.565 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:00:51.075713+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:22.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f328c06af00 msgr2=0x7f328c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f328c06af00 0x7f328c06d3c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f32a4036000 tx=0x7f32a4030000 comp rx=0 tx=0).stop 2024-01-31T05:01:22.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 msgr2=0x7f32b41f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 0x7f32b41f6510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d820 tx=0x7f329c00dce0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 shutdown_connections 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32b41f6a50 0x7f32b409bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f328c06af00 0x7f328c06d3c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b413e2e0 0x7f32b41f6510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 --2- 172.21.15.97:0/3927867478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32940039a0 0x7f32b41603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.567+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 >> 172.21.15.97:0/3927867478 conn(0x7f32b40b0a10 msgr2=0x7f32b40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.568+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 shutdown_connections 2024-01-31T05:01:22.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.568+0000 7f32baa00640 1 -- 172.21.15.97:0/3927867478 wait complete. 2024-01-31T05:01:22.586 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:01:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.798+0000 7f2462400640 1 Processor -- start 2024-01-31T05:01:22.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.798+0000 7f2462400640 1 -- start start 2024-01-31T05:01:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2462400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f245c0c9f40 0x7f245c0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2462400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f245c071e90 0x7f245c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2462400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245c15a5c0 con 0x7f245c0c9420 2024-01-31T05:01:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2462400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245c0ac400 con 0x7f245c071e90 2024-01-31T05:01:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2462400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245c0cd8d0 con 0x7f245c0c9f40 2024-01-31T05:01:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2461a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f245c071e90 0x7f245c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f245be00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f245c0c9420 0x7f245c0abcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:33172/0 (socket says 172.21.15.97:33172) 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f245be00640 1 -- 172.21.15.97:0/1816883986 learned_addr learned my addr 172.21.15.97:0/1816883986 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.800+0000 7f2461000640 1 --2- 172.21.15.97:0/1816883986 >> v2:172.21.15.188:3300/0 conn(0x7f245c0c9f40 0x7f245c0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.801+0000 7f2461a00640 1 -- 172.21.15.97:0/1816883986 >> v2:172.21.15.188:3300/0 conn(0x7f245c0c9f40 msgr2=0x7f245c0ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.801+0000 7f2461a00640 1 --2- 172.21.15.97:0/1816883986 >> v2:172.21.15.188:3300/0 conn(0x7f245c0c9f40 0x7f245c0ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.801+0000 7f2461a00640 1 -- 172.21.15.97:0/1816883986 >> v1:172.21.15.97:6789/0 conn(0x7f245c0c9420 legacy=0x7f245c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.801+0000 7f2461a00640 1 -- 172.21.15.97:0/1816883986 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c072320 con 0x7f245c071e90 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.801+0000 7f2461000640 1 --2- 172.21.15.97:0/1816883986 >> v2:172.21.15.188:3300/0 conn(0x7f245c0c9f40 0x7f245c0ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.801+0000 7f245b400640 1 -- 172.21.15.97:0/1816883986 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920520295 0 0) 0x7f245c15a5c0 con 0x7f245c0c9420 2024-01-31T05:01:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.802+0000 7f2461a00640 1 --2- 172.21.15.97:0/1816883986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c0cdd60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2454009d90 tx=0x7f245402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e47e4138a8097af server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.802+0000 7f245b400640 1 -- 172.21.15.97:0/1816883986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245402cc60 con 0x7f245c071e90 2024-01-31T05:01:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.802+0000 7f245b400640 1 -- 172.21.15.97:0/1816883986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245402cdc0 con 0x7f245c071e90 2024-01-31T05:01:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.802+0000 7f245b400640 1 -- 172.21.15.97:0/1816883986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454034660 con 0x7f245c071e90 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 -- 172.21.15.97:0/1816883986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 msgr2=0x7f245c0cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 --2- 172.21.15.97:0/1816883986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c0cdd60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2454009d90 tx=0x7f245402beb0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 -- 172.21.15.97:0/1816883986 shutdown_connections 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 --2- 172.21.15.97:0/1816883986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c0cdd60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 --2- 172.21.15.97:0/1816883986 >> v2:172.21.15.188:3300/0 conn(0x7f245c0c9f40 0x7f245c0ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 -- 172.21.15.97:0/1816883986 >> 172.21.15.97:0/1816883986 conn(0x7f245c0cec40 msgr2=0x7f245c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 -- 172.21.15.97:0/1816883986 shutdown_connections 2024-01-31T05:01:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.803+0000 7f2462400640 1 -- 172.21.15.97:0/1816883986 wait complete. 2024-01-31T05:01:22.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.805+0000 7f2462400640 1 Processor -- start 2024-01-31T05:01:22.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.805+0000 7f2462400640 1 -- start start 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.806+0000 7f2462400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c1e7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.806+0000 7f2462400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245c0c9420 0x7f245c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2462400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f245c0c9f40 0x7f245c153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2462400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1f6ce0 con 0x7f245c071e90 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2462400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c1f60f0 con 0x7f245c0c9f40 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2462400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245c153f40 con 0x7f245c0c9420 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c1e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c1e7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34586/0 (socket says 172.21.15.97:34586) 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 -- 172.21.15.97:0/2263558860 learned_addr learned my addr 172.21.15.97:0/2263558860 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 -- 172.21.15.97:0/2263558860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245c0c9420 msgr2=0x7f245c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245c0c9420 0x7f245c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 -- 172.21.15.97:0/2263558860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f245c0c9f40 msgr2=0x7f245c153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:22.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f245c0c9f40 0x7f245c153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:22.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.807+0000 7f2461000640 1 -- 172.21.15.97:0/2263558860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2454009a70 con 0x7f245c071e90 2024-01-31T05:01:22.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.808+0000 7f2461000640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c1e7030 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d7d0 tx=0x7f244c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.808+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c004030 con 0x7f245c071e90 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.808+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c004190 con 0x7f245c071e90 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.808+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c004520 con 0x7f245c071e90 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.808+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c1f6340 con 0x7f245c071e90 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.808+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245c1f6900 con 0x7f245c071e90 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.809+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f245c0676b0 con 0x7f245c071e90 2024-01-31T05:01:22.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.810+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f244c0096a0 con 0x7f245c071e90 2024-01-31T05:01:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.811+0000 7f2458c00640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f243406aed0 0x7f243406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.811+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f244c089ac0 con 0x7f245c071e90 2024-01-31T05:01:22.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.811+0000 7f245be00640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f243406aed0 0x7f243406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:22.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.812+0000 7f245be00640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f243406aed0 0x7f243406d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2444007fd0 tx=0x7f2444007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:22.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:22.818+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f244c054690 con 0x7f245c071e90 2024-01-31T05:01:23.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.048+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f245c0ac5e0 con 0x7f245c071e90 2024-01-31T05:01:23.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.049+0000 7f2458c00640 1 -- 172.21.15.97:0/2263558860 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f244c057cb0 con 0x7f245c071e90 2024-01-31T05:01:23.049 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:23.049 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:01:23.049 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:01:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f243406aed0 msgr2=0x7f243406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f243406aed0 0x7f243406d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2444007fd0 tx=0x7f2444007920 comp rx=0 tx=0).stop 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 msgr2=0x7f245c1e7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c1e7030 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d7d0 tx=0x7f244c00dc90 comp rx=0 tx=0).stop 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 shutdown_connections 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f245c0c9f40 0x7f245c153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245c0c9420 0x7f245c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f243406aed0 0x7f243406d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 --2- 172.21.15.97:0/2263558860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f245c071e90 0x7f245c1e7030 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 >> 172.21.15.97:0/2263558860 conn(0x7f245c0cec40 msgr2=0x7f245c1558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 shutdown_connections 2024-01-31T05:01:23.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.052+0000 7f2462400640 1 -- 172.21.15.97:0/2263558860 wait complete. 2024-01-31T05:01:23.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-31T05:00:46.002409+0000', 'modified': '2024-01-31T05:01:08.303380+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4257}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4257': {'gid': 4257, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/294460102', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 294460102}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 294460102}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-31T05:01:23.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-31T05:01:23.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-31T05:01:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.689+0000 7fead9a00640 1 Processor -- start 2024-01-31T05:01:23.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.689+0000 7fead9a00640 1 -- start start 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.690+0000 7fead9a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fead415b080 0x7fead40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.690+0000 7fead9a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40b3f60 con 0x7fead40ce680 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.690+0000 7fead9a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead415c540 con 0x7fead40ae360 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.690+0000 7fead9a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead40b41c0 con 0x7fead415b080 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.691+0000 7fead3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fead415b080 0x7fead40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.691+0000 7fead3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fead415b080 0x7fead40b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34594/0 (socket says 172.21.15.97:34594) 2024-01-31T05:01:23.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.691+0000 7fead3e00640 1 -- 172.21.15.97:0/2192052165 learned_addr learned my addr 172.21.15.97:0/2192052165 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.691+0000 7fead3e00640 1 -- 172.21.15.97:0/2192052165 >> v1:172.21.15.188:6789/0 conn(0x7fead40ae360 legacy=0x7fead40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.691+0000 7fead3e00640 1 -- 172.21.15.97:0/2192052165 >> v1:172.21.15.97:6789/0 conn(0x7fead40ce680 legacy=0x7fead40b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.691+0000 7fead3e00640 1 -- 172.21.15.97:0/2192052165 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead415b780 con 0x7fead415b080 2024-01-31T05:01:23.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.692+0000 7fead3e00640 1 --2- 172.21.15.97:0/2192052165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead415b080 0x7fead40b3a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feac4009870 tx=0x7feac402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0083b8778d9fb5d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.692+0000 7fead3400640 1 -- 172.21.15.97:0/2192052165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4030850 con 0x7fead415b080 2024-01-31T05:01:23.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead3400640 1 -- 172.21.15.97:0/2192052165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac40309b0 con 0x7fead415b080 2024-01-31T05:01:23.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead3400640 1 -- 172.21.15.97:0/2192052165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4030d40 con 0x7fead415b080 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 -- 172.21.15.97:0/2192052165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead415b080 msgr2=0x7fead40b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 --2- 172.21.15.97:0/2192052165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead415b080 0x7fead40b3a20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feac4009870 tx=0x7feac402b990 comp rx=0 tx=0).stop 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 -- 172.21.15.97:0/2192052165 shutdown_connections 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 --2- 172.21.15.97:0/2192052165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead415b080 0x7fead40b3a20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 -- 172.21.15.97:0/2192052165 >> 172.21.15.97:0/2192052165 conn(0x7fead406ad40 msgr2=0x7fead40b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 -- 172.21.15.97:0/2192052165 shutdown_connections 2024-01-31T05:01:23.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.693+0000 7fead9a00640 1 -- 172.21.15.97:0/2192052165 wait complete. 2024-01-31T05:01:23.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.695+0000 7fead9a00640 1 Processor -- start 2024-01-31T05:01:23.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.695+0000 7fead9a00640 1 -- start start 2024-01-31T05:01:23.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7fead9a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fead40ae360 0x7fead41419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7fead9a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 0x7fead4159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7fead9a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fead415b080 0x7fead40af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7fead9a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead41f4f90 con 0x7fead40ce680 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7fead9a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead41f4380 con 0x7fead40ae360 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7fead9a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead41f4660 con 0x7fead415b080 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7feacb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 0x7fead4159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7feacb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 0x7fead4159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34598/0 (socket says 172.21.15.97:34598) 2024-01-31T05:01:23.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.696+0000 7feacb200640 1 -- 172.21.15.97:0/196465974 learned_addr learned my addr 172.21.15.97:0/196465974 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead3e00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fead40ae360 0x7fead41419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead9000640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fead415b080 0x7fead40af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7feacb200640 1 -- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fead415b080 msgr2=0x7fead40af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7feacb200640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fead415b080 0x7fead40af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7feacb200640 1 -- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fead40ae360 msgr2=0x7fead41419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7feacb200640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fead40ae360 0x7fead41419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7feacb200640 1 -- 172.21.15.97:0/196465974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40094d0 con 0x7fead40ce680 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead3e00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fead40ae360 0x7fead41419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7feacb200640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 0x7fead4159660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b3f0 tx=0x7feabc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead0c00640 1 -- 172.21.15.97:0/196465974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc011830 con 0x7fead40ce680 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead0c00640 1 -- 172.21.15.97:0/196465974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc011990 con 0x7fead40ce680 2024-01-31T05:01:23.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead41f4940 con 0x7fead40ce680 2024-01-31T05:01:23.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead0c00640 1 -- 172.21.15.97:0/196465974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc011d40 con 0x7fead40ce680 2024-01-31T05:01:23.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.697+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4141f30 con 0x7fead40ce680 2024-01-31T05:01:23.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.700+0000 7fead0c00640 1 -- 172.21.15.97:0/196465974 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7feabc01a400 con 0x7fead40ce680 2024-01-31T05:01:23.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.701+0000 7fead0c00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feaac06af00 0x7feaac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.701+0000 7fead0c00640 1 -- 172.21.15.97:0/196465974 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feabc089780 con 0x7fead40ce680 2024-01-31T05:01:23.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.701+0000 7feaddfad740 1 Processor -- start 2024-01-31T05:01:23.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.701+0000 7fead3e00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feaac06af00 0x7feaac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.701+0000 7feaddfad740 1 -- start start 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7fead3e00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feaac06af00 0x7feaac06d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feac4036000 tx=0x7feac403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7feaddfad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 0x5617c9576650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9554e70 0x5617c9557f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c95500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7feaddfad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c949af50 con 0x5617c9557bb0 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9498640 con 0x5617c9554e70 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.702+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617c9493f30 con 0x5617c95584d0 2024-01-31T05:01:23.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 0x5617c9576650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 0x5617c9576650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34602/0 (socket says 172.21.15.97:34602) 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 -- 172.21.15.97:0/3662449540 learned_addr learned my addr 172.21.15.97:0/3662449540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7feacb200640 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9554e70 0x5617c9557f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead9000640 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c95500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 -- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 msgr2=0x5617c95500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c95500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 -- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9554e70 msgr2=0x5617c9557f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9554e70 0x5617c9557f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 -- 172.21.15.97:0/3662449540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c94562d0 con 0x5617c9557bb0 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead9000640 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c95500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.703+0000 7fead3e00640 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 0x5617c9576650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feac4009960 tx=0x7feac4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456f418700b253d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feac8a00640 1 -- 172.21.15.97:0/3662449540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4044b70 con 0x5617c9557bb0 2024-01-31T05:01:23.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feac8a00640 1 -- 172.21.15.97:0/3662449540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac404a040 con 0x5617c9557bb0 2024-01-31T05:01:23.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feac8a00640 1 -- 172.21.15.97:0/3662449540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4039450 con 0x5617c9557bb0 2024-01-31T05:01:23.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 -- 172.21.15.97:0/3662449540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 msgr2=0x5617c9576650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 0x5617c9576650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7feac4009960 tx=0x7feac4030000 comp rx=0 tx=0).stop 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 -- 172.21.15.97:0/3662449540 shutdown_connections 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c95500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9554e70 0x5617c9557f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 --2- 172.21.15.97:0/3662449540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c9557bb0 0x5617c9576650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 -- 172.21.15.97:0/3662449540 >> 172.21.15.97:0/3662449540 conn(0x5617c9529e40 msgr2=0x5617c9575f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 -- 172.21.15.97:0/3662449540 shutdown_connections 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.704+0000 7feaddfad740 1 -- 172.21.15.97:0/3662449540 wait complete. 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.705+0000 7feaddfad740 1 -- start start 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.705+0000 7feaddfad740 1 Processor -- start 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c95858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c9573210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c956f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c944d6b0 con 0x5617c956f390 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c944d350 con 0x5617c9557bb0 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617c91f5f20 con 0x5617c95584d0 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.706+0000 7feaddfad740 10 client.0 _ll_register_callbacks cb 0x5617c944f9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c956f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c956f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34610/0 (socket says 172.21.15.97:34610) 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 -- 172.21.15.97:0/822685286 learned_addr learned my addr 172.21.15.97:0/822685286 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead3e00640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c95858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7feacb200640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c9573210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 -- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 msgr2=0x5617c9573210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c9573210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 -- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 msgr2=0x5617c95858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c95858a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 -- 172.21.15.97:0/822685286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead415b780 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7fead9000640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c956f810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feacc00e500 tx=0x7feacc00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c1ac1da512a73f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.707+0000 7feac1600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feac1600640 1 -- 172.21.15.97:0/822685286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc013870 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feac1600640 1 -- 172.21.15.97:0/822685286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feacc0139d0 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617c94bbe60 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feac1600640 1 -- 172.21.15.97:0/822685286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7feacc013b90 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 10 client.4860 fetch_fsmap learned FSMap version 8 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5617c94925b0 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feac1600640 1 -- 172.21.15.97:0/822685286 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc020030 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feac1600640 1 -- 172.21.15.97:0/822685286 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7feacc013cf0 con 0x5617c956f390 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 10 client.4860 fetch_fsmap finished waiting for FSMap version 8 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 10 client.4860 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-31T05:01:23.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 20 client.4860 populate_metadata read hostname 'smithi097' 2024-01-31T05:01:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c955a3e0 0x5617c9582a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 4 client.4860 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:01:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.708+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 --> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617c9465c40 con 0x5617c955a3e0 2024-01-31T05:01:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.709+0000 7fead3e00640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c955a3e0 0x5617c9582a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.709+0000 7fead3e00640 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c955a3e0 0x5617c9582a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8f4980a80ab07a server_cookie=99a5ec8251ff8e5f in_seq=0 out_seq=0 2024-01-31T05:01:23.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.709+0000 7feac1600640 10 client.4860 ms_handle_connect on v2:172.21.15.97:6835/294460102 2024-01-31T05:01:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feac1600640 1 -- 172.21.15.97:0/822685286 <== mds.0 v2:172.21.15.97:6835/294460102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617c9465c40 con 0x5617c955a3e0 2024-01-31T05:01:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feac1600640 10 client.4860 handle_command_reply: tid=0 2024-01-31T05:01:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 client.4860 shutdown 2024-01-31T05:01:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 msgr2=0x5617c956f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c956f810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feacc00e500 tx=0x7feacc00e9c0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 shutdown_connections 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c955a3e0 0x5617c9582a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c956f810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c9573210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feac1600640 0 client.4860 ms_handle_reset on v2:172.21.15.97:6835/294460102 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 --2- 172.21.15.97:0/822685286 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c95858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 >> 172.21.15.97:0/822685286 conn(0x5617c9554e70 msgr2=0x5617c954d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 shutdown_connections 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.710+0000 7feaddfad740 1 -- 172.21.15.97:0/822685286 wait complete. 2024-01-31T05:01:23.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.711+0000 7feaddfad740 20 client.4860 trim_cache size 0 max 16384 2024-01-31T05:01:23.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.727+0000 7feaddfad740 1 Processor -- start 2024-01-31T05:01:23.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.727+0000 7feaddfad740 1 -- start start 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c9557f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feaddfad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 0x5617c956f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c956f390 0x5617c95db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feaddfad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c95dbf20 con 0x5617c95584d0 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c943e900 con 0x5617c9557bb0 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617c9573660 con 0x5617c956f390 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 0x5617c956f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 0x5617c956f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34626/0 (socket says 172.21.15.97:34626) 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 -- 172.21.15.97:0/1798598598 learned_addr learned my addr 172.21.15.97:0/1798598598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:23.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7fead3e00640 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c9557f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7fead9000640 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c956f390 0x5617c95db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 -- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c956f390 msgr2=0x5617c95db9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c956f390 0x5617c95db9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 -- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 msgr2=0x5617c9557f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c9557f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.729+0000 7feacb200640 1 -- 172.21.15.97:0/1798598598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c942f1e0 con 0x5617c95584d0 2024-01-31T05:01:23.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.730+0000 7feacb200640 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 0x5617c956f770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feabc05fd50 tx=0x7feabc0579e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7700d897d54a34f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.730+0000 7feac3e00640 1 -- 172.21.15.97:0/1798598598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc06f070 con 0x5617c95584d0 2024-01-31T05:01:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.730+0000 7feac3e00640 1 -- 172.21.15.97:0/1798598598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feabc060280 con 0x5617c95584d0 2024-01-31T05:01:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.730+0000 7feac3e00640 1 -- 172.21.15.97:0/1798598598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc05e0b0 con 0x5617c95584d0 2024-01-31T05:01:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.730+0000 7feaddfad740 1 -- 172.21.15.97:0/1798598598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 msgr2=0x5617c956f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.730+0000 7feaddfad740 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 0x5617c956f770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feabc05fd50 tx=0x7feabc0579e0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 -- 172.21.15.97:0/1798598598 shutdown_connections 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c956f390 0x5617c95db9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c95584d0 0x5617c956f770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 --2- 172.21.15.97:0/1798598598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c9557f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 -- 172.21.15.97:0/1798598598 >> 172.21.15.97:0/1798598598 conn(0x5617c9554e70 msgr2=0x5617c9527e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 -- 172.21.15.97:0/1798598598 shutdown_connections 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 -- 172.21.15.97:0/1798598598 wait complete. 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 -- start start 2024-01-31T05:01:23.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.731+0000 7feaddfad740 1 Processor -- start 2024-01-31T05:01:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c956b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c952ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c9527ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c9558360 con 0x5617c956f390 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c95720e0 con 0x5617c9557bb0 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617c95db9e0 con 0x5617c95584d0 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.732+0000 7feaddfad740 10 client.0 _ll_register_callbacks cb 0x7feacc007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c9527ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c9527ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34642/0 (socket says 172.21.15.97:34642) 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 -- 172.21.15.97:0/2304802469 learned_addr learned my addr 172.21.15.97:0/2304802469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead3e00640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c956b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7feacb200640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c952ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 -- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 msgr2=0x5617c952ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c952ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 -- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 msgr2=0x5617c956b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c956b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 -- 172.21.15.97:0/2304802469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4141f30 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7fead9000640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c9527ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feacc00ea00 tx=0x7feacc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e85d0200c2f8e54 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.733+0000 7feac1600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feac1600640 1 -- 172.21.15.97:0/2304802469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc01c3f0 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feac1600640 1 -- 172.21.15.97:0/2304802469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feacc01c550 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feac404a040 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feac1600640 1 -- 172.21.15.97:0/2304802469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7feacc01c750 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 10 client.4866 fetch_fsmap learned FSMap version 8 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5617c9454b30 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feac1600640 1 -- 172.21.15.97:0/2304802469 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac404a040 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feac1600640 1 -- 172.21.15.97:0/2304802469 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7feacc02b020 con 0x5617c956f390 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 10 client.4866 fetch_fsmap finished waiting for FSMap version 8 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 10 client.4866 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-31T05:01:23.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 20 client.4866 populate_metadata read hostname 'smithi097' 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c9582d50 0x5617c9583130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 4 client.4866 mds_command: new command op to 4257 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.734+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 --> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5617c948cdc0 con 0x5617c9582d50 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.735+0000 7fead3e00640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c9582d50 0x5617c9583130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.735+0000 7fead3e00640 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c9582d50 0x5617c9583130 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6260d569fd349bcc server_cookie=c07271a8b1416a19 in_seq=0 out_seq=0 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.735+0000 7feac1600640 10 client.4866 ms_handle_connect on v2:172.21.15.97:6835/294460102 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.735+0000 7feac1600640 1 -- 172.21.15.97:0/2304802469 <== mds.0 v2:172.21.15.97:6835/294460102 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5617c948cdc0 con 0x5617c9582d50 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.735+0000 7feac1600640 10 client.4866 handle_command_reply: tid=0 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.735+0000 7feaddfad740 1 client.4866 shutdown 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 msgr2=0x5617c9527ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c9527ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feacc00ea00 tx=0x7feacc011000 comp rx=0 tx=0).stop 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 shutdown_connections 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:6835/294460102,v1:172.21.15.97:6837/294460102] conn(0x5617c9582d50 0x5617c9583130 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5617c956f390 0x5617c9527ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5617c95584d0 0x5617c952ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feac1600640 0 client.4866 ms_handle_reset on v2:172.21.15.97:6835/294460102 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 --2- 172.21.15.97:0/2304802469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5617c9557bb0 0x5617c956b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 >> 172.21.15.97:0/2304802469 conn(0x5617c9554e70 msgr2=0x5617c95840f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 shutdown_connections 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 1 -- 172.21.15.97:0/2304802469 wait complete. 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.736+0000 7feaddfad740 20 client.4866 trim_cache size 0 max 16384 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:01:23.739 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "dc34a409-d027-49db-a591-dbd89a074581", 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4257, 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 37.148045187999998, 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 8, 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 26, 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 26, 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 44.429352647999998 2024-01-31T05:01:23.740 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feaac06af00 msgr2=0x7feaac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feaac06af00 0x7feaac06d3c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feac4036000 tx=0x7feac403b040 comp rx=0 tx=0).stop 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 msgr2=0x7fead4159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 0x7fead4159660 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feabc00b3f0 tx=0x7feabc00b8b0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 shutdown_connections 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fead415b080 0x7fead40af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feaac06af00 0x7feaac06d3c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fead40ce680 0x7fead4159660 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 --2- 172.21.15.97:0/196465974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fead40ae360 0x7fead41419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 >> 172.21.15.97:0/196465974 conn(0x7fead406ad40 msgr2=0x7fead406cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 shutdown_connections 2024-01-31T05:01:23.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:23.737+0000 7fead9a00640 1 -- 172.21.15.97:0/196465974 wait complete. 2024-01-31T05:01:23.753 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-01-31T05:01:23.778 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-01-31T05:01:23.779 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-31T05:01:23.779 INFO:tasks.ceph_fuse:client.0 config is {} 2024-01-31T05:01:23.779 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-31T05:01:23.779 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:23.779 INFO:tasks.ceph_fuse:client.2 config is {} 2024-01-31T05:01:23.779 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-31T05:01:23.779 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:23.779 INFO:tasks.ceph_fuse:client.1 config is {} 2024-01-31T05:01:23.780 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-31T05:01:23.780 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:23.780 INFO:tasks.ceph_fuse:client.3 config is {} 2024-01-31T05:01:23.780 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-31T05:01:23.780 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:23.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:23.780 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:01:23.831 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-01-31T05:01:23.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:23.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-01-31T05:01:23.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:23.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-01-31T05:01:24.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:24.043 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-31T05:01:24.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:24.099 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-01-31T05:01:24.190 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-01-31T05:01:24.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:01:24.195 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-01-31T05:01:24.195 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:01:24.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.409+0000 7f7832e00640 1 Processor -- start 2024-01-31T05:01:24.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.409+0000 7f7832e00640 1 -- start start 2024-01-31T05:01:24.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.411+0000 7f7832e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.411+0000 7f7832e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f782c06db70 0x7f782c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.411+0000 7f7832e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c06d1a0 con 0x7f782c0cb810 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.411+0000 7f7832e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782c0cc960 con 0x7f782c06db70 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.411+0000 7f7832e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c06c600 con 0x7f782c15b400 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.412+0000 7f7823e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.412+0000 7f7823e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:56908/0 (socket says 172.21.15.97:56908) 2024-01-31T05:01:24.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.412+0000 7f7830e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f782c06db70 0x7f782c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:24.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.412+0000 7f7823e00640 1 -- 172.21.15.97:0/1960000217 learned_addr learned my addr 172.21.15.97:0/1960000217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:24.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.412+0000 7f782be00640 1 --1- 172.21.15.97:0/1960000217 >> v1:172.21.15.188:6789/0 conn(0x7f782c0cb810 0x7f782c0cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:50404/0 (socket says 172.21.15.97:50404) 2024-01-31T05:01:24.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.413+0000 7f7823e00640 1 -- 172.21.15.97:0/1960000217 >> v2:172.21.15.188:3300/0 conn(0x7f782c06db70 msgr2=0x7f782c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:24.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.413+0000 7f7823e00640 1 --2- 172.21.15.97:0/1960000217 >> v2:172.21.15.188:3300/0 conn(0x7f782c06db70 0x7f782c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.413+0000 7f7823e00640 1 -- 172.21.15.97:0/1960000217 >> v1:172.21.15.188:6789/0 conn(0x7f782c0cb810 legacy=0x7f782c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:24.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.413+0000 7f782b400640 1 -- 172.21.15.97:0/1960000217 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778036452 0 0) 0x7f782c06d1a0 con 0x7f782c0cb810 2024-01-31T05:01:24.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.413+0000 7f7823e00640 1 -- 172.21.15.97:0/1960000217 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c06c870 con 0x7f782c15b400 2024-01-31T05:01:24.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.414+0000 7f7823e00640 1 --2- 172.21.15.97:0/1960000217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c06d630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7814009870 tx=0x7f781402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a87874a256b7a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:24.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.415+0000 7f782b400640 1 -- 172.21.15.97:0/1960000217 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814030840 con 0x7f782c15b400 2024-01-31T05:01:24.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.415+0000 7f782b400640 1 -- 172.21.15.97:0/1960000217 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78140309a0 con 0x7f782c15b400 2024-01-31T05:01:24.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.415+0000 7f782b400640 1 -- 172.21.15.97:0/1960000217 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814030d30 con 0x7f782c15b400 2024-01-31T05:01:24.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.415+0000 7f7832e00640 1 -- 172.21.15.97:0/1960000217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 msgr2=0x7f782c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:24.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.415+0000 7f7832e00640 1 --2- 172.21.15.97:0/1960000217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c06d630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7814009870 tx=0x7f781402b980 comp rx=0 tx=0).stop 2024-01-31T05:01:24.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.416+0000 7f7832e00640 1 -- 172.21.15.97:0/1960000217 shutdown_connections 2024-01-31T05:01:24.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.416+0000 7f7832e00640 1 --2- 172.21.15.97:0/1960000217 >> v2:172.21.15.188:3300/0 conn(0x7f782c06db70 0x7f782c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.416+0000 7f7832e00640 1 --2- 172.21.15.97:0/1960000217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c06d630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.416+0000 7f7832e00640 1 -- 172.21.15.97:0/1960000217 >> 172.21.15.97:0/1960000217 conn(0x7f782c0b0a10 msgr2=0x7f782c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:24.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.416+0000 7f7832e00640 1 -- 172.21.15.97:0/1960000217 shutdown_connections 2024-01-31T05:01:24.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.416+0000 7f7832e00640 1 -- 172.21.15.97:0/1960000217 wait complete. 2024-01-31T05:01:24.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.417+0000 7f7832e00640 1 Processor -- start 2024-01-31T05:01:24.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.418+0000 7f7832e00640 1 -- start start 2024-01-31T05:01:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.418+0000 7f7832e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 0x7f782c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.418+0000 7f7832e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782c0cb810 0x7f782c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7832e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c1f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7832e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c15d230 con 0x7f782c06db70 2024-01-31T05:01:24.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7832e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0c8250 con 0x7f782c0cb810 2024-01-31T05:01:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7832e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c15c700 con 0x7f782c15b400 2024-01-31T05:01:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 0x7f782c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 0x7f782c15d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34646/0 (socket says 172.21.15.97:34646) 2024-01-31T05:01:24.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 -- 172.21.15.97:0/4203504686 learned_addr learned my addr 172.21.15.97:0/4203504686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:24.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7830e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c1f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:24.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7823e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782c0cb810 0x7f782c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 -- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 msgr2=0x7f782c1f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c1f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 -- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782c0cb810 msgr2=0x7f782c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782c0cb810 0x7f782c15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f782be00640 1 -- 172.21.15.97:0/4203504686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140094d0 con 0x7f782c06db70 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.419+0000 7f7830e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c1f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.420+0000 7f782be00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 0x7f782c15d850 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c4d0 tx=0x7f781c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.420+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0074e0 con 0x7f782c06db70 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.420+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c007640 con 0x7f782c06db70 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.420+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c004a20 con 0x7f782c06db70 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.420+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c15c990 con 0x7f782c06db70 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.420+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c15cf50 con 0x7f782c06db70 2024-01-31T05:01:24.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.421+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c0677c0 con 0x7f782c06db70 2024-01-31T05:01:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.422+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f781c004b80 con 0x7f782c06db70 2024-01-31T05:01:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.423+0000 7f7828c00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f780406aed0 0x7f780406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.423+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f781c088b50 con 0x7f782c06db70 2024-01-31T05:01:24.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.423+0000 7f7823e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f780406aed0 0x7f780406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:24.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.424+0000 7f7823e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f780406aed0 0x7f780406d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7814036000 tx=0x7f781403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:24.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.427+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f781c0537c0 con 0x7f782c06db70 2024-01-31T05:01:24.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.659+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f782c156310 con 0x7f782c06db70 2024-01-31T05:01:24.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.659+0000 7f7828c00640 1 -- 172.21.15.97:0/4203504686 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f781c018070 con 0x7f782c06db70 2024-01-31T05:01:24.659 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f780406aed0 msgr2=0x7f780406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f780406aed0 0x7f780406d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7814036000 tx=0x7f781403b040 comp rx=0 tx=0).stop 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 msgr2=0x7f782c15d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 0x7f782c15d850 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c4d0 tx=0x7f781c00c990 comp rx=0 tx=0).stop 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 shutdown_connections 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f782c15b400 0x7f782c1f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f782c0cb810 0x7f782c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f780406aed0 0x7f780406d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 --2- 172.21.15.97:0/4203504686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f782c06db70 0x7f782c15d850 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:24.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.662+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 >> 172.21.15.97:0/4203504686 conn(0x7f782c0b0a10 msgr2=0x7f782c070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:24.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.663+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 shutdown_connections 2024-01-31T05:01:24.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:24.663+0000 7f7832e00640 1 -- 172.21.15.97:0/4203504686 wait complete. 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:01:24.681 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:01:24.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:01:24.682 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:01:24.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:24.682 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:01:24.726 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:24.726 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:01:24.727 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42439sec preferred_lft 42439sec 2024-01-31T05:01:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:01:24.728 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:24.729 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-31T05:01:24.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-31T05:01:24.729 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:24.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:24.988 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:01:25.063 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:01:25.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.067 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:01:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:01:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:01:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:01:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:01:25.118 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:01:25.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.118 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:25.118 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:25.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:01:25.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:01:25.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:01:25.119 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:25.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.299 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:01:25.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.347 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-01-31T05:01:25.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.444 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:25.444 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:25.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:25.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-31T05:01:25.445 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:25.581 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-31T05:01:25.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.581 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:25.581 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:25.581 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-31T05:01:25.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-31T05:01:25.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-31T05:01:25.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-31T05:01:25.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-31T05:01:25.582 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:25.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:25.814 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:25.815 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:25.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:01:25.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-01-31T05:01:25.815 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:25.980 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:01:25.980 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:25.981 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:26.027 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:01:26.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:26.027 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:26.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:01:26.200 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:01:26.248 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:26.249 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:26.250 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:26.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:26.251 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:01:26.293 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:01:26.294 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:01:26.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:01:26.423 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:01:26.464 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:01:26.465 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:26.466 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:26.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:26.467 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:01:26.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:01:26.521+0000 7f67fdcb9480 -1 init, newargv = 0x55e21b016770 newargc=15 2024-01-31T05:01:26.520 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[46525]: starting ceph client 2024-01-31T05:01:26.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[46525]: starting fuse 2024-01-31T05:01:26.549 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:01:26.550 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:01:26.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:01:26.550 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:01:26.550 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:01:26.550 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:01:26.550 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:01:26.550 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:01:26.551 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:01:26.552 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:01:26.553 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:01:26.553 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:26.553 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:01:26.553 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:01:26.740 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.46525.asok 2024-01-31T05:01:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:01:26.758 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46525.asok 2024-01-31T05:01:26.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:26.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46525.asok status 2024-01-31T05:01:26.977 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:01:26.977 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:01:26.977 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:01:26.977 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "46525", 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4872, 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:01:26.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4872 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 659501036 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 659501036 2024-01-31T05:01:26.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4872 172.21.15.97:0/659501036", 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "172.21.15.97:0/659501036", 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 8, 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 26, 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:01:26.980 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:01:26.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:01:27.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.192+0000 7f4951400640 1 Processor -- start 2024-01-31T05:01:27.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.192+0000 7f4951400640 1 -- start start 2024-01-31T05:01:27.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f4951400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0cfd40 0x7f494c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f4951400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f494c15da20 0x7f494c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f4951400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f494c0af8a0 0x7f494c0ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f4951400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494c0aef40 con 0x7f494c0af8a0 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f4951400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494c0d0120 con 0x7f494c15da20 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f4951400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0aebf0 con 0x7f494c0cfd40 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.193+0000 7f494b200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f494c0af8a0 0x7f494c0ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f494c0af8a0 0x7f494c0ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54652/0 (socket says 172.21.15.97:54652) 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 -- 172.21.15.97:0/731393866 learned_addr learned my addr 172.21.15.97:0/731393866 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494a800640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0cfd40 0x7f494c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f4943e00640 1 --2- 172.21.15.97:0/731393866 >> v2:172.21.15.188:3300/0 conn(0x7f494c15da20 0x7f494c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 -- 172.21.15.97:0/731393866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0cfd40 msgr2=0x7f494c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0cfd40 0x7f494c0cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 -- 172.21.15.97:0/731393866 >> v2:172.21.15.188:3300/0 conn(0x7f494c15da20 msgr2=0x7f494c0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:27.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 --2- 172.21.15.97:0/731393866 >> v2:172.21.15.188:3300/0 conn(0x7f494c15da20 0x7f494c0af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494b200640 1 -- 172.21.15.97:0/731393866 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c15e200 con 0x7f494c0af8a0 2024-01-31T05:01:27.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.194+0000 7f494a800640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0cfd40 0x7f494c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.195+0000 7f494b200640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0af8a0 0x7f494c0ce8e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f494c071940 tx=0x7f494402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6363ac7d3c158153 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.195+0000 7f4949e00640 1 -- 172.21.15.97:0/731393866 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494402ca70 con 0x7f494c0af8a0 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.195+0000 7f4949e00640 1 -- 172.21.15.97:0/731393866 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494402cbd0 con 0x7f494c0af8a0 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.195+0000 7f4949e00640 1 -- 172.21.15.97:0/731393866 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49440343f0 con 0x7f494c0af8a0 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 -- 172.21.15.97:0/731393866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0af8a0 msgr2=0x7f494c0ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0af8a0 0x7f494c0ce8e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f494c071940 tx=0x7f494402bed0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 -- 172.21.15.97:0/731393866 shutdown_connections 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0af8a0 0x7f494c0ce8e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 --2- 172.21.15.97:0/731393866 >> v2:172.21.15.188:3300/0 conn(0x7f494c15da20 0x7f494c0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 --2- 172.21.15.97:0/731393866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0cfd40 0x7f494c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 -- 172.21.15.97:0/731393866 >> 172.21.15.97:0/731393866 conn(0x7f494c071c70 msgr2=0x7f494c072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:27.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 -- 172.21.15.97:0/731393866 shutdown_connections 2024-01-31T05:01:27.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.196+0000 7f4951400640 1 -- 172.21.15.97:0/731393866 wait complete. 2024-01-31T05:01:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.199+0000 7f4951400640 1 Processor -- start 2024-01-31T05:01:27.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.200+0000 7f4951400640 1 -- start start 2024-01-31T05:01:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4951400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0af8a0 0x7f494c0c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4951400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 0x7f494c1f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4951400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f494c15da20 0x7f494c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4951400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0c3830 con 0x7f494c0cfd40 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4951400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0c3ad0 con 0x7f494c15da20 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4951400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c0c3db0 con 0x7f494c0af8a0 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4943e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 0x7f494c1f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4943e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 0x7f494c1f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54654/0 (socket says 172.21.15.97:54654) 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.201+0000 7f4943e00640 1 -- 172.21.15.97:0/2655941267 learned_addr learned my addr 172.21.15.97:0/2655941267 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f494a800640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0af8a0 0x7f494c0c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f494b200640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f494c15da20 0x7f494c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4943e00640 1 -- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0af8a0 msgr2=0x7f494c0c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4943e00640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0af8a0 0x7f494c0c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4943e00640 1 -- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f494c15da20 msgr2=0x7f494c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4943e00640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f494c15da20 0x7f494c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4943e00640 1 -- 172.21.15.97:0/2655941267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009b30 con 0x7f494c0cfd40 2024-01-31T05:01:27.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4943e00640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 0x7f494c1f2460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f493400d5f0 tx=0x7f493400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340125f0 con 0x7f494c0cfd40 2024-01-31T05:01:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934012750 con 0x7f494c0cfd40 2024-01-31T05:01:27.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c0c4090 con 0x7f494c0cfd40 2024-01-31T05:01:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.202+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934012b00 con 0x7f494c0cfd40 2024-01-31T05:01:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.203+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c0b4c20 con 0x7f494c0cfd40 2024-01-31T05:01:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.204+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c066fc0 con 0x7f494c0cfd40 2024-01-31T05:01:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.205+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4934012c60 con 0x7f494c0cfd40 2024-01-31T05:01:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.205+0000 7f4942a00640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f492406aed0 0x7f492406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.205+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4934089eb0 con 0x7f494c0cfd40 2024-01-31T05:01:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.206+0000 7f494a800640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f492406aed0 0x7f492406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.206+0000 7f494a800640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f492406aed0 0x7f492406d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f493c006b10 tx=0x7f493c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:27.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.210+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f49340549f0 con 0x7f494c0cfd40 2024-01-31T05:01:27.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.441+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f494c15d530 con 0x7f494c0cfd40 2024-01-31T05:01:27.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.442+0000 7f4942a00640 1 -- 172.21.15.97:0/2655941267 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4934058010 con 0x7f494c0cfd40 2024-01-31T05:01:27.441 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-31T05:01:27.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.444+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f492406aed0 msgr2=0x7f492406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:27.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.444+0000 7f4951400640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f492406aed0 0x7f492406d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f493c006b10 tx=0x7f493c008040 comp rx=0 tx=0).stop 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 msgr2=0x7f494c1f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 0x7f494c1f2460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f493400d5f0 tx=0x7f493400dab0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 shutdown_connections 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f494c15da20 0x7f494c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f492406aed0 0x7f492406d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f494c0cfd40 0x7f494c1f2460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 --2- 172.21.15.97:0/2655941267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f494c0af8a0 0x7f494c0c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 >> 172.21.15.97:0/2655941267 conn(0x7f494c071c70 msgr2=0x7f494c0ac720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 shutdown_connections 2024-01-31T05:01:27.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:27.445+0000 7f4951400640 1 -- 172.21.15.97:0/2655941267 wait complete. 2024-01-31T05:01:27.461 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:01:27.461 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:01:27.461 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:01:27.462 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-31T05:01:27.462 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:27.462 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:01:27.462 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:01:27.462 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:01:27.462 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-31T05:01:27.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.462 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:27.504 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42437sec preferred_lft 42437sec 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:01:27.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link tentative 2024-01-31T05:01:27.506 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:27.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.507 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:01:27.630 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:01:27.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.633 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:01:27.673 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:01:27.673 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:01:27.673 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:01:27.673 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:01:27.673 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:01:27.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.674 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:27.674 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:27.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:01:27.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:01:27.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:01:27.674 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:27.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.880 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:01:27.928 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:01:27.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.929 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-01-31T05:01:27.977 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-31T05:01:27.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:27.979 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:27.979 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:27.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:01:27.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-31T05:01:27.979 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:28.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:28.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-31T05:01:28.203 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:28.203 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:28.203 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:32:67:09:7b:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2c32:67ff:fe09:7b37/64 scope link 2024-01-31T05:01:28.204 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:28.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-31T05:01:28.301 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:28.301 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:28.301 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:28.301 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 2e:32:67:09:7b:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2c32:67ff:fe09:7b37/64 scope link 2024-01-31T05:01:28.302 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:28.306 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-31T05:01:28.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:28.306 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-31T05:01:28.307 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:28.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:28.570 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:01:28.570 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:01:28.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2024-01-31T05:01:28.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2024-01-31T05:01:28.571 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:01:28.736 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-01-31T05:01:28.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:28.736 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:28.783 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-31T05:01:28.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:28.783 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:28.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:01:28.920 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:01:28.972 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:28.973 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:01:28.974 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:01:28.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:28.975 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:01:29.025 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:01:29.025 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-01-31T05:01:29.025 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-01-31T05:01:29.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:01:29.103 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:01:29.143 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:29.144 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:01:29.145 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:01:29.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:29.146 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:01:29.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:2024-01-31T05:01:29.209+0000 7fbea3843480 -1 init, newargv = 0x55bffb409ac0 newargc=15 2024-01-31T05:01:29.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:ceph-fuse[47030]: starting ceph client 2024-01-31T05:01:29.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi097.stderr:ceph-fuse[47030]: starting fuse 2024-01-31T05:01:29.229 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:01:29.229 INFO:teuthology.orchestra.run.smithi097.stdout:42 2024-01-31T05:01:29.229 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:01:29.230 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.2")) 2024-01-31T05:01:29.231 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:01:29.425 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.2.47030.asok 2024-01-31T05:01:29.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:01:29.442 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.47030.asok 2024-01-31T05:01:29.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:29.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.47030.asok status 2024-01-31T05:01:29.680 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:01:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:01:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:01:29.680 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "2", 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "47030", 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4884, 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 4884 2024-01-31T05:01:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.2:0", 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3105116374 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.2:0", 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3105116374 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.4884 192.168.0.2:0/3105116374", 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.2:0/3105116374", 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:01:29.682 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 8, 2024-01-31T05:01:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 26, 2024-01-31T05:01:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:01:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:01:29.683 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:01:29.683 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:01:29.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:01:29.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.900+0000 7f25b0000640 1 Processor -- start 2024-01-31T05:01:29.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.900+0000 7f25b0000640 1 -- start start 2024-01-31T05:01:29.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.902+0000 7f25b0000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f25a806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.902+0000 7f25b0000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a815aef0 con 0x7f25a815a600 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.902+0000 7f25b0000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a806c8b0 con 0x7f25a80b1d10 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.902+0000 7f25b0000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a815b120 con 0x7f25a815aa60 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.902+0000 7f25acc00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f25a815a600 0x7f25a806d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:59662/0 (socket says 172.21.15.97:59662) 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.902+0000 7f25acc00640 1 -- 172.21.15.97:0/1032903572 learned_addr learned my addr 172.21.15.97:0/1032903572 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25ae000640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f25a806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066569280 0 0) 0x7f25a815aef0 con 0x7f25a815a600 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2588005730 con 0x7f25a815a600 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2650541717 0 0) 0x7f2588005730 con 0x7f25a815a600 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 msgr2=0x7f25a806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f25a806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 >> v1:172.21.15.188:6789/0 conn(0x7f25a80b1d10 legacy=0x7f25a80b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:29.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a815b400 con 0x7f25a815a600 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f2590001670 con 0x7f25a815a600 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25ae000640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f25a806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 >> v1:172.21.15.97:6789/0 conn(0x7f25a815a600 legacy=0x7f25a806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f2588005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25880061f0 0x7f25880069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2588006f10 0x7f2588007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a815b120 con 0x7f25880061f0 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258800dd50 con 0x7f2588006f10 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258800df10 con 0x7f25a815aa60 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.903+0000 7f25acc00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25880061f0 0x7f25880069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25ad600640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f2588005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25ae000640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2588006f10 0x7f2588007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25acc00640 1 -- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 msgr2=0x7f2588005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:29.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25acc00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a815aa60 0x7f2588005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:29.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25acc00640 1 -- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2588006f10 msgr2=0x7f2588007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:29.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25acc00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2588006f10 0x7f2588007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:29.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.904+0000 7f25acc00640 1 -- 172.21.15.97:0/1032903572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588005730 con 0x7f25880061f0 2024-01-31T05:01:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.905+0000 7f25acc00640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25880061f0 0x7f25880069d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f259000c0e0 tx=0x7f2588007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f45cda84295e14 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.905+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590005ad0 con 0x7f25880061f0 2024-01-31T05:01:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.905+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590005c30 con 0x7f25880061f0 2024-01-31T05:01:29.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.905+0000 7f25a7e00640 1 -- 172.21.15.97:0/1032903572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259002ca50 con 0x7f25880061f0 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25b0000640 1 -- 172.21.15.97:0/1032903572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25880061f0 msgr2=0x7f25880069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25b0000640 1 --2- 172.21.15.97:0/1032903572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25880061f0 0x7f25880069d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f259000c0e0 tx=0x7f2588007a00 comp rx=0 tx=0).stop 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25ae000640 1 -- 172.21.15.97:0/1032903572 reap_dead start 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25b0000640 1 -- 172.21.15.97:0/1032903572 shutdown_connections 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25b0000640 1 -- 172.21.15.97:0/1032903572 >> 172.21.15.97:0/1032903572 conn(0x7f25a80b0a10 msgr2=0x7f25a80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25b0000640 1 -- 172.21.15.97:0/1032903572 shutdown_connections 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.906+0000 7f25b0000640 1 -- 172.21.15.97:0/1032903572 wait complete. 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.908+0000 7f25b0000640 1 Processor -- start 2024-01-31T05:01:29.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.908+0000 7f25b0000640 1 -- start start 2024-01-31T05:01:29.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25b0000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25880061f0 0x7f25a815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25b0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 0x7f25a80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25b0000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a806cf90 0x7f25a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25b0000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a806d960 con 0x7f25a815d280 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25b0000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a815d6e0 con 0x7f25880061f0 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25b0000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a815bd20 con 0x7f25a806cf90 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 0x7f25a80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 0x7f25a80cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54666/0 (socket says 172.21.15.97:54666) 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 -- 172.21.15.97:0/2662727795 learned_addr learned my addr 172.21.15.97:0/2662727795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25ae000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a806cf90 0x7f25a806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25ad600640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25880061f0 0x7f25a815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 -- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a806cf90 msgr2=0x7f25a806d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a806cf90 0x7f25a806d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 -- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25880061f0 msgr2=0x7f25a815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25880061f0 0x7f25a815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.909+0000 7f25acc00640 1 -- 172.21.15.97:0/2662727795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80adb80 con 0x7f25a815d280 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.910+0000 7f25acc00640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 0x7f25a80cb810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f259000c0e0 tx=0x7f2590001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.910+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590039070 con 0x7f25a815d280 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.910+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259002cc20 con 0x7f25a815d280 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.910+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590007070 con 0x7f25a815d280 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.910+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a815c000 con 0x7f25a815d280 2024-01-31T05:01:29.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.910+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a815c5c0 con 0x7f25a815d280 2024-01-31T05:01:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.912+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a80677c0 con 0x7f25a815d280 2024-01-31T05:01:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.912+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2590032070 con 0x7f25a815d280 2024-01-31T05:01:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.913+0000 7f25a5600640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f258006af00 0x7f258006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.913+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f259003c040 con 0x7f25a815d280 2024-01-31T05:01:29.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.914+0000 7f25ad600640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f258006af00 0x7f258006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:29.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.914+0000 7f25ad600640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f258006af00 0x7f258006d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2598007fd0 tx=0x7f2598009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:29.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:29.918+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f25900756e0 con 0x7f25a815d280 2024-01-31T05:01:30.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.152+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f25a806e740 con 0x7f25a815d280 2024-01-31T05:01:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.153+0000 7f25a5600640 1 -- 172.21.15.97:0/2662727795 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2590078d00 con 0x7f25a815d280 2024-01-31T05:01:30.152 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f258006af00 msgr2=0x7f258006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f258006af00 0x7f258006d3c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2598007fd0 tx=0x7f2598009040 comp rx=0 tx=0).stop 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 msgr2=0x7f25a80cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 0x7f25a80cb810 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f259000c0e0 tx=0x7f2590001930 comp rx=0 tx=0).stop 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 shutdown_connections 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a806cf90 0x7f25a806d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f258006af00 0x7f258006d3c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25a815d280 0x7f25a80cb810 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 --2- 172.21.15.97:0/2662727795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25880061f0 0x7f25a815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 >> 172.21.15.97:0/2662727795 conn(0x7f25a80b0a10 msgr2=0x7f25a80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.155+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 shutdown_connections 2024-01-31T05:01:30.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:30.156+0000 7f25b0000640 1 -- 172.21.15.97:0/2662727795 wait complete. 2024-01-31T05:01:30.171 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:01:30.171 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:01:30.172 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-31T05:01:30.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:30.172 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-31T05:01:30.228 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:30.229 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:01:30.230 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:30.230 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:01:30.230 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42552sec preferred_lft 42552sec 2024-01-31T05:01:30.230 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-01-31T05:01:30.230 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:30.231 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-31T05:01:30.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:30.231 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:30.231 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:30.231 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-01-31T05:01:30.231 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-01-31T05:01:30.231 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-01-31T05:01:30.231 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-31T05:01:30.232 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:30.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:30.674 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:01:30.752 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-31T05:01:30.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:30.756 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-31T05:01:30.812 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-31T05:01:30.812 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:01:30.812 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:01:30.812 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:01:30.812 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:01:30.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:30.813 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:30.813 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:30.813 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:01:30.813 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:01:30.813 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:01:30.813 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:31.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:31.516 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-31T05:01:31.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:31.571 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-01-31T05:01:31.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:31.621 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:31.621 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:31.621 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:31.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-31T05:01:31.622 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:31.755 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-31T05:01:31.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-31T05:01:31.756 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:32.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:32.043 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:32.043 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:32.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-01-31T05:01:32.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-01-31T05:01:32.043 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:32.203 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-31T05:01:32.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:32.203 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:32.250 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-31T05:01:32.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:32.250 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:32.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-01-31T05:01:32.433 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:32.493 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/selinux 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2024-01-31T05:01:32.494 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2024-01-31T05:01:32.495 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2024-01-31T05:01:32.496 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2024-01-31T05:01:32.496 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:32.496 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:32.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:32.497 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-01-31T05:01:32.545 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:01:32.545 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-31T05:01:32.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-01-31T05:01:32.622 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/selinux 2024-01-31T05:01:32.661 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2024-01-31T05:01:32.662 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2024-01-31T05:01:32.663 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2024-01-31T05:01:32.663 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2024-01-31T05:01:32.663 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:32.663 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:32.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:32.664 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-01-31T05:01:32.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:2024-01-31T05:01:32.742+0000 7fbb84e40480 -1 init, newargv = 0x55795290ab20 newargc=15 2024-01-31T05:01:32.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[41945]: starting ceph client 2024-01-31T05:01:32.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi188.stderr:ceph-fuse[41945]: starting fuse 2024-01-31T05:01:33.850 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-01-31T05:01:33.951 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2024-01-31T05:01:34.005 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/selinux 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2024-01-31T05:01:34.006 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:34.007 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:34.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:34.008 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-01-31T05:01:34.060 INFO:teuthology.orchestra.run.smithi188.stdout:39 2024-01-31T05:01:34.061 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2024-01-31T05:01:34.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:01:34.061 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2024-01-31T05:01:34.061 DEBUG:teuthology.orchestra.run.smithi188:> import re 2024-01-31T05:01:34.061 DEBUG:teuthology.orchestra.run.smithi188:> import os 2024-01-31T05:01:34.061 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:01:34.062 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2024-01-31T05:01:34.063 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2024-01-31T05:01:34.063 DEBUG:teuthology.orchestra.run.smithi188:> return f 2024-01-31T05:01:34.063 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:01:34.063 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:34.063 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.1")) 2024-01-31T05:01:34.063 DEBUG:teuthology.orchestra.run.smithi188:> ' 2024-01-31T05:01:34.226 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.1.41945.asok 2024-01-31T05:01:34.230 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:01:34.243 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41945.asok 2024-01-31T05:01:34.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:34.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41945.asok status 2024-01-31T05:01:34.480 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-01-31T05:01:34.480 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2024-01-31T05:01:34.480 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:01:34.480 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "1", 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "41945", 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 4654, 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 4654 2024-01-31T05:01:34.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 705495899 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 705495899 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.4654 192.168.0.1:0/705495899", 2024-01-31T05:01:34.482 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.1:0/705495899", 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 8, 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 27, 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2024-01-31T05:01:34.483 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-01-31T05:01:34.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:01:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.714+0000 7fc99fc00640 1 Processor -- start 2024-01-31T05:01:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.714+0000 7fc99fc00640 1 -- start start 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9980cb810 0x7fc9980cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99fc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc99815b400 0x7fc99806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99fc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc99806db70 0x7fc9980ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99fc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9980cc960 con 0x7fc99815b400 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99fc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99806d1a0 con 0x7fc99806db70 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99806c600 con 0x7fc9980cb810 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc997e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc99815b400 0x7fc99806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:34.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc997e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc99815b400 0x7fc99806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54682/0 (socket says 172.21.15.97:54682) 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc997e00640 1 -- 172.21.15.97:0/308156014 learned_addr learned my addr 172.21.15.97:0/308156014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99d200640 1 --2- 172.21.15.97:0/308156014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9980cb810 0x7fc9980cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.716+0000 7fc99dc00640 1 --2- 172.21.15.97:0/308156014 >> v2:172.21.15.188:3300/0 conn(0x7fc99806db70 0x7fc9980ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.717+0000 7fc997e00640 1 -- 172.21.15.97:0/308156014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9980cb810 msgr2=0x7fc9980cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.717+0000 7fc997e00640 1 --2- 172.21.15.97:0/308156014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9980cb810 0x7fc9980cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.717+0000 7fc997e00640 1 -- 172.21.15.97:0/308156014 >> v2:172.21.15.188:3300/0 conn(0x7fc99806db70 msgr2=0x7fc9980ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.717+0000 7fc997e00640 1 --2- 172.21.15.97:0/308156014 >> v2:172.21.15.188:3300/0 conn(0x7fc99806db70 0x7fc9980ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.717+0000 7fc997e00640 1 -- 172.21.15.97:0/308156014 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9980cc130 con 0x7fc99815b400 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.718+0000 7fc997e00640 1 --2- 172.21.15.97:0/308156014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815b400 0x7fc99806d630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc99806c1c0 tx=0x7fc98002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2256bace4eaa1011 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.718+0000 7fc997400640 1 -- 172.21.15.97:0/308156014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980031870 con 0x7fc99815b400 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.718+0000 7fc997400640 1 -- 172.21.15.97:0/308156014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9800319d0 con 0x7fc99815b400 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.718+0000 7fc997400640 1 -- 172.21.15.97:0/308156014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980031d60 con 0x7fc99815b400 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.718+0000 7fc99fc00640 1 -- 172.21.15.97:0/308156014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815b400 msgr2=0x7fc99806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 --2- 172.21.15.97:0/308156014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815b400 0x7fc99806d630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc99806c1c0 tx=0x7fc98002b910 comp rx=0 tx=0).stop 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 -- 172.21.15.97:0/308156014 shutdown_connections 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 --2- 172.21.15.97:0/308156014 >> v2:172.21.15.188:3300/0 conn(0x7fc99806db70 0x7fc9980ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 --2- 172.21.15.97:0/308156014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815b400 0x7fc99806d630 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 --2- 172.21.15.97:0/308156014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9980cb810 0x7fc9980cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 -- 172.21.15.97:0/308156014 >> 172.21.15.97:0/308156014 conn(0x7fc9980b0a10 msgr2=0x7fc9980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 -- 172.21.15.97:0/308156014 shutdown_connections 2024-01-31T05:01:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.719+0000 7fc99fc00640 1 -- 172.21.15.97:0/308156014 wait complete. 2024-01-31T05:01:34.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.720+0000 7fc99fc00640 1 Processor -- start 2024-01-31T05:01:34.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.721+0000 7fc99fc00640 1 -- start start 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.721+0000 7fc99fc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 0x7fc998152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99fc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9980cb810 0x7fc9981f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815b400 0x7fc9980999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99fc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9981f70c0 con 0x7fc99806db70 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99fc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980b2320 con 0x7fc9980cb810 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9981f6b00 con 0x7fc99815b400 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 0x7fc998152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 0x7fc998152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54690/0 (socket says 172.21.15.97:54690) 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 -- 172.21.15.97:0/1162551189 learned_addr learned my addr 172.21.15.97:0/1162551189 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 -- 172.21.15.97:0/1162551189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815b400 msgr2=0x7fc9980999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815b400 0x7fc9980999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 -- 172.21.15.97:0/1162551189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9980cb810 msgr2=0x7fc9981f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:34.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9980cb810 0x7fc9981f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.722+0000 7fc99d200640 1 -- 172.21.15.97:0/1162551189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800094d0 con 0x7fc99806db70 2024-01-31T05:01:34.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.723+0000 7fc99d200640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 0x7fc998152a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c7f0 tx=0x7fc98800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.723+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988004220 con 0x7fc99806db70 2024-01-31T05:01:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.723+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9880043a0 con 0x7fc99806db70 2024-01-31T05:01:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.723+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9981f6de0 con 0x7fc99806db70 2024-01-31T05:01:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.723+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988007860 con 0x7fc99806db70 2024-01-31T05:01:34.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.723+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc998152f90 con 0x7fc99806db70 2024-01-31T05:01:34.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.724+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9980b1e10 con 0x7fc99806db70 2024-01-31T05:01:34.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.725+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc988007bd0 con 0x7fc99806db70 2024-01-31T05:01:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.726+0000 7fc994c00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006aed0 0x7fc97006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.726+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc9880899b0 con 0x7fc99806db70 2024-01-31T05:01:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.727+0000 7fc997e00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006aed0 0x7fc97006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:34.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.727+0000 7fc997e00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006aed0 0x7fc97006d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc980036000 tx=0x7fc980030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:34.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.731+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc9980b1e10 con 0x7fc99806db70 2024-01-31T05:01:34.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.965+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9980b1e10 con 0x7fc99806db70 2024-01-31T05:01:34.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.966+0000 7fc994c00640 1 -- 172.21.15.97:0/1162551189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc98801e070 con 0x7fc99806db70 2024-01-31T05:01:34.967 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-01-31T05:01:34.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006aed0 msgr2=0x7fc97006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:34.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006aed0 0x7fc97006d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc980036000 tx=0x7fc980030000 comp rx=0 tx=0).stop 2024-01-31T05:01:34.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 msgr2=0x7fc998152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:34.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 0x7fc998152a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc98800c7f0 tx=0x7fc98800ccb0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 shutdown_connections 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815b400 0x7fc9980999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9980cb810 0x7fc9981f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006aed0 0x7fc97006d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 --2- 172.21.15.97:0/1162551189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99806db70 0x7fc998152a50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 >> 172.21.15.97:0/1162551189 conn(0x7fc9980b0a10 msgr2=0x7fc9980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 shutdown_connections 2024-01-31T05:01:34.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:34.969+0000 7fc99fc00640 1 -- 172.21.15.97:0/1162551189 wait complete. 2024-01-31T05:01:34.987 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:01:34.988 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-31T05:01:34.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:34.988 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:35.045 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42547sec preferred_lft 42547sec 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 1a:19:a2:9d:64:6d brd ff:ff:ff:ff:ff:ff 2024-01-31T05:01:35.046 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ac1a:88ff:fe32:bec5/64 scope link 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 1a:19:a2:9d:64:6d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::1819:a2ff:fe9d:646d/64 scope link 2024-01-31T05:01:35.047 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.048 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:01:35.138 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-01-31T05:01:35.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.143 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-01-31T05:01:35.194 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-01-31T05:01:35.235 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:01:35.235 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:01:35.235 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:01:35.235 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:01:35.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.236 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:35.236 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:35.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:01:35.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:01:35.236 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:01:35.236 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:35.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.425 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-31T05:01:35.471 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-31T05:01:35.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.473 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-01-31T05:01:35.568 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-31T05:01:35.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.569 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:35.569 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:35.569 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-31T05:01:35.569 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-31T05:01:35.570 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:35.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.711 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-31T05:01:35.798 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:35.798 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:35.798 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:35.798 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether e2:aa:89:e1:88:df brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::e0aa:89ff:fee1:88df/64 scope link 2024-01-31T05:01:35.799 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.802 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether e2:aa:89:e1:88:df brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-31T05:01:35.895 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-31T05:01:35.896 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.896 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::e0aa:89ff:fee1:88df/64 scope link 2024-01-31T05:01:35.896 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:01:35.899 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-31T05:01:35.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-31T05:01:35.900 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:36.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:36.139 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2024-01-31T05:01:36.139 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-01-31T05:01:36.139 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2024-01-31T05:01:36.139 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2024-01-31T05:01:36.140 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-01-31T05:01:36.296 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-01-31T05:01:36.297 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:36.297 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:36.345 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-31T05:01:36.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:01:36.346 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:36.390 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-01-31T05:01:36.524 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:36.579 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/selinux 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:36.580 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2024-01-31T05:01:36.581 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2024-01-31T05:01:36.582 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2024-01-31T05:01:36.582 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:36.582 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:36.582 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:36.582 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-31T05:01:36.582 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-31T05:01:36.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:36.583 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-01-31T05:01:36.630 INFO:teuthology.orchestra.run.smithi188.stdout:39 2024-01-31T05:01:36.631 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2024-01-31T05:01:36.631 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-01-31T05:01:36.637 DEBUG:teuthology.orchestra.run.smithi188:> sudo modprobe fuse 2024-01-31T05:01:36.711 DEBUG:teuthology.orchestra.run.smithi188:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:01:36.752 INFO:teuthology.orchestra.run.smithi188.stdout:/proc 2024-01-31T05:01:36.752 INFO:teuthology.orchestra.run.smithi188.stdout:/sys 2024-01-31T05:01:36.752 INFO:teuthology.orchestra.run.smithi188.stdout:/dev 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/security 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/shm 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/pts 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/run 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/cgroup 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/pstore 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/ 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/selinux 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/hugepages 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/dev/mqueue 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/debug 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/tracing 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/fs/fuse/connections 2024-01-31T05:01:36.753 INFO:teuthology.orchestra.run.smithi188.stdout:/sys/kernel/config 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/user/1000 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-4 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-5 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-6 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/var/lib/ceph/osd/ceph-7 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:36.754 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:01:36.755 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:36.755 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-31T05:01:36.755 INFO:teuthology.orchestra.run.smithi188.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-31T05:01:36.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:36.756 DEBUG:teuthology.orchestra.run.smithi188:> ls /sys/fs/fuse/connections 2024-01-31T05:01:36.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:2024-01-31T05:01:36.818+0000 7f76c3f87480 -1 init, newargv = 0x55bfc0dd2b40 newargc=15 2024-01-31T05:01:36.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:ceph-fuse[42465]: starting ceph client 2024-01-31T05:01:36.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi188.stderr:ceph-fuse[42465]: starting fuse 2024-01-31T05:01:36.841 INFO:teuthology.orchestra.run.smithi188.stdout:39 2024-01-31T05:01:36.841 INFO:teuthology.orchestra.run.smithi188.stdout:42 2024-01-31T05:01:36.842 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 42] 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> import glob 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> import re 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> import os 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> import subprocess 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> def _find_admin_socket(client_name): 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-01-31T05:01:36.842 DEBUG:teuthology.orchestra.run.smithi188:> files = glob.glob(asok_path) 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> # Given a non-glob path, it better be there 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> if "*" not in asok_path: 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> assert(len(files) == 1) 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> return files[0] 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> for f in files: 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> contents = proc_f.read() 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> if mountpoint in contents: 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> return f 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:01:36.843 DEBUG:teuthology.orchestra.run.smithi188:> 2024-01-31T05:01:36.844 DEBUG:teuthology.orchestra.run.smithi188:> print(_find_admin_socket("client.3")) 2024-01-31T05:01:36.844 DEBUG:teuthology.orchestra.run.smithi188:> ' 2024-01-31T05:01:37.037 INFO:teuthology.orchestra.run.smithi188.stdout:/var/run/ceph/ceph-client.3.42465.asok 2024-01-31T05:01:37.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-01-31T05:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:01:37.054 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42465.asok 2024-01-31T05:01:37.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.054 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42465.asok status 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "metadata": { 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "entity_id": "3", 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "hostname": "smithi188", 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "pid": "42465", 2024-01-31T05:01:37.285 INFO:teuthology.orchestra.run.smithi188.stdout: "root": "/" 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_count": 0, 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "dentry_pinned_count": 0, 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "id": 4666, 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "inst": { 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "name": { 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "client", 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "num": 4666 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.2:0", 2024-01-31T05:01:37.286 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 3247746593 2024-01-31T05:01:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-01-31T05:01:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": { 2024-01-31T05:01:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "type": "v1", 2024-01-31T05:01:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "addr": "192.168.0.2:0", 2024-01-31T05:01:37.287 INFO:teuthology.orchestra.run.smithi188.stdout: "nonce": 3247746593 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "inst_str": "client.4666 192.168.0.2:0/3247746593", 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "addr_str": "192.168.0.2:0/3247746593", 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "inode_count": 1, 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_epoch": 8, 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch": 28, 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "blocklisted": false, 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout: "fs_name": "cephfs" 2024-01-31T05:01:37.288 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-01-31T05:01:37.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.299 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:01:37.300 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:37.356 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:01:37.356 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:37.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.356 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:37.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.448 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:01:37.448 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:37.503 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:01:37.503 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:37.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.503 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-31T05:01:37.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.590 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2024-01-31T05:01:37.590 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:37.649 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2024-01-31T05:01:37.649 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:37.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.650 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-31T05:01:37.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.745 DEBUG:teuthology.orchestra.run.smithi188:> stat --file-system '--printf=%T 2024-01-31T05:01:37.745 DEBUG:teuthology.orchestra.run.smithi188:> ' -- /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:37.796 INFO:teuthology.orchestra.run.smithi188.stdout:fuseblk 2024-01-31T05:01:37.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:37.796 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-31T05:01:37.878 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-31T05:01:38.031 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.250+0000 7f9826e00640 1 Processor -- start 2024-01-31T05:01:38.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.251+0000 7f9826e00640 1 -- start start 2024-01-31T05:01:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.252+0000 7f9826e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f98200cb810 0x7f98200cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.252+0000 7f9826e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982006d1a0 con 0x7f98200ccd00 2024-01-31T05:01:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.252+0000 7f9826e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982006c600 con 0x7f982015b400 2024-01-31T05:01:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.252+0000 7f9826e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98200cc960 con 0x7f98200cb810 2024-01-31T05:01:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.252+0000 7f981fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f98200cb810 0x7f98200cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f981fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f98200cb810 0x7f98200cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43650/0 (socket says 172.21.15.97:43650) 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f981fe00640 1 -- 172.21.15.97:0/1674841755 learned_addr learned my addr 172.21.15.97:0/1674841755 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f9824e00640 1 --1- 172.21.15.97:0/1674841755 >> v1:172.21.15.97:6789/0 conn(0x7f98200ccd00 0x7f98200cd170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54904/0 (socket says 172.21.15.97:54904) 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f981fe00640 1 -- 172.21.15.97:0/1674841755 >> v1:172.21.15.188:6789/0 conn(0x7f982015b400 legacy=0x7f982006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f981fe00640 1 -- 172.21.15.97:0/1674841755 >> v1:172.21.15.97:6789/0 conn(0x7f98200ccd00 legacy=0x7f98200cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f981fe00640 1 -- 172.21.15.97:0/1674841755 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98200cc130 con 0x7f98200cb810 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.253+0000 7f981ea00640 1 -- 172.21.15.97:0/1674841755 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817128410 0 0) 0x7f982006d1a0 con 0x7f98200ccd00 2024-01-31T05:01:38.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.254+0000 7f981fe00640 1 --2- 172.21.15.97:0/1674841755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200cb810 0x7f98200cbbf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9810009870 tx=0x7f981002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ede4de88e3c400 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:38.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.254+0000 7f981ea00640 1 -- 172.21.15.97:0/1674841755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810030860 con 0x7f98200cb810 2024-01-31T05:01:38.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.254+0000 7f981ea00640 1 -- 172.21.15.97:0/1674841755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98100309c0 con 0x7f98200cb810 2024-01-31T05:01:38.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.254+0000 7f981ea00640 1 -- 172.21.15.97:0/1674841755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9810030d50 con 0x7f98200cb810 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 -- 172.21.15.97:0/1674841755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200cb810 msgr2=0x7f98200cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 --2- 172.21.15.97:0/1674841755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200cb810 0x7f98200cbbf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9810009870 tx=0x7f981002b9a0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 -- 172.21.15.97:0/1674841755 shutdown_connections 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 --2- 172.21.15.97:0/1674841755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200cb810 0x7f98200cbbf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 -- 172.21.15.97:0/1674841755 >> 172.21.15.97:0/1674841755 conn(0x7f98200b0a10 msgr2=0x7f98200b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 -- 172.21.15.97:0/1674841755 shutdown_connections 2024-01-31T05:01:38.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.255+0000 7f9826e00640 1 -- 172.21.15.97:0/1674841755 wait complete. 2024-01-31T05:01:38.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.257+0000 7f9826e00640 1 Processor -- start 2024-01-31T05:01:38.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.257+0000 7f9826e00640 1 -- start start 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f9826e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98200cb810 0x7f98200b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f9826e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 0x7f98200cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f9826e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f982015b400 0x7f98201523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f9826e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98200c8230 con 0x7f98200ccd00 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f9826e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98200b2510 con 0x7f982015b400 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f9826e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98200b27f0 con 0x7f98200cb810 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f981f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 0x7f98200cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f981f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 0x7f98200cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43664/0 (socket says 172.21.15.97:43664) 2024-01-31T05:01:38.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.258+0000 7f981f400640 1 -- 172.21.15.97:0/702522852 learned_addr learned my addr 172.21.15.97:0/702522852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981fe00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98200cb810 0x7f98200b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f9824e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f982015b400 0x7f98201523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981f400640 1 -- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98200cb810 msgr2=0x7f98200b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981f400640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98200cb810 0x7f98200b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981f400640 1 -- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f982015b400 msgr2=0x7f98201523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981f400640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f982015b400 0x7f98201523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981f400640 1 -- 172.21.15.97:0/702522852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98100094d0 con 0x7f98200ccd00 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f981f400640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 0x7f98200cf100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f980800b3c0 tx=0x7f980800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808011830 con 0x7f98200ccd00 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.259+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9808011990 con 0x7f98200ccd00 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.260+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98200b2ad0 con 0x7f98200ccd00 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.260+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9808011d80 con 0x7f98200ccd00 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.260+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982015d590 con 0x7f98200ccd00 2024-01-31T05:01:38.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.261+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98200677c0 con 0x7f98200ccd00 2024-01-31T05:01:38.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.262+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f980801b460 con 0x7f98200ccd00 2024-01-31T05:01:38.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.262+0000 7f9817e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97f806aed0 0x7f97f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.263+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f98080895c0 con 0x7f98200ccd00 2024-01-31T05:01:38.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.263+0000 7f981fe00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97f806aed0 0x7f97f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.263+0000 7f981fe00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97f806aed0 0x7f97f806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9810036000 tx=0x7f9810008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:38.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.267+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f98080540a0 con 0x7f98200ccd00 2024-01-31T05:01:38.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.460+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9820068100 con 0x7f98200ccd00 2024-01-31T05:01:38.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.462+0000 7f9817e00640 1 -- 172.21.15.97:0/702522852 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f98080576c0 con 0x7f98200ccd00 2024-01-31T05:01:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:38.463 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97f806aed0 msgr2=0x7f97f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97f806aed0 0x7f97f806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9810036000 tx=0x7f9810008030 comp rx=0 tx=0).stop 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 msgr2=0x7f98200cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 0x7f98200cf100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f980800b3c0 tx=0x7f980800b880 comp rx=0 tx=0).stop 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 shutdown_connections 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f982015b400 0x7f98201523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97f806aed0 0x7f97f806d390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98200ccd00 0x7f98200cf100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 --2- 172.21.15.97:0/702522852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98200cb810 0x7f98200b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 >> 172.21.15.97:0/702522852 conn(0x7f98200b0a10 msgr2=0x7f98200cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:38.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 shutdown_connections 2024-01-31T05:01:38.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.465+0000 7f9826e00640 1 -- 172.21.15.97:0/702522852 wait complete. 2024-01-31T05:01:38.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:38.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:38.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.697+0000 7f2d03a00640 1 Processor -- start 2024-01-31T05:01:38.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.697+0000 7f2d03a00640 1 -- start start 2024-01-31T05:01:38.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.698+0000 7f2d03a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2cfc0cbf20 0x7f2cfc0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.698+0000 7f2d03a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2cfc0abcf0 0x7f2cfc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.698+0000 7f2d03a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cfc06cd20 con 0x7f2cfc15b470 2024-01-31T05:01:38.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.698+0000 7f2d03a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc0ce970 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.698+0000 7f2d03a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc15b850 con 0x7f2cfc0cbf20 2024-01-31T05:01:38.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2cfc15b470 0x7f2cfc0cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54918/0 (socket says 172.21.15.97:54918) 2024-01-31T05:01:38.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01000640 1 -- 172.21.15.97:0/3288680014 learned_addr learned my addr 172.21.15.97:0/3288680014 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:38.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01a00640 1 --2- 172.21.15.97:0/3288680014 >> v2:172.21.15.97:3300/0 conn(0x7f2cfc0abcf0 0x7f2cfc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2cfb400640 1 -- 172.21.15.97:0/3288680014 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166320305 0 0) 0x7f2cfc06cd20 con 0x7f2cfc15b470 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2cfb400640 1 -- 172.21.15.97:0/3288680014 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cdc003520 con 0x7f2cfc15b470 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2cfbe00640 1 --2- 172.21.15.97:0/3288680014 >> v2:172.21.15.188:3300/0 conn(0x7f2cfc0cbf20 0x7f2cfc0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01a00640 1 -- 172.21.15.97:0/3288680014 >> v2:172.21.15.188:3300/0 conn(0x7f2cfc0cbf20 msgr2=0x7f2cfc0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01a00640 1 --2- 172.21.15.97:0/3288680014 >> v2:172.21.15.188:3300/0 conn(0x7f2cfc0cbf20 0x7f2cfc0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01a00640 1 -- 172.21.15.97:0/3288680014 >> v1:172.21.15.97:6789/0 conn(0x7f2cfc15b470 legacy=0x7f2cfc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2d01a00640 1 -- 172.21.15.97:0/3288680014 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0ceb60 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.699+0000 7f2cfbe00640 1 --2- 172.21.15.97:0/3288680014 >> v2:172.21.15.188:3300/0 conn(0x7f2cfc0cbf20 0x7f2cfc0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:38.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.700+0000 7f2d01a00640 1 --2- 172.21.15.97:0/3288680014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc0ce430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc072d90 tx=0x7f2cf402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4c9dc7d6e55152 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:38.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2cfb400640 1 -- 172.21.15.97:0/3288680014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf402cc60 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2cfb400640 1 -- 172.21.15.97:0/3288680014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf402cdc0 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2cfb400640 1 -- 172.21.15.97:0/3288680014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4034660 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 -- 172.21.15.97:0/3288680014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 msgr2=0x7f2cfc0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 --2- 172.21.15.97:0/3288680014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc0ce430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc072d90 tx=0x7f2cf402beb0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 -- 172.21.15.97:0/3288680014 shutdown_connections 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 --2- 172.21.15.97:0/3288680014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc0ce430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 --2- 172.21.15.97:0/3288680014 >> v2:172.21.15.188:3300/0 conn(0x7f2cfc0cbf20 0x7f2cfc0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 -- 172.21.15.97:0/3288680014 >> 172.21.15.97:0/3288680014 conn(0x7f2cfc0b0a10 msgr2=0x7f2cfc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.701+0000 7f2d03a00640 1 -- 172.21.15.97:0/3288680014 shutdown_connections 2024-01-31T05:01:38.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.702+0000 7f2d03a00640 1 -- 172.21.15.97:0/3288680014 wait complete. 2024-01-31T05:01:38.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.703+0000 7f2d03a00640 1 Processor -- start 2024-01-31T05:01:38.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.703+0000 7f2d03a00640 1 -- start start 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.704+0000 7f2d03a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.704+0000 7f2d03a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cfc0cbf20 0x7f2cfc153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.704+0000 7f2d03a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cfc15b470 0x7f2cfc145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.704+0000 7f2d03a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc09c160 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.704+0000 7f2d03a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc154100 con 0x7f2cfc0cbf20 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.704+0000 7f2d03a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0ac640 con 0x7f2cfc15b470 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43686/0 (socket says 172.21.15.97:43686) 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 -- 172.21.15.97:0/2712653770 learned_addr learned my addr 172.21.15.97:0/2712653770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2cfbe00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cfc0cbf20 0x7f2cfc153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cfc15b470 0x7f2cfc145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 -- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cfc15b470 msgr2=0x7f2cfc145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cfc15b470 0x7f2cfc145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 -- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cfc0cbf20 msgr2=0x7f2cfc153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cfc0cbf20 0x7f2cfc153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01000640 1 -- 172.21.15.97:0/2712653770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4009a70 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.705+0000 7f2d01a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cfc15b470 0x7f2cfc145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:38.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.706+0000 7f2d01000640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc143410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c430 tx=0x7f2cec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.706+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec0074e0 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.706+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec0057a0 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.706+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007660 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.706+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc143bf0 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.706+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc1441b0 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.707+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc143a50 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.708+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2cec005b50 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.709+0000 7f2cf8c00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cd406aed0 0x7f2cd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:38.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.709+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2cec088ff0 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.709+0000 7f2cfbe00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cd406aed0 0x7f2cd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:38.709 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.710+0000 7f2cfbe00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cd406aed0 0x7f2cd406d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0cbec0 tx=0x7f2ce4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:38.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.713+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2cfc143a50 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.906+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cfc143a50 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.908+0000 7f2cf8c00640 1 -- 172.21.15.97:0/2712653770 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cfc143a50 con 0x7f2cfc0abcf0 2024-01-31T05:01:38.907 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cd406aed0 msgr2=0x7f2cd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cd406aed0 0x7f2cd406d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0cbec0 tx=0x7f2ce4008040 comp rx=0 tx=0).stop 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 msgr2=0x7f2cfc143410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc143410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c430 tx=0x7f2cec00c8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 shutdown_connections 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cfc15b470 0x7f2cfc145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cfc0cbf20 0x7f2cfc153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cd406aed0 0x7f2cd406d390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 --2- 172.21.15.97:0/2712653770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2cfc0abcf0 0x7f2cfc143410 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 >> 172.21.15.97:0/2712653770 conn(0x7f2cfc0b0a10 msgr2=0x7f2cfc15d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 shutdown_connections 2024-01-31T05:01:38.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:38.911+0000 7f2d03a00640 1 -- 172.21.15.97:0/2712653770 wait complete. 2024-01-31T05:01:38.928 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:39.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.140+0000 7f747d800640 1 Processor -- start 2024-01-31T05:01:39.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.140+0000 7f747d800640 1 -- start start 2024-01-31T05:01:39.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f747d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f747815b470 0x7f74780722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f747d800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74780cbf20 0x7f74780ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f747d800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747806cd20 con 0x7f74780abcf0 2024-01-31T05:01:39.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f747d800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74780ce970 con 0x7f74780cbf20 2024-01-31T05:01:39.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f747d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747815b850 con 0x7f747815b470 2024-01-31T05:01:39.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f7477400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f74780cbf20 0x7f74780ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f746de00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f74780abcf0 0x7f74780cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54932/0 (socket says 172.21.15.97:54932) 2024-01-31T05:01:39.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f746de00640 1 -- 172.21.15.97:0/578507759 learned_addr learned my addr 172.21.15.97:0/578507759 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:39.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f7476a00640 1 --2- 172.21.15.97:0/578507759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f747815b470 0x7f74780722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f7477400640 1 -- 172.21.15.97:0/578507759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f747815b470 msgr2=0x7f74780722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:39.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f7477400640 1 --2- 172.21.15.97:0/578507759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f747815b470 0x7f74780722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f7477400640 1 -- 172.21.15.97:0/578507759 >> v1:172.21.15.97:6789/0 conn(0x7f74780abcf0 legacy=0x7f74780cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.143+0000 7f7477400640 1 -- 172.21.15.97:0/578507759 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780ceb60 con 0x7f74780cbf20 2024-01-31T05:01:39.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.142+0000 7f7476000640 1 -- 172.21.15.97:0/578507759 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686324097 0 0) 0x7f747806cd20 con 0x7f74780abcf0 2024-01-31T05:01:39.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.144+0000 7f7477400640 1 --2- 172.21.15.97:0/578507759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74780cbf20 0x7f74780ce430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7470009800 tx=0x7f747002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d443f53625c5ce server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:39.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.144+0000 7f7476000640 1 -- 172.21.15.97:0/578507759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470031800 con 0x7f74780cbf20 2024-01-31T05:01:39.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.144+0000 7f7476000640 1 -- 172.21.15.97:0/578507759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7470031960 con 0x7f74780cbf20 2024-01-31T05:01:39.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.144+0000 7f7476000640 1 -- 172.21.15.97:0/578507759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470031cf0 con 0x7f74780cbf20 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.144+0000 7f747d800640 1 -- 172.21.15.97:0/578507759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74780cbf20 msgr2=0x7f74780ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.144+0000 7f747d800640 1 --2- 172.21.15.97:0/578507759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74780cbf20 0x7f74780ce430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7470009800 tx=0x7f747002b8a0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.145+0000 7f747d800640 1 -- 172.21.15.97:0/578507759 shutdown_connections 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.145+0000 7f747d800640 1 --2- 172.21.15.97:0/578507759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74780cbf20 0x7f74780ce430 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.145+0000 7f747d800640 1 --2- 172.21.15.97:0/578507759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f747815b470 0x7f74780722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.145+0000 7f747d800640 1 -- 172.21.15.97:0/578507759 >> 172.21.15.97:0/578507759 conn(0x7f74780b0a10 msgr2=0x7f74780b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.145+0000 7f747d800640 1 -- 172.21.15.97:0/578507759 shutdown_connections 2024-01-31T05:01:39.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.145+0000 7f747d800640 1 -- 172.21.15.97:0/578507759 wait complete. 2024-01-31T05:01:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.146+0000 7f747d800640 1 Processor -- start 2024-01-31T05:01:39.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.147+0000 7f747d800640 1 -- start start 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.147+0000 7f747d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74780abcf0 0x7f747814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f747d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74780cbf20 0x7f74781f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f747d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 0x7f747809bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f747d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74780c4270 con 0x7f747815b470 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f747d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74780c3c80 con 0x7f74780cbf20 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f747d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74781f6cc0 con 0x7f74780abcf0 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 0x7f747809bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 0x7f747809bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43700/0 (socket says 172.21.15.97:43700) 2024-01-31T05:01:39.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 -- 172.21.15.97:0/2882533766 learned_addr learned my addr 172.21.15.97:0/2882533766 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7476a00640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74780abcf0 0x7f747814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f746de00640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74780cbf20 0x7f74781f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 -- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74780abcf0 msgr2=0x7f747814f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74780abcf0 0x7f747814f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 -- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74780cbf20 msgr2=0x7f74781f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74780cbf20 0x7f74781f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f7477400640 1 -- 172.21.15.97:0/2882533766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470009400 con 0x7f747815b470 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.148+0000 7f746de00640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74780cbf20 0x7f74781f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.149+0000 7f7477400640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 0x7f747809bb90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7470009920 tx=0x7f74700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.149+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747003bcb0 con 0x7f747815b470 2024-01-31T05:01:39.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.149+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747003be10 con 0x7f747815b470 2024-01-31T05:01:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.149+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74781f6fa0 con 0x7f747815b470 2024-01-31T05:01:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.149+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470045630 con 0x7f747815b470 2024-01-31T05:01:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.149+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747814f750 con 0x7f747815b470 2024-01-31T05:01:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.150+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74780c43b0 con 0x7f747815b470 2024-01-31T05:01:39.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.151+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7470045790 con 0x7f747815b470 2024-01-31T05:01:39.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.152+0000 7f746f400640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f745006aed0 0x7f745006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.153+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f747002e070 con 0x7f747815b470 2024-01-31T05:01:39.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.153+0000 7f7476a00640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f745006aed0 0x7f745006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.153+0000 7f7476a00640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f745006aed0 0x7f745006d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f746800b9f0 tx=0x7f7468008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:39.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.157+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f74780c43b0 con 0x7f747815b470 2024-01-31T05:01:39.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.350+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74780c43b0 con 0x7f747815b470 2024-01-31T05:01:39.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.351+0000 7f746f400640 1 -- 172.21.15.97:0/2882533766 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74780c43b0 con 0x7f747815b470 2024-01-31T05:01:39.350 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.353+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f745006aed0 msgr2=0x7f745006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.353+0000 7f747d800640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f745006aed0 0x7f745006d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f746800b9f0 tx=0x7f7468008040 comp rx=0 tx=0).stop 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.353+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 msgr2=0x7f747809bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.353+0000 7f747d800640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 0x7f747809bb90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7470009920 tx=0x7f74700046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 shutdown_connections 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f745006aed0 0x7f745006d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f747815b470 0x7f747809bb90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74780cbf20 0x7f74781f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 --2- 172.21.15.97:0/2882533766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74780abcf0 0x7f747814f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 >> 172.21.15.97:0/2882533766 conn(0x7f74780b0a10 msgr2=0x7f74780ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 shutdown_connections 2024-01-31T05:01:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.354+0000 7f747d800640 1 -- 172.21.15.97:0/2882533766 wait complete. 2024-01-31T05:01:39.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:39.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.577+0000 7fa6e0000640 1 Processor -- start 2024-01-31T05:01:39.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.577+0000 7fa6e0000640 1 -- start start 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6e0000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6d815d3e0 0x7fa6d80cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6e0000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6d81610b0 0x7fa6d80ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6e0000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d80ce890 0x7fa6d80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6e0000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d81620c0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6e0000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d8161820 con 0x7fa6d81610b0 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6e0000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8161b00 con 0x7fa6d80ce890 2024-01-31T05:01:39.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6dd600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6d815d3e0 0x7fa6d80cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6dd600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6d815d3e0 0x7fa6d80cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43706/0 (socket says 172.21.15.97:43706) 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6dd600640 1 -- 172.21.15.97:0/3277462592 learned_addr learned my addr 172.21.15.97:0/3277462592 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6de000640 1 --2- 172.21.15.97:0/3277462592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d80ce890 0x7fa6d80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.579+0000 7fa6dcc00640 1 --2- 172.21.15.97:0/3277462592 >> v2:172.21.15.188:3300/0 conn(0x7fa6d81610b0 0x7fa6d80ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.580+0000 7fa6dd600640 1 -- 172.21.15.97:0/3277462592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d80ce890 msgr2=0x7fa6d80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.580+0000 7fa6dd600640 1 --2- 172.21.15.97:0/3277462592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d80ce890 0x7fa6d80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.580+0000 7fa6dd600640 1 -- 172.21.15.97:0/3277462592 >> v2:172.21.15.188:3300/0 conn(0x7fa6d81610b0 msgr2=0x7fa6d80ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.580+0000 7fa6dd600640 1 --2- 172.21.15.97:0/3277462592 >> v2:172.21.15.188:3300/0 conn(0x7fa6d81610b0 0x7fa6d80ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.580+0000 7fa6dd600640 1 -- 172.21.15.97:0/3277462592 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8161de0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.581+0000 7fa6dd600640 1 --2- 172.21.15.97:0/3277462592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d80cbb30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80cd200 tx=0x7fa6d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d599a4375d8da6c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:39.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.581+0000 7fa6d7e00640 1 -- 172.21.15.97:0/3277462592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d002ca70 con 0x7fa6d815d3e0 2024-01-31T05:01:39.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.581+0000 7fa6d7e00640 1 -- 172.21.15.97:0/3277462592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d002cbd0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.581+0000 7fa6d7e00640 1 -- 172.21.15.97:0/3277462592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00343f0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 -- 172.21.15.97:0/3277462592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 msgr2=0x7fa6d80cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 --2- 172.21.15.97:0/3277462592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d80cbb30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80cd200 tx=0x7fa6d002bed0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 -- 172.21.15.97:0/3277462592 shutdown_connections 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 --2- 172.21.15.97:0/3277462592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d80ce890 0x7fa6d80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 --2- 172.21.15.97:0/3277462592 >> v2:172.21.15.188:3300/0 conn(0x7fa6d81610b0 0x7fa6d80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 --2- 172.21.15.97:0/3277462592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d80cbb30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 -- 172.21.15.97:0/3277462592 >> 172.21.15.97:0/3277462592 conn(0x7fa6d80cf6e0 msgr2=0x7fa6d80cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 -- 172.21.15.97:0/3277462592 shutdown_connections 2024-01-31T05:01:39.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.582+0000 7fa6e0000640 1 -- 172.21.15.97:0/3277462592 wait complete. 2024-01-31T05:01:39.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.584+0000 7fa6e0000640 1 Processor -- start 2024-01-31T05:01:39.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.584+0000 7fa6e0000640 1 -- start start 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6e0000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6d80ce890 0x7fa6d8160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6e0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d81535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6e0000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d81610b0 0x7fa6d8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6e0000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d815f5d0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6e0000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d815f8b0 con 0x7fa6d80ce890 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6e0000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d815fb90 con 0x7fa6d81610b0 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d81535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d81535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43718/0 (socket says 172.21.15.97:43718) 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6dcc00640 1 -- 172.21.15.97:0/1119035427 learned_addr learned my addr 172.21.15.97:0/1119035427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6dd600640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6d80ce890 0x7fa6d8160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.585+0000 7fa6de000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d81610b0 0x7fa6d8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dcc00640 1 -- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d81610b0 msgr2=0x7fa6d8159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dcc00640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d81610b0 0x7fa6d8159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dcc00640 1 -- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6d80ce890 msgr2=0x7fa6d8160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dcc00640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6d80ce890 0x7fa6d8160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dcc00640 1 -- 172.21.15.97:0/1119035427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0009b30 con 0x7fa6d815d3e0 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dd600640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6d80ce890 0x7fa6d8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6de000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d81610b0 0x7fa6d8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6dcc00640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d81535d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000d800 tx=0x7fa6c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000b6c0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c000b820 con 0x7fa6d815d3e0 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c000bbb0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d815fe70 con 0x7fa6d815d3e0 2024-01-31T05:01:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.586+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d8151a10 con 0x7fa6d815d3e0 2024-01-31T05:01:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.588+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d80738a0 con 0x7fa6d815d3e0 2024-01-31T05:01:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.588+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa6c000bd10 con 0x7fa6d815d3e0 2024-01-31T05:01:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.589+0000 7fa6d5600640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa6b006aed0 0x7fa6b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.589+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa6c0089e20 con 0x7fa6d815d3e0 2024-01-31T05:01:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.589+0000 7fa6dd600640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa6b006aed0 0x7fa6b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:39.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.590+0000 7fa6dd600640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa6b006aed0 0x7fa6b006d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0035000 tx=0x7fa6d0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.594+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa6c0054a30 con 0x7fa6d815d3e0 2024-01-31T05:01:39.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.786+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6d8067830 con 0x7fa6d815d3e0 2024-01-31T05:01:39.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.787+0000 7fa6d5600640 1 -- 172.21.15.97:0/1119035427 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa6c0058050 con 0x7fa6d815d3e0 2024-01-31T05:01:39.786 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:39.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.789+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa6b006aed0 msgr2=0x7fa6b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.789+0000 7fa6e0000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa6b006aed0 0x7fa6b006d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0035000 tx=0x7fa6d0030000 comp rx=0 tx=0).stop 2024-01-31T05:01:39.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.789+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 msgr2=0x7fa6d81535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.789+0000 7fa6e0000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d81535d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000d800 tx=0x7fa6c000dcc0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 shutdown_connections 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6d81610b0 0x7fa6d8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa6b006aed0 0x7fa6b006d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d815d3e0 0x7fa6d81535d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 --2- 172.21.15.97:0/1119035427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6d80ce890 0x7fa6d8160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 >> 172.21.15.97:0/1119035427 conn(0x7fa6d80cf6e0 msgr2=0x7fa6d80cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 shutdown_connections 2024-01-31T05:01:39.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:39.790+0000 7fa6e0000640 1 -- 172.21.15.97:0/1119035427 wait complete. 2024-01-31T05:01:39.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.018+0000 7f1386e00640 1 Processor -- start 2024-01-31T05:01:40.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.019+0000 7f1386e00640 1 -- start start 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f1386e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f138015d590 0x7f138015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f1386e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13800cfae0 0x7f13800cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f1386e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1380161080 0x7f1380161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f1386e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13800d0460 con 0x7f138015d590 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f1386e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13800cded0 con 0x7f1380161080 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f1386e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138015d970 con 0x7f13800cfae0 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f137fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f138015d590 0x7f138015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f137fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f138015d590 0x7f138015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43734/0 (socket says 172.21.15.97:43734) 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.020+0000 7f137fe00640 1 -- 172.21.15.97:0/2629428217 learned_addr learned my addr 172.21.15.97:0/2629428217 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f137f400640 1 --2- 172.21.15.97:0/2629428217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13800cfae0 0x7f13800cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f137fe00640 1 -- 172.21.15.97:0/2629428217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13800cfae0 msgr2=0x7f13800cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f137fe00640 1 --2- 172.21.15.97:0/2629428217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13800cfae0 0x7f13800cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f137fe00640 1 -- 172.21.15.97:0/2629428217 >> v2:172.21.15.188:3300/0 conn(0x7f1380161080 msgr2=0x7f1380161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f1384e00640 1 --2- 172.21.15.97:0/2629428217 >> v2:172.21.15.188:3300/0 conn(0x7f1380161080 0x7f1380161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f137fe00640 1 --2- 172.21.15.97:0/2629428217 >> v2:172.21.15.188:3300/0 conn(0x7f1380161080 0x7f1380161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.021+0000 7f137fe00640 1 -- 172.21.15.97:0/2629428217 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800cf210 con 0x7f138015d590 2024-01-31T05:01:40.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.022+0000 7f137fe00640 1 --2- 172.21.15.97:0/2629428217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f138015d590 0x7f138015e540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f13800b4cd0 tx=0x7f136802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dba3b6a2440c46 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.023+0000 7f137ea00640 1 -- 172.21.15.97:0/2629428217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368030860 con 0x7f138015d590 2024-01-31T05:01:40.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.023+0000 7f137ea00640 1 -- 172.21.15.97:0/2629428217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13680309c0 con 0x7f138015d590 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.023+0000 7f137ea00640 1 -- 172.21.15.97:0/2629428217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368030d50 con 0x7f138015d590 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.023+0000 7f1386e00640 1 -- 172.21.15.97:0/2629428217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f138015d590 msgr2=0x7f138015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.023+0000 7f1386e00640 1 --2- 172.21.15.97:0/2629428217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f138015d590 0x7f138015e540 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f13800b4cd0 tx=0x7f136802b9a0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 -- 172.21.15.97:0/2629428217 shutdown_connections 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 --2- 172.21.15.97:0/2629428217 >> v2:172.21.15.188:3300/0 conn(0x7f1380161080 0x7f1380161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 --2- 172.21.15.97:0/2629428217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13800cfae0 0x7f13800cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 --2- 172.21.15.97:0/2629428217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f138015d590 0x7f138015e540 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 -- 172.21.15.97:0/2629428217 >> 172.21.15.97:0/2629428217 conn(0x7f138009c560 msgr2=0x7f13800d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 -- 172.21.15.97:0/2629428217 shutdown_connections 2024-01-31T05:01:40.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.024+0000 7f1386e00640 1 -- 172.21.15.97:0/2629428217 wait complete. 2024-01-31T05:01:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.025+0000 7f1386e00640 1 Processor -- start 2024-01-31T05:01:40.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.026+0000 7f1386e00640 1 -- start start 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.026+0000 7f1386e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 0x7f13800b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.026+0000 7f1386e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138015d590 0x7f13801603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f1386e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1380161080 0x7f13801608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f1386e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800b7a30 con 0x7f13800cfae0 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f1386e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800b7d10 con 0x7f138015d590 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f1386e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800b7ff0 con 0x7f1380161080 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 0x7f13800b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 0x7f13800b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43738/0 (socket says 172.21.15.97:43738) 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 -- 172.21.15.97:0/2303338254 learned_addr learned my addr 172.21.15.97:0/2303338254 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 -- 172.21.15.97:0/2303338254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1380161080 msgr2=0x7f13801608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1380161080 0x7f13801608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 -- 172.21.15.97:0/2303338254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138015d590 msgr2=0x7f13801603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138015d590 0x7f13801603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.027+0000 7f137fe00640 1 -- 172.21.15.97:0/2303338254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13680094d0 con 0x7f13800cfae0 2024-01-31T05:01:40.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.028+0000 7f137fe00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 0x7f13800b6fb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13680097c0 tx=0x7f1368004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.028+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368039c90 con 0x7f13800cfae0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.028+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1368039df0 con 0x7f13800cfae0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.028+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368043620 con 0x7f13800cfae0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.028+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138009e2f0 con 0x7f13800cfae0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.028+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13800c8e20 con 0x7f13800cfae0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.029+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13800b8130 con 0x7f13800cfae0 2024-01-31T05:01:40.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.030+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1368043780 con 0x7f13800cfae0 2024-01-31T05:01:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.031+0000 7f1377e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f135806aed0 0x7f135806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.031+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f136803e070 con 0x7f13800cfae0 2024-01-31T05:01:40.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.031+0000 7f137f400640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f135806aed0 0x7f135806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.032+0000 7f137f400640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f135806aed0 0x7f135806d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f13700070b0 tx=0x7f1370007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.035+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1368077680 con 0x7f13800cfae0 2024-01-31T05:01:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.230+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1380067830 con 0x7f13800cfae0 2024-01-31T05:01:40.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.231+0000 7f1377e00640 1 -- 172.21.15.97:0/2303338254 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f13680340a0 con 0x7f13800cfae0 2024-01-31T05:01:40.230 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f135806aed0 msgr2=0x7f135806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f135806aed0 0x7f135806d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f13700070b0 tx=0x7f1370007040 comp rx=0 tx=0).stop 2024-01-31T05:01:40.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 msgr2=0x7f13800b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 0x7f13800b6fb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f13680097c0 tx=0x7f1368004730 comp rx=0 tx=0).stop 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 shutdown_connections 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1380161080 0x7f13801608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f138015d590 0x7f13801603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f135806aed0 0x7f135806d390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.233+0000 7f1386e00640 1 --2- 172.21.15.97:0/2303338254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13800cfae0 0x7f13800b6fb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.234+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 >> 172.21.15.97:0/2303338254 conn(0x7f138009c560 msgr2=0x7f138015f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.234+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 shutdown_connections 2024-01-31T05:01:40.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.234+0000 7f1386e00640 1 -- 172.21.15.97:0/2303338254 wait complete. 2024-01-31T05:01:40.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.461+0000 7fcc61800640 1 Processor -- start 2024-01-31T05:01:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.461+0000 7fcc61800640 1 -- start start 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc61800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc5c160e00 0x7fcc5c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc61800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcc5c0d1b90 0x7fcc5c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc61800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0cde30 0x7fcc5c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc61800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c0ce2c0 con 0x7fcc5c160e00 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc61800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c06b570 con 0x7fcc5c0d1b90 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc61800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0cb6c0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc5c160e00 0x7fcc5c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc5c160e00 0x7fcc5c0b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43748/0 (socket says 172.21.15.97:43748) 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 -- 172.21.15.97:0/2699171828 learned_addr learned my addr 172.21.15.97:0/2699171828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 -- 172.21.15.97:0/2699171828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0cde30 msgr2=0x7fcc5c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 --2- 172.21.15.97:0/2699171828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0cde30 0x7fcc5c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 -- 172.21.15.97:0/2699171828 >> v2:172.21.15.188:3300/0 conn(0x7fcc5c0d1b90 msgr2=0x7fcc5c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 --2- 172.21.15.97:0/2699171828 >> v2:172.21.15.188:3300/0 conn(0x7fcc5c0d1b90 0x7fcc5c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.463+0000 7fcc5aa00640 1 -- 172.21.15.97:0/2699171828 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c1611e0 con 0x7fcc5c160e00 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.464+0000 7fcc5aa00640 1 --2- 172.21.15.97:0/2699171828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0b2330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06e590 tx=0x7fcc4402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401f197b730896ec server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc59600640 1 -- 172.21.15.97:0/2699171828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440317d0 con 0x7fcc5c160e00 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc59600640 1 -- 172.21.15.97:0/2699171828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44031930 con 0x7fcc5c160e00 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc59600640 1 -- 172.21.15.97:0/2699171828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44031cc0 con 0x7fcc5c160e00 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 -- 172.21.15.97:0/2699171828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c160e00 msgr2=0x7fcc5c0b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 --2- 172.21.15.97:0/2699171828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0b2330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06e590 tx=0x7fcc4402b870 comp rx=0 tx=0).stop 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 -- 172.21.15.97:0/2699171828 shutdown_connections 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 --2- 172.21.15.97:0/2699171828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0cde30 0x7fcc5c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 --2- 172.21.15.97:0/2699171828 >> v2:172.21.15.188:3300/0 conn(0x7fcc5c0d1b90 0x7fcc5c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 --2- 172.21.15.97:0/2699171828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0b2330 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.465+0000 7fcc61800640 1 -- 172.21.15.97:0/2699171828 >> 172.21.15.97:0/2699171828 conn(0x7fcc5c0d1070 msgr2=0x7fcc5c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.466+0000 7fcc61800640 1 -- 172.21.15.97:0/2699171828 shutdown_connections 2024-01-31T05:01:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.466+0000 7fcc61800640 1 -- 172.21.15.97:0/2699171828 wait complete. 2024-01-31T05:01:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.467+0000 7fcc61800640 1 Processor -- start 2024-01-31T05:01:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.467+0000 7fcc61800640 1 -- start start 2024-01-31T05:01:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc61800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 0x7fcc5c1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc61800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0d1b90 0x7fcc5c0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc61800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc61800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0c89a0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc61800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c09eaf0 con 0x7fcc5c160e00 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc61800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c152a20 con 0x7fcc5c0d1b90 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc5aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 0x7fcc5c1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc5aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 0x7fcc5c1524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43756/0 (socket says 172.21.15.97:43756) 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.468+0000 7fcc5aa00640 1 -- 172.21.15.97:0/905415561 learned_addr learned my addr 172.21.15.97:0/905415561 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5a000640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0d1b90 0x7fcc5c0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5aa00640 1 -- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0d1b90 msgr2=0x7fcc5c0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5aa00640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0d1b90 0x7fcc5c0c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5b400640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5aa00640 1 -- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc5c160e00 msgr2=0x7fcc5c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5aa00640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5aa00640 1 -- 172.21.15.97:0/905415561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc440094d0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc5aa00640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 0x7fcc5c1524e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc4402b8b0 tx=0x7fcc440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.469+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440329b0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.470+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc44032b30 con 0x7fcc5c0cde30 2024-01-31T05:01:40.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.470+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c09cf20 con 0x7fcc5c0cde30 2024-01-31T05:01:40.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.470+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440453f0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.470+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c0c82e0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.471+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c0c8ae0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.472+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fcc44030070 con 0x7fcc5c0cde30 2024-01-31T05:01:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.472+0000 7fcc52a00640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc3406af00 0x7fcc3406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.473+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcc440ab5e0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.473+0000 7fcc5a000640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc3406af00 0x7fcc3406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.473+0000 7fcc5a000640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc3406af00 0x7fcc3406d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c007dc0 tx=0x7fcc4c0072f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.477+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fcc5c0c8ae0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.672+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc5c0c8ae0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.675+0000 7fcc52a00640 1 -- 172.21.15.97:0/905415561 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7fcc5c0c8ae0 con 0x7fcc5c0cde30 2024-01-31T05:01:40.675 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:40.675 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.677+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc3406af00 msgr2=0x7fcc3406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.677+0000 7fcc61800640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc3406af00 0x7fcc3406d3c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c007dc0 tx=0x7fcc4c0072f0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.677+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 msgr2=0x7fcc5c1524e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.677+0000 7fcc61800640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 0x7fcc5c1524e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc4402b8b0 tx=0x7fcc440046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 shutdown_connections 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc5c160e00 0x7fcc5c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc5c0d1b90 0x7fcc5c0c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc3406af00 0x7fcc3406d3c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 --2- 172.21.15.97:0/905415561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc5c0cde30 0x7fcc5c1524e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 >> 172.21.15.97:0/905415561 conn(0x7fcc5c0d1070 msgr2=0x7fcc5c0c7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 shutdown_connections 2024-01-31T05:01:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.678+0000 7fcc61800640 1 -- 172.21.15.97:0/905415561 wait complete. 2024-01-31T05:01:40.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:40.696 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.901+0000 7f1ec4a00640 1 Processor -- start 2024-01-31T05:01:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.901+0000 7f1ec4a00640 1 -- start start 2024-01-31T05:01:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec4a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ebc0cbf20 0x7f1ebc0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 0x7f1ebc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec4a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc06cd20 con 0x7f1ebc15b470 2024-01-31T05:01:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec4a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0ce970 con 0x7f1ebc0cbf20 2024-01-31T05:01:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc15b850 con 0x7f1ebc0abcf0 2024-01-31T05:01:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec2000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1ebc15b470 0x7f1ebc0cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54936/0 (socket says 172.21.15.97:54936) 2024-01-31T05:01:40.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.903+0000 7f1ec2000640 1 -- 172.21.15.97:0/3033422196 learned_addr learned my addr 172.21.15.97:0/3033422196 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec2a00640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 0x7f1ebc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec1600640 1 --2- 172.21.15.97:0/3033422196 >> v2:172.21.15.188:3300/0 conn(0x7f1ebc0cbf20 0x7f1ebc0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823518444 0 0) 0x7f1ebc06cd20 con 0x7f1ebc15b470 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e9c003520 con 0x7f1ebc15b470 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104526236 0 0) 0x7f1e9c003520 con 0x7f1ebc15b470 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 msgr2=0x7f1ebc0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 0x7f1ebc0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 >> v2:172.21.15.188:3300/0 conn(0x7f1ebc0cbf20 msgr2=0x7f1ebc0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 --2- 172.21.15.97:0/3033422196 >> v2:172.21.15.188:3300/0 conn(0x7f1ebc0cbf20 0x7f1ebc0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0ceb60 con 0x7f1ebc15b470 2024-01-31T05:01:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f1eb40017f0 con 0x7f1ebc15b470 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 >> v1:172.21.15.97:6789/0 conn(0x7f1ebc15b470 legacy=0x7f1ebc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c0039a0 0x7f1e9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 0x7f1e9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c004d00 0x7f1e9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc15b850 con 0x7f1e9c0039a0 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c00b980 con 0x7f1e9c004d00 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.904+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c00bac0 con 0x7f1ebc0abcf0 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c0039a0 0x7f1e9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec1600640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 0x7f1e9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec2000640 1 -- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 msgr2=0x7f1e9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0abcf0 0x7f1e9c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec2000640 1 -- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c004d00 msgr2=0x7f1e9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c004d00 0x7f1e9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.905+0000 7f1ec2000640 1 -- 172.21.15.97:0/3033422196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c003520 con 0x7f1e9c0039a0 2024-01-31T05:01:40.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.906+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c0039a0 0x7f1e9c003dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400c1c0 tx=0x7f1e9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb432ed0027ba55 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.906+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4007270 con 0x7f1e9c0039a0 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.906+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40073f0 con 0x7f1e9c0039a0 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.906+0000 7f1ec0c00640 1 -- 172.21.15.97:0/3033422196 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb402c890 con 0x7f1e9c0039a0 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3033422196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c0039a0 msgr2=0x7f1e9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3033422196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c0039a0 0x7f1e9c003dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400c1c0 tx=0x7f1e9c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec2a00640 1 -- 172.21.15.97:0/3033422196 reap_dead start 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3033422196 shutdown_connections 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3033422196 >> 172.21.15.97:0/3033422196 conn(0x7f1ebc0b0a10 msgr2=0x7f1ebc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3033422196 shutdown_connections 2024-01-31T05:01:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.907+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3033422196 wait complete. 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.909+0000 7f1ec4a00640 1 Processor -- start 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.909+0000 7f1ec4a00640 1 -- start start 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c0039a0 0x7f1ebc0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0ad5b0 0x7f1ebc1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 0x7f1ebc0b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec4a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0ad270 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec4a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc099f80 con 0x7f1e9c0039a0 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc150e00 con 0x7f1ebc0ad5b0 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 0x7f1ebc0b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 0x7f1ebc0b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43760/0 (socket says 172.21.15.97:43760) 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.910+0000 7f1ec2a00640 1 -- 172.21.15.97:0/3211195152 learned_addr learned my addr 172.21.15.97:0/3211195152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c0039a0 0x7f1ebc0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec1600640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0ad5b0 0x7f1ebc1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2a00640 1 -- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0ad5b0 msgr2=0x7f1ebc1508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0ad5b0 0x7f1ebc1508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2a00640 1 -- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c0039a0 msgr2=0x7f1ebc0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c0039a0 0x7f1ebc0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2a00640 1 -- 172.21.15.97:0/3211195152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0add00 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c0039a0 0x7f1ebc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec1600640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0ad5b0 0x7f1ebc1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec2a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 0x7f1ebc0b2f00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0700b0 tx=0x7f1eac00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00c680 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac00c7e0 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac00cb90 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc15ba70 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.911+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc15c030 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.913+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc067830 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.913+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1eac00ccf0 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.915+0000 7f1eba000640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e9406aed0 0x7f1e9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.915+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1eac089780 con 0x7f1ebc0b2a80 2024-01-31T05:01:40.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.915+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e9406aed0 0x7f1e9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:40.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.916+0000 7f1ec2000640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e9406aed0 0x7f1e9406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034000 tx=0x7f1eb4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:40.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:40.919+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1eac0542f0 con 0x7f1ebc0b2a80 2024-01-31T05:01:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.116+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ebc072360 con 0x7f1ebc0b2a80 2024-01-31T05:01:41.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.117+0000 7f1eba000640 1 -- 172.21.15.97:0/3211195152 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1eac057910 con 0x7f1ebc0b2a80 2024-01-31T05:01:41.116 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e9406aed0 msgr2=0x7f1e9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e9406aed0 0x7f1e9406d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4034000 tx=0x7f1eb4009030 comp rx=0 tx=0).stop 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 msgr2=0x7f1ebc0b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 0x7f1ebc0b2f00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0700b0 tx=0x7f1eac00ec00 comp rx=0 tx=0).stop 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 shutdown_connections 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e9406aed0 0x7f1e9406d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ebc0b2a80 0x7f1ebc0b2f00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ebc0ad5b0 0x7f1ebc1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 --2- 172.21.15.97:0/3211195152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c0039a0 0x7f1ebc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 >> 172.21.15.97:0/3211195152 conn(0x7f1ebc0b0a10 msgr2=0x7f1ebc0cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 shutdown_connections 2024-01-31T05:01:41.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.120+0000 7f1ec4a00640 1 -- 172.21.15.97:0/3211195152 wait complete. 2024-01-31T05:01:41.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.348+0000 7f6eefe00640 1 Processor -- start 2024-01-31T05:01:41.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.348+0000 7f6eefe00640 1 -- start start 2024-01-31T05:01:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eefe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ee806d580 0x7f6ee806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eefe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80ceb40 con 0x7f6ee8161020 2024-01-31T05:01:41.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eefe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee806dea0 con 0x7f6ee815d690 2024-01-31T05:01:41.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eefe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee815e7c0 con 0x7f6ee806d580 2024-01-31T05:01:41.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eed400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ee806d580 0x7f6ee806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eede00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6ee8161020 0x7f6ee81614b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54942/0 (socket says 172.21.15.97:54942) 2024-01-31T05:01:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.350+0000 7f6eede00640 1 -- 172.21.15.97:0/115016112 learned_addr learned my addr 172.21.15.97:0/115016112 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.351+0000 7f6eed400640 1 -- 172.21.15.97:0/115016112 >> v1:172.21.15.188:6789/0 conn(0x7f6ee815d690 legacy=0x7f6ee80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.351+0000 7f6eed400640 1 -- 172.21.15.97:0/115016112 >> v1:172.21.15.97:6789/0 conn(0x7f6ee8161020 legacy=0x7f6ee81614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.351+0000 7f6eed400640 1 -- 172.21.15.97:0/115016112 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee815db40 con 0x7f6ee806d580 2024-01-31T05:01:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.351+0000 7f6ee7e00640 1 -- 172.21.15.97:0/115016112 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71888087 0 0) 0x7f6ee80ceb40 con 0x7f6ee8161020 2024-01-31T05:01:41.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.352+0000 7f6eed400640 1 --2- 172.21.15.97:0/115016112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee806d960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009790 tx=0x7f6ed802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd65b70c8c160459 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.352+0000 7f6ee7e00640 1 -- 172.21.15.97:0/115016112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8031830 con 0x7f6ee806d580 2024-01-31T05:01:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6ee7e00640 1 -- 172.21.15.97:0/115016112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed8031990 con 0x7f6ee806d580 2024-01-31T05:01:41.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6ee7e00640 1 -- 172.21.15.97:0/115016112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8031d20 con 0x7f6ee806d580 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 -- 172.21.15.97:0/115016112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 msgr2=0x7f6ee806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 --2- 172.21.15.97:0/115016112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee806d960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009790 tx=0x7f6ed802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 -- 172.21.15.97:0/115016112 shutdown_connections 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 --2- 172.21.15.97:0/115016112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee806d960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 -- 172.21.15.97:0/115016112 >> 172.21.15.97:0/115016112 conn(0x7f6ee80b0a10 msgr2=0x7f6ee80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 -- 172.21.15.97:0/115016112 shutdown_connections 2024-01-31T05:01:41.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.353+0000 7f6eefe00640 1 -- 172.21.15.97:0/115016112 wait complete. 2024-01-31T05:01:41.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.355+0000 7f6eefe00640 1 Processor -- start 2024-01-31T05:01:41.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.355+0000 7f6eefe00640 1 -- start start 2024-01-31T05:01:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.356+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.356+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee815d690 0x7f6ee8159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.356+0000 7f6eefe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee8161020 0x7f6ee80cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.356+0000 7f6eefe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815bb60 con 0x7f6ee806d580 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.356+0000 7f6eefe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8153200 con 0x7f6ee815d690 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.356+0000 7f6eefe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815b260 con 0x7f6ee8161020 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee815ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43770/0 (socket says 172.21.15.97:43770) 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 -- 172.21.15.97:0/1261331867 learned_addr learned my addr 172.21.15.97:0/1261331867 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eeca00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee815d690 0x7f6ee8159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 -- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee8161020 msgr2=0x7f6ee80cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee8161020 0x7f6ee80cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 -- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee815d690 msgr2=0x7f6ee8159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee815d690 0x7f6ee8159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 -- 172.21.15.97:0/1261331867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80093a0 con 0x7f6ee806d580 2024-01-31T05:01:41.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.357+0000 7f6eed400640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee815ad20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098b0 tx=0x7f6ed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.358+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed803aa60 con 0x7f6ee806d580 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.358+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed803abe0 con 0x7f6ee806d580 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.358+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee809c7a0 con 0x7f6ee806d580 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.358+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80443f0 con 0x7f6ee806d580 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.358+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee81f4240 con 0x7f6ee806d580 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.359+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8067830 con 0x7f6ee806d580 2024-01-31T05:01:41.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.360+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6ed804d450 con 0x7f6ee806d580 2024-01-31T05:01:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.361+0000 7f6ee5600640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6ec006aed0 0x7f6ec006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.362+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6ed80abb70 con 0x7f6ee806d580 2024-01-31T05:01:41.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.362+0000 7f6eeca00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6ec006aed0 0x7f6ec006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.362+0000 7f6eeca00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6ec006aed0 0x7f6ec006d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ee81f0f00 tx=0x7f6ed0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:41.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.366+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6ed8076770 con 0x7f6ee806d580 2024-01-31T05:01:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.560+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ee815cd20 con 0x7f6ee806d580 2024-01-31T05:01:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.560+0000 7f6ee5600640 1 -- 172.21.15.97:0/1261331867 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ed8035020 con 0x7f6ee806d580 2024-01-31T05:01:41.560 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:41.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6ec006aed0 msgr2=0x7f6ec006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6ec006aed0 0x7f6ec006d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ee81f0f00 tx=0x7f6ed0008040 comp rx=0 tx=0).stop 2024-01-31T05:01:41.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 msgr2=0x7f6ee815ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee815ad20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098b0 tx=0x7f6ed80046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 shutdown_connections 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee8161020 0x7f6ee80cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee815d690 0x7f6ee8159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6ec006aed0 0x7f6ec006d390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 --2- 172.21.15.97:0/1261331867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ee806d580 0x7f6ee815ad20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 >> 172.21.15.97:0/1261331867 conn(0x7f6ee80b0a10 msgr2=0x7f6ee80ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 shutdown_connections 2024-01-31T05:01:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.563+0000 7f6eefe00640 1 -- 172.21.15.97:0/1261331867 wait complete. 2024-01-31T05:01:41.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:41.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.794+0000 7fc984400640 1 Processor -- start 2024-01-31T05:01:41.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.794+0000 7fc984400640 1 -- start start 2024-01-31T05:01:41.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc984400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc984400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc97c0abcf0 0x7fc97c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc984400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c06cd20 con 0x7fc97c0cb810 2024-01-31T05:01:41.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc984400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0ce970 con 0x7fc97c0abcf0 2024-01-31T05:01:41.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc984400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15b850 con 0x7fc97c15b470 2024-01-31T05:01:41.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc981a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.796+0000 7fc981000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc97c0abcf0 0x7fc97c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c0722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59618/0 (socket says 172.21.15.97:59618) 2024-01-31T05:01:41.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc97c0abcf0 0x7fc97c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:42140/0 (socket says 172.21.15.97:42140) 2024-01-31T05:01:41.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981a00640 1 -- 172.21.15.97:0/4043802639 learned_addr learned my addr 172.21.15.97:0/4043802639 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:41.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981000640 1 -- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 msgr2=0x7fc97c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:41.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981000640 1 --2- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981000640 1 -- 172.21.15.97:0/4043802639 >> v1:172.21.15.188:6789/0 conn(0x7fc97c0cb810 legacy=0x7fc97c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.797+0000 7fc981000640 1 -- 172.21.15.97:0/4043802639 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0ceb60 con 0x7fc97c0abcf0 2024-01-31T05:01:41.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.799+0000 7fc981000640 1 --2- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0abcf0 0x7fc97c0ce430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097d0 tx=0x7fc96402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a8745df83dd469f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:41.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.799+0000 7fc97be00640 1 -- 172.21.15.97:0/4043802639 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964031800 con 0x7fc97c0abcf0 2024-01-31T05:01:41.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.799+0000 7fc97be00640 1 -- 172.21.15.97:0/4043802639 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964031960 con 0x7fc97c0abcf0 2024-01-31T05:01:41.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.799+0000 7fc97be00640 1 -- 172.21.15.97:0/4043802639 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964031cf0 con 0x7fc97c0abcf0 2024-01-31T05:01:41.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 -- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0abcf0 msgr2=0x7fc97c0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 --2- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0abcf0 0x7fc97c0ce430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc9640097d0 tx=0x7fc96402b8a0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 -- 172.21.15.97:0/4043802639 shutdown_connections 2024-01-31T05:01:41.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 --2- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0abcf0 0x7fc97c0ce430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 --2- 172.21.15.97:0/4043802639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 -- 172.21.15.97:0/4043802639 >> 172.21.15.97:0/4043802639 conn(0x7fc97c0b0a10 msgr2=0x7fc97c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:41.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 -- 172.21.15.97:0/4043802639 shutdown_connections 2024-01-31T05:01:41.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.800+0000 7fc984400640 1 -- 172.21.15.97:0/4043802639 wait complete. 2024-01-31T05:01:41.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.802+0000 7fc984400640 1 Processor -- start 2024-01-31T05:01:41.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.802+0000 7fc984400640 1 -- start start 2024-01-31T05:01:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.803+0000 7fc984400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 0x7fc97c149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.803+0000 7fc984400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0cb810 0x7fc97c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.803+0000 7fc984400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.803+0000 7fc984400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15d050 con 0x7fc97c0abcf0 2024-01-31T05:01:41.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.803+0000 7fc984400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c149f90 con 0x7fc97c0cb810 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.803+0000 7fc984400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15c440 con 0x7fc97c15b470 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 0x7fc97c149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 0x7fc97c149a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43784/0 (socket says 172.21.15.97:43784) 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 -- 172.21.15.97:0/3581798015 learned_addr learned my addr 172.21.15.97:0/3581798015 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc982400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981000640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0cb810 0x7fc97c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 -- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 msgr2=0x7fc97c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c1563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 -- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0cb810 msgr2=0x7fc97c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0cb810 0x7fc97c15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 -- 172.21.15.97:0/3581798015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc964009400 con 0x7fc97c0abcf0 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.804+0000 7fc981a00640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 0x7fc97c149a50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00e7d0 tx=0x7fc96c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.805+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c009560 con 0x7fc97c0abcf0 2024-01-31T05:01:41.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.805+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00c940 con 0x7fc97c0abcf0 2024-01-31T05:01:41.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.805+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00eee0 con 0x7fc97c0abcf0 2024-01-31T05:01:41.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.805+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c15c720 con 0x7fc97c0abcf0 2024-01-31T05:01:41.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.805+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c15cce0 con 0x7fc97c0abcf0 2024-01-31T05:01:41.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.806+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c067830 con 0x7fc97c0abcf0 2024-01-31T05:01:41.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.807+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc96c00ccf0 con 0x7fc97c0abcf0 2024-01-31T05:01:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.808+0000 7fc979600640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc95406aed0 0x7fc95406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:41.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.808+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc96c089f10 con 0x7fc97c0abcf0 2024-01-31T05:01:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.808+0000 7fc981000640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc95406aed0 0x7fc95406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:41.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.809+0000 7fc981000640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc95406aed0 0x7fc95406d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc964036000 tx=0x7fc96403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:41.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:41.813+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc96c0549f0 con 0x7fc97c0abcf0 2024-01-31T05:01:42.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.008+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc97c06e1b0 con 0x7fc97c0abcf0 2024-01-31T05:01:42.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.009+0000 7fc979600640 1 -- 172.21.15.97:0/3581798015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc96c0210a0 con 0x7fc97c0abcf0 2024-01-31T05:01:42.009 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:42.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.011+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc95406aed0 msgr2=0x7fc95406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.011+0000 7fc984400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc95406aed0 0x7fc95406d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc964036000 tx=0x7fc96403c040 comp rx=0 tx=0).stop 2024-01-31T05:01:42.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.011+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 msgr2=0x7fc97c149a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.011+0000 7fc984400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 0x7fc97c149a50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00e7d0 tx=0x7fc96c00ec90 comp rx=0 tx=0).stop 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 shutdown_connections 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc97c15b470 0x7fc97c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc97c0cb810 0x7fc97c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc95406aed0 0x7fc95406d390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 --2- 172.21.15.97:0/3581798015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc97c0abcf0 0x7fc97c149a50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 >> 172.21.15.97:0/3581798015 conn(0x7fc97c0b0a10 msgr2=0x7fc97c06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 shutdown_connections 2024-01-31T05:01:42.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.012+0000 7fc984400640 1 -- 172.21.15.97:0/3581798015 wait complete. 2024-01-31T05:01:42.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.237+0000 7f1027e00640 1 Processor -- start 2024-01-31T05:01:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.237+0000 7f1027e00640 1 -- start start 2024-01-31T05:01:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.238+0000 7f1027e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f102015a350 0x7f10200abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.238+0000 7f1027e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f10200ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.238+0000 7f1027e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102015abc0 con 0x7f102015a730 2024-01-31T05:01:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1027e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10200ce7e0 con 0x7f102015a350 2024-01-31T05:01:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1027e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10200b2c90 con 0x7f10200cde10 2024-01-31T05:01:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f102015a350 0x7f10200abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f102015a350 0x7f10200abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43800/0 (socket says 172.21.15.97:43800) 2024-01-31T05:01:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1024a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f102015a730 0x7f102006da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54950/0 (socket says 172.21.15.97:54950) 2024-01-31T05:01:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 -- 172.21.15.97:0/1344622117 learned_addr learned my addr 172.21.15.97:0/1344622117 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025e00640 1 --2- 172.21.15.97:0/1344622117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f10200ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 -- 172.21.15.97:0/1344622117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 msgr2=0x7f10200ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 --2- 172.21.15.97:0/1344622117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f10200ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 -- 172.21.15.97:0/1344622117 >> v1:172.21.15.97:6789/0 conn(0x7f102015a730 legacy=0x7f102006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f1025400640 1 -- 172.21.15.97:0/1344622117 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102015aea0 con 0x7f102015a350 2024-01-31T05:01:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.239+0000 7f101fe00640 1 -- 172.21.15.97:0/1344622117 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464437741 0 0) 0x7f102015abc0 con 0x7f102015a730 2024-01-31T05:01:42.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.240+0000 7f1025400640 1 --2- 172.21.15.97:0/1344622117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f10200abcf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1010009890 tx=0x7f101002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc3c3a81db94b8f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:42.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f101fe00640 1 -- 172.21.15.97:0/1344622117 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010030610 con 0x7f102015a350 2024-01-31T05:01:42.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f101fe00640 1 -- 172.21.15.97:0/1344622117 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010030770 con 0x7f102015a350 2024-01-31T05:01:42.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f101fe00640 1 -- 172.21.15.97:0/1344622117 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010030b00 con 0x7f102015a350 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f1027e00640 1 -- 172.21.15.97:0/1344622117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 msgr2=0x7f10200abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f1027e00640 1 --2- 172.21.15.97:0/1344622117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f10200abcf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1010009890 tx=0x7f101002b960 comp rx=0 tx=0).stop 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f1027e00640 1 -- 172.21.15.97:0/1344622117 shutdown_connections 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f1027e00640 1 --2- 172.21.15.97:0/1344622117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f10200ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f1027e00640 1 --2- 172.21.15.97:0/1344622117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f10200abcf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.241+0000 7f1027e00640 1 -- 172.21.15.97:0/1344622117 >> 172.21.15.97:0/1344622117 conn(0x7f10200b0a10 msgr2=0x7f10200b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.242+0000 7f1027e00640 1 -- 172.21.15.97:0/1344622117 shutdown_connections 2024-01-31T05:01:42.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.242+0000 7f1027e00640 1 -- 172.21.15.97:0/1344622117 wait complete. 2024-01-31T05:01:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.243+0000 7f1027e00640 1 Processor -- start 2024-01-31T05:01:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.243+0000 7f1027e00640 1 -- start start 2024-01-31T05:01:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.244+0000 7f1027e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f1020145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.244+0000 7f1027e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f1020152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.244+0000 7f1027e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f102009ad70 0x7f10200cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.244+0000 7f1027e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200cc8c0 con 0x7f102015a350 2024-01-31T05:01:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.244+0000 7f1027e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10200cb9f0 con 0x7f102009ad70 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.244+0000 7f1027e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10200cbcb0 con 0x7f10200cde10 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f1020152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f1020152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43806/0 (socket says 172.21.15.97:43806) 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 -- 172.21.15.97:0/946587512 learned_addr learned my addr 172.21.15.97:0/946587512 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 -- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 msgr2=0x7f1020145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1025400640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f1020145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f1020145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 -- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f102009ad70 msgr2=0x7f10200cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f102009ad70 0x7f10200cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 -- 172.21.15.97:0/946587512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009470 con 0x7f102015a350 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.245+0000 7f1024a00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f1020152110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f100800c410 tx=0x7f100800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.246+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080074e0 con 0x7f102015a350 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.246+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1008007660 con 0x7f102015a350 2024-01-31T05:01:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.246+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008004a20 con 0x7f102015a350 2024-01-31T05:01:42.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.246+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10200cbf90 con 0x7f102015a350 2024-01-31T05:01:42.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.246+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10200cc550 con 0x7f102015a350 2024-01-31T05:01:42.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.247+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10201459a0 con 0x7f102015a350 2024-01-31T05:01:42.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.248+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1008004b80 con 0x7f102015a350 2024-01-31T05:01:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.249+0000 7f101d600640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ff806aed0 0x7f0ff806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.249+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f10080898e0 con 0x7f102015a350 2024-01-31T05:01:42.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.249+0000 7f1025400640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ff806aed0 0x7f0ff806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.250+0000 7f1025400640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ff806aed0 0x7f0ff806d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1010036000 tx=0x7f101003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:42.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.254+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f10201459a0 con 0x7f102015a350 2024-01-31T05:01:42.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.445+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f10201459a0 con 0x7f102015a350 2024-01-31T05:01:42.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.446+0000 7f101d600640 1 -- 172.21.15.97:0/946587512 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f10201459a0 con 0x7f102015a350 2024-01-31T05:01:42.445 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:42.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ff806aed0 msgr2=0x7f0ff806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ff806aed0 0x7f0ff806d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1010036000 tx=0x7f101003b040 comp rx=0 tx=0).stop 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 msgr2=0x7f1020152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f1020152110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f100800c410 tx=0x7f100800c8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 shutdown_connections 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f102009ad70 0x7f10200cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ff806aed0 0x7f0ff806d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f102015a350 0x7f1020152110 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 --2- 172.21.15.97:0/946587512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10200cde10 0x7f1020145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.448+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 >> 172.21.15.97:0/946587512 conn(0x7f10200b0a10 msgr2=0x7f10200ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.449+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 shutdown_connections 2024-01-31T05:01:42.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.449+0000 7f1027e00640 1 -- 172.21.15.97:0/946587512 wait complete. 2024-01-31T05:01:42.466 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:42.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:01:42.517 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:01:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.669+0000 7f5454e00640 1 Processor -- start 2024-01-31T05:01:42.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.669+0000 7f5454e00640 1 -- start start 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5454e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f545015b400 0x7f545006d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5454e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545006d1a0 con 0x7f545006db70 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5454e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545006c600 con 0x7f54500cb810 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5454e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500cc960 con 0x7f545015b400 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5447e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f545015b400 0x7f545006d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5447e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f545015b400 0x7f545006d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43820/0 (socket says 172.21.15.97:43820) 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.671+0000 7f5447e00640 1 -- 172.21.15.97:0/593789425 learned_addr learned my addr 172.21.15.97:0/593789425 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:42.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.672+0000 7f5447e00640 1 -- 172.21.15.97:0/593789425 >> v1:172.21.15.188:6789/0 conn(0x7f54500cb810 legacy=0x7f54500cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.672+0000 7f5447e00640 1 -- 172.21.15.97:0/593789425 >> v1:172.21.15.97:6789/0 conn(0x7f545006db70 legacy=0x7f54500ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.672+0000 7f5447e00640 1 -- 172.21.15.97:0/593789425 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545006c870 con 0x7f545015b400 2024-01-31T05:01:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.673+0000 7f5447e00640 1 --2- 172.21.15.97:0/593789425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545015b400 0x7f545006d630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5438009870 tx=0x7f543802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16eeab3ac2d094a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:42.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.673+0000 7f544d400640 1 -- 172.21.15.97:0/593789425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438030850 con 0x7f545015b400 2024-01-31T05:01:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.673+0000 7f544d400640 1 -- 172.21.15.97:0/593789425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54380309b0 con 0x7f545015b400 2024-01-31T05:01:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.673+0000 7f544d400640 1 -- 172.21.15.97:0/593789425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438030d40 con 0x7f545015b400 2024-01-31T05:01:42.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 -- 172.21.15.97:0/593789425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545015b400 msgr2=0x7f545006d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 --2- 172.21.15.97:0/593789425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545015b400 0x7f545006d630 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5438009870 tx=0x7f543802b990 comp rx=0 tx=0).stop 2024-01-31T05:01:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 -- 172.21.15.97:0/593789425 shutdown_connections 2024-01-31T05:01:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 --2- 172.21.15.97:0/593789425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f545015b400 0x7f545006d630 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 -- 172.21.15.97:0/593789425 >> 172.21.15.97:0/593789425 conn(0x7f54500b0a10 msgr2=0x7f54500b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 -- 172.21.15.97:0/593789425 shutdown_connections 2024-01-31T05:01:42.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.674+0000 7f5454e00640 1 -- 172.21.15.97:0/593789425 wait complete. 2024-01-31T05:01:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.676+0000 7f5454e00640 1 Processor -- start 2024-01-31T05:01:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.676+0000 7f5454e00640 1 -- start start 2024-01-31T05:01:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5454e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f545006db70 0x7f54500999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5454e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 0x7f5450158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5454e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f545015b400 0x7f545009c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5454e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54501573d0 con 0x7f54500cb810 2024-01-31T05:01:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5454e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450155170 con 0x7f545015b400 2024-01-31T05:01:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5454e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5450159ff0 con 0x7f545006db70 2024-01-31T05:01:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 0x7f5450158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 0x7f5450158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43824/0 (socket says 172.21.15.97:43824) 2024-01-31T05:01:42.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 -- 172.21.15.97:0/2656598978 learned_addr learned my addr 172.21.15.97:0/2656598978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 -- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f545006db70 msgr2=0x7f54500999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f545006db70 0x7f54500999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 -- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f545015b400 msgr2=0x7f545009c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f544e800640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f545015b400 0x7f545009c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f545015b400 0x7f545009c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f5447e00640 1 -- 172.21.15.97:0/2656598978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380094d0 con 0x7f54500cb810 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.677+0000 7f544e800640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f545015b400 0x7f545009c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.678+0000 7f5447e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 0x7f5450158d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5438000c00 tx=0x7f5438004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.678+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438030800 con 0x7f54500cb810 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.678+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438039d90 con 0x7f54500cb810 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.678+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438043650 con 0x7f54500cb810 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.678+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5450157620 con 0x7f54500cb810 2024-01-31T05:01:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.678+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5450099f00 con 0x7f54500cb810 2024-01-31T05:01:42.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.679+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5450157b40 con 0x7f54500cb810 2024-01-31T05:01:42.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.680+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f543802e070 con 0x7f54500cb810 2024-01-31T05:01:42.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.681+0000 7f5446000640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f542806aed0 0x7f542806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:42.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.681+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f54380abab0 con 0x7f54500cb810 2024-01-31T05:01:42.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.681+0000 7f544de00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f542806aed0 0x7f542806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:42.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.682+0000 7f544de00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f542806aed0 0x7f542806d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5450157360 tx=0x7f54400079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:42.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.686+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5450157b40 con 0x7f54500cb810 2024-01-31T05:01:42.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.883+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5450157b40 con 0x7f54500cb810 2024-01-31T05:01:42.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.886+0000 7f5446000640 1 -- 172.21.15.97:0/2656598978 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f5450157b40 con 0x7f54500cb810 2024-01-31T05:01:42.886 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:42.886 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.888+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f542806aed0 msgr2=0x7f542806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.888+0000 7f5454e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f542806aed0 0x7f542806d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5450157360 tx=0x7f54400079b0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.888+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 msgr2=0x7f5450158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.888+0000 7f5454e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 0x7f5450158d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5438000c00 tx=0x7f5438004730 comp rx=0 tx=0).stop 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 shutdown_connections 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f545015b400 0x7f545009c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f542806aed0 0x7f542806d390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54500cb810 0x7f5450158d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 --2- 172.21.15.97:0/2656598978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f545006db70 0x7f54500999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 >> 172.21.15.97:0/2656598978 conn(0x7f54500b0a10 msgr2=0x7f54500ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 shutdown_connections 2024-01-31T05:01:42.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:42.889+0000 7f5454e00640 1 -- 172.21.15.97:0/2656598978 wait complete. 2024-01-31T05:01:42.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:42.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.165+0000 7fd866c00640 1 Processor -- start 2024-01-31T05:01:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.165+0000 7fd866c00640 1 -- start start 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd866c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd8600722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd866c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8600abcf0 0x7fd8600ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd866c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86006cd20 con 0x7fd8600cb810 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd866c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8600ce970 con 0x7fd8600abcf0 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd866c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86015b850 con 0x7fd86015b470 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd85f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8600abcf0 0x7fd8600ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd85f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8600abcf0 0x7fd8600ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43830/0 (socket says 172.21.15.97:43830) 2024-01-31T05:01:43.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.167+0000 7fd85f400640 1 -- 172.21.15.97:0/1450045019 learned_addr learned my addr 172.21.15.97:0/1450045019 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.168+0000 7fd85fe00640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd8600722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.168+0000 7fd85f400640 1 -- 172.21.15.97:0/1450045019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 msgr2=0x7fd8600722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.168+0000 7fd85f400640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd8600722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.168+0000 7fd85f400640 1 -- 172.21.15.97:0/1450045019 >> v1:172.21.15.188:6789/0 conn(0x7fd8600cb810 legacy=0x7fd8600ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.168+0000 7fd85f400640 1 -- 172.21.15.97:0/1450045019 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600ceb60 con 0x7fd8600abcf0 2024-01-31T05:01:43.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.168+0000 7fd85fe00640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd8600722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.169+0000 7fd85f400640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600abcf0 0x7fd8600ce430 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8480098a0 tx=0x7fd84802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea0fb8c5b424f3e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.169+0000 7fd85ea00640 1 -- 172.21.15.97:0/1450045019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848031640 con 0x7fd8600abcf0 2024-01-31T05:01:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.169+0000 7fd85ea00640 1 -- 172.21.15.97:0/1450045019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8480317a0 con 0x7fd8600abcf0 2024-01-31T05:01:43.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.169+0000 7fd85ea00640 1 -- 172.21.15.97:0/1450045019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848031b30 con 0x7fd8600abcf0 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 -- 172.21.15.97:0/1450045019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600abcf0 msgr2=0x7fd8600ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600abcf0 0x7fd8600ce430 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8480098a0 tx=0x7fd84802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 -- 172.21.15.97:0/1450045019 shutdown_connections 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600abcf0 0x7fd8600ce430 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 --2- 172.21.15.97:0/1450045019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd8600722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 -- 172.21.15.97:0/1450045019 >> 172.21.15.97:0/1450045019 conn(0x7fd8600b0a10 msgr2=0x7fd8600b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 -- 172.21.15.97:0/1450045019 shutdown_connections 2024-01-31T05:01:43.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.170+0000 7fd866c00640 1 -- 172.21.15.97:0/1450045019 wait complete. 2024-01-31T05:01:43.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.172+0000 7fd866c00640 1 Processor -- start 2024-01-31T05:01:43.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.172+0000 7fd866c00640 1 -- start start 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.173+0000 7fd866c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600abcf0 0x7fd8601408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.173+0000 7fd866c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 0x7fd860151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.173+0000 7fd866c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd860151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.173+0000 7fd866c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860140e00 con 0x7fd8600cb810 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.173+0000 7fd866c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86015e3c0 con 0x7fd8600abcf0 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.173+0000 7fd866c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86009acd0 con 0x7fd86015b470 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 0x7fd860151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 0x7fd860151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43836/0 (socket says 172.21.15.97:43836) 2024-01-31T05:01:43.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 -- 172.21.15.97:0/3102508816 learned_addr learned my addr 172.21.15.97:0/3102508816 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 -- 172.21.15.97:0/3102508816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 msgr2=0x7fd860151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd860151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 -- 172.21.15.97:0/3102508816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600abcf0 msgr2=0x7fd8601408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600abcf0 0x7fd8601408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 -- 172.21.15.97:0/3102508816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480094d0 con 0x7fd8600cb810 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.174+0000 7fd85f400640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 0x7fd860151350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd848000c00 tx=0x7fd8480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.175+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84803ac60 con 0x7fd8600cb810 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.175+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84803adc0 con 0x7fd8600cb810 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.175+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86015e0e0 con 0x7fd8600cb810 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.175+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848044630 con 0x7fd8600cb810 2024-01-31T05:01:43.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.175+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86009a960 con 0x7fd8600cb810 2024-01-31T05:01:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.176+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd860067830 con 0x7fd8600cb810 2024-01-31T05:01:43.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.177+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd84802f050 con 0x7fd8600cb810 2024-01-31T05:01:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.178+0000 7fd857e00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd83806aed0 0x7fd83806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.178+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd8480ab8f0 con 0x7fd8600cb810 2024-01-31T05:01:43.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.178+0000 7fd85fe00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd83806aed0 0x7fd83806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.178+0000 7fd85fe00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd83806aed0 0x7fd83806d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd85000a870 tx=0x7fd850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:43.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.182+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd848076500 con 0x7fd8600cb810 2024-01-31T05:01:43.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.374+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd86009ae10 con 0x7fd8600cb810 2024-01-31T05:01:43.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.375+0000 7fd857e00640 1 -- 172.21.15.97:0/3102508816 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd86009ae10 con 0x7fd8600cb810 2024-01-31T05:01:43.374 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:43.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.377+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd83806aed0 msgr2=0x7fd83806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.377+0000 7fd866c00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd83806aed0 0x7fd83806d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd85000a870 tx=0x7fd850008040 comp rx=0 tx=0).stop 2024-01-31T05:01:43.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.377+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 msgr2=0x7fd860151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.377+0000 7fd866c00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 0x7fd860151350 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd848000c00 tx=0x7fd8480046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 shutdown_connections 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd86015b470 0x7fd860151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd83806aed0 0x7fd83806d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8600cb810 0x7fd860151350 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 --2- 172.21.15.97:0/3102508816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8600abcf0 0x7fd8601408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 >> 172.21.15.97:0/3102508816 conn(0x7fd8600b0a10 msgr2=0x7fd860159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 shutdown_connections 2024-01-31T05:01:43.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.378+0000 7fd866c00640 1 -- 172.21.15.97:0/3102508816 wait complete. 2024-01-31T05:01:43.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:43.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.612+0000 7fef44800640 1 Processor -- start 2024-01-31T05:01:43.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.612+0000 7fef44800640 1 -- start start 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.614+0000 7fef44800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef3c0cbf20 0x7fef3c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.614+0000 7fef44800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c06cd20 con 0x7fef3c15b470 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.614+0000 7fef44800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c0ceb60 con 0x7fef3c0abcf0 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.614+0000 7fef44800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3c15b850 con 0x7fef3c0cbf20 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.614+0000 7fef41e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fef3c15b470 0x7fef3c0cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54970/0 (socket says 172.21.15.97:54970) 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.614+0000 7fef41e00640 1 -- 172.21.15.97:0/4049136401 learned_addr learned my addr 172.21.15.97:0/4049136401 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:43.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef41400640 1 --2- 172.21.15.97:0/4049136401 >> v2:172.21.15.188:3300/0 conn(0x7fef3c0cbf20 0x7fef3c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145899554 0 0) 0x7fef3c06cd20 con 0x7fef3c15b470 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef1c003520 con 0x7fef3c15b470 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 389662658 0 0) 0x7fef1c003520 con 0x7fef3c15b470 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 >> v2:172.21.15.188:3300/0 conn(0x7fef3c0cbf20 msgr2=0x7fef3c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 --2- 172.21.15.97:0/4049136401 >> v2:172.21.15.188:3300/0 conn(0x7fef3c0cbf20 0x7fef3c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 >> v1:172.21.15.188:6789/0 conn(0x7fef3c0abcf0 legacy=0x7fef3c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c06c3f0 con 0x7fef3c15b470 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fef2c0017f0 con 0x7fef3c15b470 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 >> v1:172.21.15.97:6789/0 conn(0x7fef3c15b470 legacy=0x7fef3c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef1c003970 0x7fef1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef1c0047c0 0x7fef1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c15b850 con 0x7fef1c005180 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c00c020 con 0x7fef1c0047c0 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.615+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c00c1e0 con 0x7fef1c003970 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef42800640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef42800640 1 -- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef1c003970 msgr2=0x7fef1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef42800640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef1c003970 0x7fef1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef41400640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef1c0047c0 0x7fef1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef42800640 1 -- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef1c0047c0 msgr2=0x7fef1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef42800640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef1c0047c0 0x7fef1c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.616+0000 7fef42800640 1 -- 172.21.15.97:0/4049136401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c003520 con 0x7fef1c005180 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.617+0000 7fef42800640 1 -- 172.21.15.97:0/4049136401 reap_dead start 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.617+0000 7fef42800640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef1c00b980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fef34001e20 tx=0x7fef1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c09562e01ce2a5c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.617+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34003b70 con 0x7fef1c005180 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.617+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef34003cd0 con 0x7fef1c005180 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.617+0000 7fef40a00640 1 -- 172.21.15.97:0/4049136401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34005310 con 0x7fef1c005180 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 -- 172.21.15.97:0/4049136401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 msgr2=0x7fef1c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef1c00b980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fef34001e20 tx=0x7fef1c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 -- 172.21.15.97:0/4049136401 shutdown_connections 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 --2- 172.21.15.97:0/4049136401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef1c00b980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 -- 172.21.15.97:0/4049136401 >> 172.21.15.97:0/4049136401 conn(0x7fef3c0b0a10 msgr2=0x7fef3c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 -- 172.21.15.97:0/4049136401 shutdown_connections 2024-01-31T05:01:43.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.618+0000 7fef44800640 1 -- 172.21.15.97:0/4049136401 wait complete. 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.620+0000 7fef44800640 1 Processor -- start 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.620+0000 7fef44800640 1 -- start start 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef44800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef3c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef44800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3c148300 0x7fef3c15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef44800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c144f90 0x7fef3c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef44800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c15e050 con 0x7fef1c005180 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef44800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c148760 con 0x7fef3c144f90 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef44800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c15bed0 con 0x7fef3c148300 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef41e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef3c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef41e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef3c15e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43850/0 (socket says 172.21.15.97:43850) 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.621+0000 7fef41e00640 1 -- 172.21.15.97:0/3435984589 learned_addr learned my addr 172.21.15.97:0/3435984589 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:43.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41400640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3c148300 0x7fef3c15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef42800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c144f90 0x7fef3c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41e00640 1 -- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3c148300 msgr2=0x7fef3c15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41e00640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3c148300 0x7fef3c15b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41e00640 1 -- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c144f90 msgr2=0x7fef3c159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41e00640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c144f90 0x7fef3c159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41e00640 1 -- 172.21.15.97:0/3435984589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c15d6b0 con 0x7fef1c005180 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41400640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3c148300 0x7fef3c15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef42800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c144f90 0x7fef3c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef41e00640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef3c15e470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00c730 tx=0x7fef2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c007270 con 0x7fef1c005180 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c0073d0 con 0x7fef1c005180 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0057a0 con 0x7fef1c005180 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.622+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c15d990 con 0x7fef1c005180 2024-01-31T05:01:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.623+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c15d360 con 0x7fef1c005180 2024-01-31T05:01:43.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.624+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c15e2c0 con 0x7fef1c005180 2024-01-31T05:01:43.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.624+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fef2c005900 con 0x7fef1c005180 2024-01-31T05:01:43.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.625+0000 7fef3a000640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef1406af00 0x7fef1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:43.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.625+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fef2c088f40 con 0x7fef1c005180 2024-01-31T05:01:43.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.625+0000 7fef41400640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef1406af00 0x7fef1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:43.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.626+0000 7fef41400640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef1406af00 0x7fef1406d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef3c14bfb0 tx=0x7fef24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:43.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.630+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fef2c053b70 con 0x7fef1c005180 2024-01-31T05:01:43.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.830+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef3c067830 con 0x7fef1c005180 2024-01-31T05:01:43.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.831+0000 7fef3a000640 1 -- 172.21.15.97:0/3435984589 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef2c017020 con 0x7fef1c005180 2024-01-31T05:01:43.830 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef1406af00 msgr2=0x7fef1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef1406af00 0x7fef1406d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef3c14bfb0 tx=0x7fef24008040 comp rx=0 tx=0).stop 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 msgr2=0x7fef3c15e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef3c15e470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00c730 tx=0x7fef2c00cbf0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 shutdown_connections 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3c144f90 0x7fef3c159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3c148300 0x7fef3c15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef1406af00 0x7fef1406d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 --2- 172.21.15.97:0/3435984589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c005180 0x7fef3c15e470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 >> 172.21.15.97:0/3435984589 conn(0x7fef3c0b0a10 msgr2=0x7fef3c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 shutdown_connections 2024-01-31T05:01:43.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:43.833+0000 7fef44800640 1 -- 172.21.15.97:0/3435984589 wait complete. 2024-01-31T05:01:43.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:44.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.062+0000 7fdcfac00640 1 Processor -- start 2024-01-31T05:01:44.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.062+0000 7fdcfac00640 1 -- start start 2024-01-31T05:01:44.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcfac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcf40abcf0 0x7fdcf40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcfac00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf406cd20 con 0x7fdcf40cbf20 2024-01-31T05:01:44.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcfac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf40ceb60 con 0x7fdcf415b470 2024-01-31T05:01:44.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcfac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf415b850 con 0x7fdcf40abcf0 2024-01-31T05:01:44.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcf3400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdcf40cbf20 0x7fdcf406d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54984/0 (socket says 172.21.15.97:54984) 2024-01-31T05:01:44.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcf3400640 1 -- 172.21.15.97:0/2764224077 learned_addr learned my addr 172.21.15.97:0/2764224077 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122711145 0 0) 0x7fdcf406cd20 con 0x7fdcf40cbf20 2024-01-31T05:01:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcf8c00640 1 --2- 172.21.15.97:0/2764224077 >> v2:172.21.15.188:3300/0 conn(0x7fdcf40abcf0 0x7fdcf40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.064+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd4003520 con 0x7fdcf40cbf20 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2651327867 0 0) 0x7fdcd4003520 con 0x7fdcf40cbf20 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 >> v2:172.21.15.188:3300/0 conn(0x7fdcf40abcf0 msgr2=0x7fdcf40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 --2- 172.21.15.97:0/2764224077 >> v2:172.21.15.188:3300/0 conn(0x7fdcf40abcf0 0x7fdcf40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 >> v1:172.21.15.188:6789/0 conn(0x7fdcf415b470 legacy=0x7fdcf40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf406c3f0 con 0x7fdcf40cbf20 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fdcdc0019f0 con 0x7fdcf40cbf20 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 >> v1:172.21.15.97:6789/0 conn(0x7fdcf40cbf20 legacy=0x7fdcf406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd4003970 0x7fdcd4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd40047c0 0x7fdcd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd4005180 0x7fdcd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf415b850 con 0x7fdcd4003970 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd400c0d0 con 0x7fdcd4005180 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd400c290 con 0x7fdcd40047c0 2024-01-31T05:01:44.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf8c00640 1 --2- 172.21.15.97:0/2764224077 >> v2:172.21.15.188:3300/0 conn(0x7fdcf40abcf0 0x7fdcf40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd4003970 0x7fdcd4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf3e00640 1 -- 172.21.15.97:0/2764224077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd40047c0 msgr2=0x7fdcd4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.065+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd40047c0 0x7fdcd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.066+0000 7fdcf3e00640 1 -- 172.21.15.97:0/2764224077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd4005180 msgr2=0x7fdcd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.066+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcd4005180 0x7fdcd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.066+0000 7fdcf3e00640 1 -- 172.21.15.97:0/2764224077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4003520 con 0x7fdcd4003970 2024-01-31T05:01:44.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.066+0000 7fdcf8c00640 1 -- 172.21.15.97:0/2764224077 reap_dead start 2024-01-31T05:01:44.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.067+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd4003970 0x7fdcd4003d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdce4001770 tx=0x7fdcd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c997c51cefd4b49 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.067+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce40052d0 con 0x7fdcd4003970 2024-01-31T05:01:44.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.067+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce402e050 con 0x7fdcd4003970 2024-01-31T05:01:44.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.067+0000 7fdcf2a00640 1 -- 172.21.15.97:0/2764224077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003640 con 0x7fdcd4003970 2024-01-31T05:01:44.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.067+0000 7fdcfac00640 1 -- 172.21.15.97:0/2764224077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd4003970 msgr2=0x7fdcd4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.067+0000 7fdcfac00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd4003970 0x7fdcd4003d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdce4001770 tx=0x7fdcd40057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.068+0000 7fdcfac00640 1 -- 172.21.15.97:0/2764224077 shutdown_connections 2024-01-31T05:01:44.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.068+0000 7fdcfac00640 1 --2- 172.21.15.97:0/2764224077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd4003970 0x7fdcd4003d90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.068+0000 7fdcfac00640 1 -- 172.21.15.97:0/2764224077 >> 172.21.15.97:0/2764224077 conn(0x7fdcf40b0a10 msgr2=0x7fdcf40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:44.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.068+0000 7fdcfac00640 1 -- 172.21.15.97:0/2764224077 shutdown_connections 2024-01-31T05:01:44.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.068+0000 7fdcfac00640 1 -- 172.21.15.97:0/2764224077 wait complete. 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.069+0000 7fdcfac00640 1 Processor -- start 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.069+0000 7fdcfac00640 1 -- start start 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.070+0000 7fdcfac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd4003970 0x7fdcf41502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.070+0000 7fdcfac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf415d5b0 0x7fdcf415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.070+0000 7fdcfac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 0x7fdcf40b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.070+0000 7fdcfac00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40b3640 con 0x7fdcf40b2c80 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.070+0000 7fdcfac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf415e610 con 0x7fdcf415d5b0 2024-01-31T05:01:44.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.070+0000 7fdcfac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40c8560 con 0x7fdcd4003970 2024-01-31T05:01:44.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 0x7fdcf40b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 0x7fdcf40b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43868/0 (socket says 172.21.15.97:43868) 2024-01-31T05:01:44.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 -- 172.21.15.97:0/3365574036 learned_addr learned my addr 172.21.15.97:0/3365574036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:44.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd4003970 0x7fdcf41502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf3400640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf415d5b0 0x7fdcf415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 -- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd4003970 msgr2=0x7fdcf41502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd4003970 0x7fdcf41502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 -- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf415d5b0 msgr2=0x7fdcf415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf415d5b0 0x7fdcf415da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 -- 172.21.15.97:0/3365574036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40c8230 con 0x7fdcf40b2c80 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.071+0000 7fdcf8c00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 0x7fdcf40b3100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40700b0 tx=0x7fdcec00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.072+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec00c070 con 0x7fdcf40b2c80 2024-01-31T05:01:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.072+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcec013610 con 0x7fdcf40b2c80 2024-01-31T05:01:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.072+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0139c0 con 0x7fdcf40b2c80 2024-01-31T05:01:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.072+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf415e050 con 0x7fdcf40b2c80 2024-01-31T05:01:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.072+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf4150d20 con 0x7fdcf40b2c80 2024-01-31T05:01:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.073+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf40c8090 con 0x7fdcf40b2c80 2024-01-31T05:01:44.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.074+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fdcec013b20 con 0x7fdcf40b2c80 2024-01-31T05:01:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.075+0000 7fdcebe00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdccc06af00 0x7fdccc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.075+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdcec088ee0 con 0x7fdcf40b2c80 2024-01-31T05:01:44.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.075+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdccc06af00 0x7fdccc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.076+0000 7fdcf3e00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdccc06af00 0x7fdccc06d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdce4033000 tx=0x7fdce4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.079+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fdcf40c8090 con 0x7fdcf40b2c80 2024-01-31T05:01:44.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.272+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcf40c8090 con 0x7fdcf40b2c80 2024-01-31T05:01:44.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.273+0000 7fdcebe00640 1 -- 172.21.15.97:0/3365574036 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcf40c8090 con 0x7fdcf40b2c80 2024-01-31T05:01:44.272 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.275+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdccc06af00 msgr2=0x7fdccc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.275+0000 7fdcfac00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdccc06af00 0x7fdccc06d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdce4033000 tx=0x7fdce4009030 comp rx=0 tx=0).stop 2024-01-31T05:01:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.275+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 msgr2=0x7fdcf40b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.275+0000 7fdcfac00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 0x7fdcf40b3100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40700b0 tx=0x7fdcec00e830 comp rx=0 tx=0).stop 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 shutdown_connections 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdccc06af00 0x7fdccc06d3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcf40b2c80 0x7fdcf40b3100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcf415d5b0 0x7fdcf415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 --2- 172.21.15.97:0/3365574036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcd4003970 0x7fdcf41502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 >> 172.21.15.97:0/3365574036 conn(0x7fdcf40b0a10 msgr2=0x7fdcf40ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 shutdown_connections 2024-01-31T05:01:44.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.276+0000 7fdcfac00640 1 -- 172.21.15.97:0/3365574036 wait complete. 2024-01-31T05:01:44.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:44.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.503+0000 7f3291000640 1 Processor -- start 2024-01-31T05:01:44.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.503+0000 7f3291000640 1 -- start start 2024-01-31T05:01:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.505+0000 7f3291000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.505+0000 7f3291000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 0x7f328c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.505+0000 7f3291000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328c06cd20 con 0x7f328c0abcf0 2024-01-31T05:01:44.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.505+0000 7f3291000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328c0ce970 con 0x7f328c0cbf20 2024-01-31T05:01:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.505+0000 7f3291000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c15b850 con 0x7f328c15b470 2024-01-31T05:01:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f328a800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 0x7f328c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f3289e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f328a800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 0x7f328c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:42238/0 (socket says 172.21.15.97:42238) 2024-01-31T05:01:44.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f328a800640 1 -- 172.21.15.97:0/1322333415 learned_addr learned my addr 172.21.15.97:0/1322333415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f3289e00640 1 -- 172.21.15.97:0/1322333415 >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 msgr2=0x7f328c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f3289e00640 1 --2- 172.21.15.97:0/1322333415 >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 0x7f328c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f3289e00640 1 -- 172.21.15.97:0/1322333415 >> v1:172.21.15.188:6789/0 conn(0x7f328c0abcf0 legacy=0x7f328c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f3289e00640 1 -- 172.21.15.97:0/1322333415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0ceb60 con 0x7f328c15b470 2024-01-31T05:01:44.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.506+0000 7f328a800640 1 --2- 172.21.15.97:0/1322333415 >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 0x7f328c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:01:44.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.508+0000 7f3289e00640 1 --2- 172.21.15.97:0/1322333415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c0722c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e0ca2d9c71676 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.508+0000 7f3288a00640 1 -- 172.21.15.97:0/1322333415 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c031800 con 0x7f328c15b470 2024-01-31T05:01:44.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.508+0000 7f3288a00640 1 -- 172.21.15.97:0/1322333415 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c031960 con 0x7f328c15b470 2024-01-31T05:01:44.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.508+0000 7f3288a00640 1 -- 172.21.15.97:0/1322333415 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c031cf0 con 0x7f328c15b470 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 -- 172.21.15.97:0/1322333415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 msgr2=0x7f328c0722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 --2- 172.21.15.97:0/1322333415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c0722c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c02b8a0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 -- 172.21.15.97:0/1322333415 shutdown_connections 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 --2- 172.21.15.97:0/1322333415 >> v2:172.21.15.188:3300/0 conn(0x7f328c0cbf20 0x7f328c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 --2- 172.21.15.97:0/1322333415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c0722c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 -- 172.21.15.97:0/1322333415 >> 172.21.15.97:0/1322333415 conn(0x7f328c0b0a10 msgr2=0x7f328c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 -- 172.21.15.97:0/1322333415 shutdown_connections 2024-01-31T05:01:44.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.509+0000 7f3291000640 1 -- 172.21.15.97:0/1322333415 wait complete. 2024-01-31T05:01:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.511+0000 7f3291000640 1 Processor -- start 2024-01-31T05:01:44.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.511+0000 7f3291000640 1 -- start start 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3291000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f328c0abcf0 0x7f328c09a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3291000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 0x7f328c14ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3291000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3291000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c1e3a50 con 0x7f328c0cbf20 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3291000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c09aab0 con 0x7f328c0abcf0 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3291000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328c09ad90 con 0x7f328c15b470 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.512+0000 7f3289400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 0x7f328c14ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 0x7f328c14ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43884/0 (socket says 172.21.15.97:43884) 2024-01-31T05:01:44.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 -- 172.21.15.97:0/257004650 learned_addr learned my addr 172.21.15.97:0/257004650 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f328a800640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289e00640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f328c0abcf0 0x7f328c09a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 -- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 msgr2=0x7f328c147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 -- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f328c0abcf0 msgr2=0x7f328c09a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f328c0abcf0 0x7f328c09a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 -- 172.21.15.97:0/257004650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284009b30 con 0x7f328c0cbf20 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f328a800640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289e00640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f328c0abcf0 0x7f328c09a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.513+0000 7f3289400640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 0x7f328c14ae10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f327400c440 tx=0x7f3274001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.514+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32740052d0 con 0x7f328c0cbf20 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.514+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3274005430 con 0x7f328c0cbf20 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.514+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274007260 con 0x7f328c0cbf20 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.514+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c0093a0 con 0x7f328c0cbf20 2024-01-31T05:01:44.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.514+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c09b350 con 0x7f328c0cbf20 2024-01-31T05:01:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.515+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c06d190 con 0x7f328c0cbf20 2024-01-31T05:01:44.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.516+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f32740073c0 con 0x7f328c0cbf20 2024-01-31T05:01:44.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.517+0000 7f3282000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f326406aed0 0x7f326406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.517+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3274088e60 con 0x7f328c0cbf20 2024-01-31T05:01:44.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.517+0000 7f3289e00640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f326406aed0 0x7f326406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.518+0000 7f3289e00640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f326406aed0 0x7f326406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.521+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f328c06d190 con 0x7f328c0cbf20 2024-01-31T05:01:44.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.714+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f328c06d190 con 0x7f328c0cbf20 2024-01-31T05:01:44.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.716+0000 7f3282000640 1 -- 172.21.15.97:0/257004650 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f328c06d190 con 0x7f328c0cbf20 2024-01-31T05:01:44.715 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f326406aed0 msgr2=0x7f326406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f326406aed0 0x7f326406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f327c0097c0 tx=0x7f327c03c040 comp rx=0 tx=0).stop 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 msgr2=0x7f328c14ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 0x7f328c14ae10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f327400c440 tx=0x7f3274001e70 comp rx=0 tx=0).stop 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 shutdown_connections 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f328c15b470 0x7f328c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f326406aed0 0x7f326406d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f328c0cbf20 0x7f328c14ae10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 --2- 172.21.15.97:0/257004650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f328c0abcf0 0x7f328c09a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 >> 172.21.15.97:0/257004650 conn(0x7f328c0b0a10 msgr2=0x7f328c15a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:44.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 shutdown_connections 2024-01-31T05:01:44.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.718+0000 7f3291000640 1 -- 172.21.15.97:0/257004650 wait complete. 2024-01-31T05:01:44.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.949+0000 7ff106400640 1 Processor -- start 2024-01-31T05:01:44.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.949+0000 7ff106400640 1 -- start start 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff106400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff1000b1d10 0x7ff1000b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff106400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 0x7ff10015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff106400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1000b2630 con 0x7ff10015a600 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff106400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10006c8b0 con 0x7ff1000b1d10 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff106400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015b120 con 0x7ff10006e0a0 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff0f6800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff10015a600 0x7ff10015dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36768/0 (socket says 172.21.15.97:36768) 2024-01-31T05:01:44.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.951+0000 7ff0f6800640 1 -- 172.21.15.97:0/4046979799 learned_addr learned my addr 172.21.15.97:0/4046979799 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0ff400640 1 --2- 172.21.15.97:0/4046979799 >> v2:172.21.15.188:3300/0 conn(0x7ff1000b1d10 0x7ff1000b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0ffe00640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 0x7ff10015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068196907 0 0) 0x7ff1000b2630 con 0x7ff10015a600 2024-01-31T05:01:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e0003520 con 0x7ff10015a600 2024-01-31T05:01:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3655693439 0 0) 0x7ff0e0003520 con 0x7ff10015a600 2024-01-31T05:01:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 msgr2=0x7ff10015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 0x7ff10015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 >> v2:172.21.15.188:3300/0 conn(0x7ff1000b1d10 msgr2=0x7ff1000b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 --2- 172.21.15.97:0/4046979799 >> v2:172.21.15.188:3300/0 conn(0x7ff1000b1d10 0x7ff1000b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10015b400 con 0x7ff10015a600 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff0e8001670 con 0x7ff10015a600 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 >> v1:172.21.15.97:6789/0 conn(0x7ff10015a600 legacy=0x7ff10015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 0x7ff0e0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.952+0000 7ff0fea00640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff0e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0fea00640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e0004d00 0x7ff0e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10006c8b0 con 0x7ff0e0003ef0 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015b120 con 0x7ff0e0004d00 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e000b980 con 0x7ff10006e0a0 2024-01-31T05:01:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0f6800640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff0e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0ff400640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 0x7ff0e0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0ffe00640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e0004d00 0x7ff0e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0f6800640 1 -- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 msgr2=0x7ff0e0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0f6800640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff10006e0a0 0x7ff0e0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0f6800640 1 -- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e0004d00 msgr2=0x7ff0e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0f6800640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0e0004d00 0x7ff0e0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.953+0000 7ff0f6800640 1 -- 172.21.15.97:0/4046979799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0003520 con 0x7ff0e0003ef0 2024-01-31T05:01:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.954+0000 7ff0f6800640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff0e00047c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800c240 tx=0x7ff0e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ab2d7123bff93 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e802cae0 con 0x7ff0e0003ef0 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e802cc40 con 0x7ff0e0003ef0 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff0fea00640 1 -- 172.21.15.97:0/4046979799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8005af0 con 0x7ff0e0003ef0 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff106400640 1 -- 172.21.15.97:0/4046979799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 msgr2=0x7ff0e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff106400640 1 --2- 172.21.15.97:0/4046979799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff0e00047c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800c240 tx=0x7ff0e00057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff0ffe00640 1 -- 172.21.15.97:0/4046979799 reap_dead start 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff106400640 1 -- 172.21.15.97:0/4046979799 shutdown_connections 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff106400640 1 -- 172.21.15.97:0/4046979799 >> 172.21.15.97:0/4046979799 conn(0x7ff1000b0a10 msgr2=0x7ff1000b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff106400640 1 -- 172.21.15.97:0/4046979799 shutdown_connections 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.955+0000 7ff106400640 1 -- 172.21.15.97:0/4046979799 wait complete. 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.957+0000 7ff106400640 1 Processor -- start 2024-01-31T05:01:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.957+0000 7ff106400640 1 -- start start 2024-01-31T05:01:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.958+0000 7ff106400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff100143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.958+0000 7ff106400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100147f90 0x7ff1000af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.958+0000 7ff106400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff100148410 0x7ff1000afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.958+0000 7ff106400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100145010 con 0x7ff0e0003ef0 2024-01-31T05:01:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.958+0000 7ff106400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100145430 con 0x7ff100148410 2024-01-31T05:01:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.958+0000 7ff106400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1001489e0 con 0x7ff100147f90 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff100143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff100143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33372/0 (socket says 172.21.15.97:33372) 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 -- 172.21.15.97:0/1446050738 learned_addr learned my addr 172.21.15.97:0/1446050738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ffe00640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff100148410 0x7ff1000afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0f6800640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100147f90 0x7ff1000af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 -- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100147f90 msgr2=0x7ff1000af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100147f90 0x7ff1000af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 -- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff100148410 msgr2=0x7ff1000afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff100148410 0x7ff1000afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 -- 172.21.15.97:0/1446050738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000adb80 con 0x7ff0e0003ef0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0ff400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff100143d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0011630 tx=0x7ff0f0011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.959+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000f8c0 con 0x7ff0e0003ef0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.960+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f000fa20 con 0x7ff0e0003ef0 2024-01-31T05:01:44.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.960+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000fd90 con 0x7ff0e0003ef0 2024-01-31T05:01:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.960+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff100142440 con 0x7ff0e0003ef0 2024-01-31T05:01:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.960+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff100144950 con 0x7ff0e0003ef0 2024-01-31T05:01:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.961+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1000677c0 con 0x7ff0e0003ef0 2024-01-31T05:01:44.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.962+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff0f0015460 con 0x7ff0e0003ef0 2024-01-31T05:01:44.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.963+0000 7ff0f7e00640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff0d806aea0 0x7ff0d806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:44.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.963+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff0f0089d70 con 0x7ff0e0003ef0 2024-01-31T05:01:44.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.963+0000 7ff0f6800640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff0d806aea0 0x7ff0d806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:44.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.964+0000 7ff0f6800640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff0d806aea0 0x7ff0d806d360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8034000 tx=0x7ff0e8007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:44.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:44.967+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff0f0054980 con 0x7ff0e0003ef0 2024-01-31T05:01:45.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.163+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff10006e740 con 0x7ff0e0003ef0 2024-01-31T05:01:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.166+0000 7ff0f7e00640 1 -- 172.21.15.97:0/1446050738 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7ff0f0057fa0 con 0x7ff0e0003ef0 2024-01-31T05:01:45.166 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:45.166 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:45.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff0d806aea0 msgr2=0x7ff0d806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff0d806aea0 0x7ff0d806d360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8034000 tx=0x7ff0e8007000 comp rx=0 tx=0).stop 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 msgr2=0x7ff100143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff100143d30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0011630 tx=0x7ff0f0011af0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 shutdown_connections 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff100148410 0x7ff1000afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff100147f90 0x7ff1000af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff0d806aea0 0x7ff0d806d360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.168+0000 7ff106400640 1 --2- 172.21.15.97:0/1446050738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0e0003ef0 0x7ff100143d30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.169+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 >> 172.21.15.97:0/1446050738 conn(0x7ff1000b0a10 msgr2=0x7ff1000ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.169+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 shutdown_connections 2024-01-31T05:01:45.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.169+0000 7ff106400640 1 -- 172.21.15.97:0/1446050738 wait complete. 2024-01-31T05:01:45.186 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:45.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:45.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.399+0000 7f4bd7200640 1 Processor -- start 2024-01-31T05:01:45.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.399+0000 7f4bd7200640 1 -- start start 2024-01-31T05:01:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.400+0000 7f4bd7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd015b080 0x7f4bd00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.400+0000 7f4bd7200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bd00ae360 0x7f4bd00cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.400+0000 7f4bd7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4bd00ce4b0 0x7f4bd015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd7200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd00b41c0 con 0x7f4bd00ae360 2024-01-31T05:01:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd00ce920 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd00b3f60 con 0x7f4bd015b080 2024-01-31T05:01:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bd00ae360 0x7f4bd00cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bd00ae360 0x7f4bd00cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33378/0 (socket says 172.21.15.97:33378) 2024-01-31T05:01:45.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 -- 172.21.15.97:0/3603658269 learned_addr learned my addr 172.21.15.97:0/3603658269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3603658269 >> v2:172.21.15.188:3300/0 conn(0x7f4bd00ce4b0 0x7f4bd015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5e00640 1 --2- 172.21.15.97:0/3603658269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd015b080 0x7f4bd00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 -- 172.21.15.97:0/3603658269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd015b080 msgr2=0x7f4bd00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 --2- 172.21.15.97:0/3603658269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd015b080 0x7f4bd00b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 -- 172.21.15.97:0/3603658269 >> v2:172.21.15.188:3300/0 conn(0x7f4bd00ce4b0 msgr2=0x7f4bd015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 --2- 172.21.15.97:0/3603658269 >> v2:172.21.15.188:3300/0 conn(0x7f4bd00ce4b0 0x7f4bd015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd5400640 1 -- 172.21.15.97:0/3603658269 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd015c540 con 0x7f4bd00ae360 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.401+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3603658269 >> v2:172.21.15.188:3300/0 conn(0x7f4bd00ce4b0 0x7f4bd015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:45.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.403+0000 7f4bd5400640 1 --2- 172.21.15.97:0/3603658269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ae360 0x7f4bd00cdf70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00cfd20 tx=0x7f4bb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d054b1b78f81e0d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:45.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.403+0000 7f4bd4a00640 1 -- 172.21.15.97:0/3603658269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8031640 con 0x7f4bd00ae360 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.403+0000 7f4bd4a00640 1 -- 172.21.15.97:0/3603658269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb80317a0 con 0x7f4bd00ae360 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.403+0000 7f4bd4a00640 1 -- 172.21.15.97:0/3603658269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8031b30 con 0x7f4bd00ae360 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 -- 172.21.15.97:0/3603658269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ae360 msgr2=0x7f4bd00cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3603658269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ae360 0x7f4bd00cdf70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00cfd20 tx=0x7f4bb802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 -- 172.21.15.97:0/3603658269 shutdown_connections 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3603658269 >> v2:172.21.15.188:3300/0 conn(0x7f4bd00ce4b0 0x7f4bd015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3603658269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ae360 0x7f4bd00cdf70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3603658269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd015b080 0x7f4bd00b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 -- 172.21.15.97:0/3603658269 >> 172.21.15.97:0/3603658269 conn(0x7f4bd006ad40 msgr2=0x7f4bd00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 -- 172.21.15.97:0/3603658269 shutdown_connections 2024-01-31T05:01:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.404+0000 7f4bd7200640 1 -- 172.21.15.97:0/3603658269 wait complete. 2024-01-31T05:01:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.405+0000 7f4bd7200640 1 Processor -- start 2024-01-31T05:01:45.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.406+0000 7f4bd7200640 1 -- start start 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.406+0000 7f4bd7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00ae360 0x7f4bd01501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.406+0000 7f4bd7200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 0x7f4bd014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd015b080 0x7f4bd0159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd7200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd014fd60 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd014f150 con 0x7f4bd015b080 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd014f430 con 0x7f4bd00ae360 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 0x7f4bd014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 0x7f4bd014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33382/0 (socket says 172.21.15.97:33382) 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 -- 172.21.15.97:0/3019618499 learned_addr learned my addr 172.21.15.97:0/3019618499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5e00640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00ae360 0x7f4bd01501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd6800640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd015b080 0x7f4bd0159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 -- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00ae360 msgr2=0x7f4bd01501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00ae360 0x7f4bd01501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 -- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd015b080 msgr2=0x7f4bd0159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd015b080 0x7f4bd0159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5400640 1 -- 172.21.15.97:0/3019618499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80094d0 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.407+0000 7f4bd5e00640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00ae360 0x7f4bd01501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:45.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.408+0000 7f4bd5400640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 0x7f4bd014e9e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8000c00 tx=0x7f4bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.408+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb803ac60 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.408+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb803ade0 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.408+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8044630 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.408+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd014f710 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.408+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd0159a40 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.409+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd00676b0 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.410+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4bb802f050 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.411+0000 7f4bce000640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ba806af00 0x7f4ba806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.411+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4bb80aba60 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.411+0000 7f4bd5e00640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ba806af00 0x7f4ba806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.412+0000 7f4bd5e00640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ba806af00 0x7f4ba806d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00abfe0 tx=0x7f4bc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:45.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.415+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4bb80765d0 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.611+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bd015e470 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.611+0000 7f4bce000640 1 -- 172.21.15.97:0/3019618499 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bb8079bf0 con 0x7f4bd00ce4b0 2024-01-31T05:01:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:45.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.614+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ba806af00 msgr2=0x7f4ba806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.614+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ba806af00 0x7f4ba806d3c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00abfe0 tx=0x7f4bc0008040 comp rx=0 tx=0).stop 2024-01-31T05:01:45.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.614+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 msgr2=0x7f4bd014e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.614+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 0x7f4bd014e9e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8000c00 tx=0x7f4bb80046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 shutdown_connections 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd015b080 0x7f4bd0159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ba806af00 0x7f4ba806d3c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bd00ce4b0 0x7f4bd014e9e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 --2- 172.21.15.97:0/3019618499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00ae360 0x7f4bd01501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 >> 172.21.15.97:0/3019618499 conn(0x7f4bd006ad40 msgr2=0x7f4bd00b3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 shutdown_connections 2024-01-31T05:01:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.615+0000 7f4bd7200640 1 -- 172.21.15.97:0/3019618499 wait complete. 2024-01-31T05:01:45.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:45.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.837+0000 7f5eba000640 1 Processor -- start 2024-01-31T05:01:45.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.837+0000 7f5eba000640 1 -- start start 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.838+0000 7f5eba000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eb40cbf20 0x7f5eb40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eba000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5eb40abcf0 0x7f5eb40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eba000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb406cd20 con 0x7f5eb415b470 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eba000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb40ce970 con 0x7f5eb40abcf0 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eba000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb415b850 con 0x7f5eb40cbf20 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5eb40abcf0 0x7f5eb40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5eb40abcf0 0x7f5eb40ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33398/0 (socket says 172.21.15.97:33398) 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb3e00640 1 -- 172.21.15.97:0/2279761694 learned_addr learned my addr 172.21.15.97:0/2279761694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:45.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb2a00640 1 --2- 172.21.15.97:0/2279761694 >> v2:172.21.15.188:3300/0 conn(0x7f5eb40cbf20 0x7f5eb40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb3e00640 1 -- 172.21.15.97:0/2279761694 >> v2:172.21.15.188:3300/0 conn(0x7f5eb40cbf20 msgr2=0x7f5eb40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb3e00640 1 --2- 172.21.15.97:0/2279761694 >> v2:172.21.15.188:3300/0 conn(0x7f5eb40cbf20 0x7f5eb40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.839+0000 7f5eb3e00640 1 -- 172.21.15.97:0/2279761694 >> v1:172.21.15.188:6789/0 conn(0x7f5eb415b470 legacy=0x7f5eb40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.840+0000 7f5eb3e00640 1 -- 172.21.15.97:0/2279761694 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40ceb60 con 0x7f5eb40abcf0 2024-01-31T05:01:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.841+0000 7f5eb3e00640 1 --2- 172.21.15.97:0/2279761694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40abcf0 0x7f5eb40ce430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4072d90 tx=0x7f5eac02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f8c4806479ed48 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:45.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.841+0000 7f5eb2000640 1 -- 172.21.15.97:0/2279761694 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac02cb20 con 0x7f5eb40abcf0 2024-01-31T05:01:45.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.841+0000 7f5eb2000640 1 -- 172.21.15.97:0/2279761694 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac02cc80 con 0x7f5eb40abcf0 2024-01-31T05:01:45.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.841+0000 7f5eb2000640 1 -- 172.21.15.97:0/2279761694 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0343f0 con 0x7f5eb40abcf0 2024-01-31T05:01:45.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 -- 172.21.15.97:0/2279761694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40abcf0 msgr2=0x7f5eb40ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 --2- 172.21.15.97:0/2279761694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40abcf0 0x7f5eb40ce430 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4072d90 tx=0x7f5eac02bf80 comp rx=0 tx=0).stop 2024-01-31T05:01:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 -- 172.21.15.97:0/2279761694 shutdown_connections 2024-01-31T05:01:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 --2- 172.21.15.97:0/2279761694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40abcf0 0x7f5eb40ce430 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 --2- 172.21.15.97:0/2279761694 >> v2:172.21.15.188:3300/0 conn(0x7f5eb40cbf20 0x7f5eb40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 -- 172.21.15.97:0/2279761694 >> 172.21.15.97:0/2279761694 conn(0x7f5eb40b0a10 msgr2=0x7f5eb40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 -- 172.21.15.97:0/2279761694 shutdown_connections 2024-01-31T05:01:45.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.842+0000 7f5eba000640 1 -- 172.21.15.97:0/2279761694 wait complete. 2024-01-31T05:01:45.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.844+0000 7f5eba000640 1 Processor -- start 2024-01-31T05:01:45.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.844+0000 7f5eba000640 1 -- start start 2024-01-31T05:01:45.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eba000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb40abcf0 0x7f5eb4143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eba000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 0x7f5eb41531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eba000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb415b470 0x7f5eb4159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eba000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4153c40 con 0x7f5eb40cbf20 2024-01-31T05:01:45.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eba000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4153f20 con 0x7f5eb40abcf0 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eba000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4154200 con 0x7f5eb415b470 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 0x7f5eb41531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 0x7f5eb41531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33406/0 (socket says 172.21.15.97:33406) 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 -- 172.21.15.97:0/1598351128 learned_addr learned my addr 172.21.15.97:0/1598351128 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb3400640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb40abcf0 0x7f5eb4143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb3e00640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb415b470 0x7f5eb4159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 -- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb415b470 msgr2=0x7f5eb4159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb415b470 0x7f5eb4159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 -- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb40abcf0 msgr2=0x7f5eb4143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb40abcf0 0x7f5eb4143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.845+0000 7f5eb2a00640 1 -- 172.21.15.97:0/1598351128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac009b30 con 0x7f5eb40cbf20 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.846+0000 7f5eb2a00640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 0x7f5eb41531f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00d800 tx=0x7f5e9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.846+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00b6c0 con 0x7f5eb40cbf20 2024-01-31T05:01:45.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.846+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c00b820 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.846+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb41544e0 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.846+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00bbb0 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.846+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb41443f0 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.847+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb4067830 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.848+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5e9c00bd10 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.849+0000 7f5eab400640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5e8c06aed0 0x7f5e8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.849+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5e9c089fe0 con 0x7f5eb40cbf20 2024-01-31T05:01:45.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.849+0000 7f5eb3400640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5e8c06aed0 0x7f5e8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:45.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.850+0000 7f5eb3400640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5e8c06aed0 0x7f5e8c06d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4001630 tx=0x7f5ea4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:45.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:45.853+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5e9c054a70 con 0x7f5eb40cbf20 2024-01-31T05:01:46.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.033+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5eb40cf840 con 0x7f5eb40cbf20 2024-01-31T05:01:46.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.034+0000 7f5eab400640 1 -- 172.21.15.97:0/1598351128 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e9c058090 con 0x7f5eb40cbf20 2024-01-31T05:01:46.034 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:46.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5e8c06aed0 msgr2=0x7f5e8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5e8c06aed0 0x7f5e8c06d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4001630 tx=0x7f5ea4009040 comp rx=0 tx=0).stop 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 msgr2=0x7f5eb41531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 0x7f5eb41531f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00d800 tx=0x7f5e9c00dcc0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 shutdown_connections 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb415b470 0x7f5eb4159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5e8c06aed0 0x7f5e8c06d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5eb40cbf20 0x7f5eb41531f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 --2- 172.21.15.97:0/1598351128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb40abcf0 0x7f5eb4143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 >> 172.21.15.97:0/1598351128 conn(0x7f5eb40b0a10 msgr2=0x7f5eb40cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 shutdown_connections 2024-01-31T05:01:46.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.037+0000 7f5eba000640 1 -- 172.21.15.97:0/1598351128 wait complete. 2024-01-31T05:01:46.056 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.261+0000 7fe514a00640 1 Processor -- start 2024-01-31T05:01:46.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.261+0000 7fe514a00640 1 -- start start 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe514a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe514a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe50c0cb7d0 0x7fe50c0cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe514a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50c161fc0 con 0x7fe50c0ce1e0 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe514a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50c162c40 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe514a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1622a0 con 0x7fe50c161b10 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe512a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe50c0cb7d0 0x7fe50c0cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe512000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe50c0ce1e0 0x7fe50c0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36774/0 (socket says 172.21.15.97:36774) 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.263+0000 7fe512000640 1 -- 172.21.15.97:0/2874636895 learned_addr learned my addr 172.21.15.97:0/2874636895 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:46.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.264+0000 7fe511600640 1 --2- 172.21.15.97:0/2874636895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.264+0000 7fe510c00640 1 -- 172.21.15.97:0/2874636895 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654033740 0 0) 0x7fe50c161fc0 con 0x7fe50c0ce1e0 2024-01-31T05:01:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.264+0000 7fe512a00640 1 -- 172.21.15.97:0/2874636895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 msgr2=0x7fe50c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.264+0000 7fe512a00640 1 --2- 172.21.15.97:0/2874636895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.264+0000 7fe512a00640 1 -- 172.21.15.97:0/2874636895 >> v1:172.21.15.97:6789/0 conn(0x7fe50c0ce1e0 legacy=0x7fe50c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.264+0000 7fe512a00640 1 -- 172.21.15.97:0/2874636895 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c162580 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.265+0000 7fe512a00640 1 --2- 172.21.15.97:0/2874636895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0cbc80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097a0 tx=0x7fe50402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5da9c516f32b261 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe510c00640 1 -- 172.21.15.97:0/2874636895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50402e070 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe510c00640 1 -- 172.21.15.97:0/2874636895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5040315a0 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe510c00640 1 -- 172.21.15.97:0/2874636895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504031930 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 -- 172.21.15.97:0/2874636895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 msgr2=0x7fe50c0cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 --2- 172.21.15.97:0/2874636895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0cbc80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097a0 tx=0x7fe50402b840 comp rx=0 tx=0).stop 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 -- 172.21.15.97:0/2874636895 shutdown_connections 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 --2- 172.21.15.97:0/2874636895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0cbc80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 --2- 172.21.15.97:0/2874636895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 -- 172.21.15.97:0/2874636895 >> 172.21.15.97:0/2874636895 conn(0x7fe50c0b0a10 msgr2=0x7fe50c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:46.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 -- 172.21.15.97:0/2874636895 shutdown_connections 2024-01-31T05:01:46.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.266+0000 7fe514a00640 1 -- 172.21.15.97:0/2874636895 wait complete. 2024-01-31T05:01:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.268+0000 7fe514a00640 1 Processor -- start 2024-01-31T05:01:46.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.269+0000 7fe514a00640 1 -- start start 2024-01-31T05:01:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe514a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe514a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe50c0ce1e0 0x7fe50c09a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe514a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c09a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe514a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c0caa80 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe514a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c071a10 con 0x7fe50c0ce1e0 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe514a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c071cf0 con 0x7fe50c161b10 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33418/0 (socket says 172.21.15.97:33418) 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 -- 172.21.15.97:0/540750032 learned_addr learned my addr 172.21.15.97:0/540750032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 -- 172.21.15.97:0/540750032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 msgr2=0x7fe50c09a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c09a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 -- 172.21.15.97:0/540750032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe50c0ce1e0 msgr2=0x7fe50c09a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe50c0ce1e0 0x7fe50c09a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.270+0000 7fe512000640 1 -- 172.21.15.97:0/540750032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009400 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.271+0000 7fe512000640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0ca540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c840 tx=0x7fe4fc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.271+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00a640 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.271+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc00a7a0 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.271+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50c0ac4a0 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.271+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00ab50 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.271+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c0aca60 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.272+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c0acc80 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.273+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe4fc00acb0 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.274+0000 7fe50a000640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe4e406aed0 0x7fe4e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.275+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe4fc089900 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.275+0000 7fe511600640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe4e406aed0 0x7fe4e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.275+0000 7fe511600640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe4e406aed0 0x7fe4e406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe50c15edb0 tx=0x7fe4f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:46.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.279+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe4fc0543e0 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.472+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe50c067830 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.472+0000 7fe50a000640 1 -- 172.21.15.97:0/540750032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4fc057a00 con 0x7fe50c0cb7d0 2024-01-31T05:01:46.472 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe4e406aed0 msgr2=0x7fe4e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe4e406aed0 0x7fe4e406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe50c15edb0 tx=0x7fe4f4008040 comp rx=0 tx=0).stop 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 msgr2=0x7fe50c0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0ca540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c840 tx=0x7fe4fc00cd00 comp rx=0 tx=0).stop 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 shutdown_connections 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe50c161b10 0x7fe50c09a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe50c0ce1e0 0x7fe50c09a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe4e406aed0 0x7fe4e406d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 --2- 172.21.15.97:0/540750032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe50c0cb7d0 0x7fe50c0ca540 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.475+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 >> 172.21.15.97:0/540750032 conn(0x7fe50c0b0a10 msgr2=0x7fe50c0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.476+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 shutdown_connections 2024-01-31T05:01:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.476+0000 7fe514a00640 1 -- 172.21.15.97:0/540750032 wait complete. 2024-01-31T05:01:46.494 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:46.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.702+0000 7fe8d9e00640 1 Processor -- start 2024-01-31T05:01:46.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.703+0000 7fe8d9e00640 1 -- start start 2024-01-31T05:01:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d9e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8d40cbf20 0x7fe8d40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d9e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8d40abcf0 0x7fe8d40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d9e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d406cd20 con 0x7fe8d415b470 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d9e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d40ce970 con 0x7fe8d40cbf20 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d9e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d415b850 con 0x7fe8d40abcf0 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8cbe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8d40cbf20 0x7fe8d40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d2a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe8d415b470 0x7fe8d40cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36780/0 (socket says 172.21.15.97:36780) 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8d2a00640 1 -- 172.21.15.97:0/3787679655 learned_addr learned my addr 172.21.15.97:0/3787679655 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.704+0000 7fe8cbe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8d40cbf20 0x7fe8d40722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33426/0 (socket says 172.21.15.97:33426) 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.705+0000 7fe8d3400640 1 --2- 172.21.15.97:0/3787679655 >> v2:172.21.15.188:3300/0 conn(0x7fe8d40abcf0 0x7fe8d40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.705+0000 7fe8cbe00640 1 -- 172.21.15.97:0/3787679655 >> v2:172.21.15.188:3300/0 conn(0x7fe8d40abcf0 msgr2=0x7fe8d40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.705+0000 7fe8cbe00640 1 --2- 172.21.15.97:0/3787679655 >> v2:172.21.15.188:3300/0 conn(0x7fe8d40abcf0 0x7fe8d40ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.705+0000 7fe8cbe00640 1 -- 172.21.15.97:0/3787679655 >> v1:172.21.15.97:6789/0 conn(0x7fe8d415b470 legacy=0x7fe8d40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.705+0000 7fe8cbe00640 1 -- 172.21.15.97:0/3787679655 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40ceb60 con 0x7fe8d40cbf20 2024-01-31T05:01:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.705+0000 7fe8d2000640 1 -- 172.21.15.97:0/3787679655 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009851369 0 0) 0x7fe8d406cd20 con 0x7fe8d415b470 2024-01-31T05:01:46.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.706+0000 7fe8cbe00640 1 --2- 172.21.15.97:0/3787679655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d40722c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40711b0 tx=0x7fe8bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd31056c05f54898 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.706+0000 7fe8d2000640 1 -- 172.21.15.97:0/3787679655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc031830 con 0x7fe8d40cbf20 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.706+0000 7fe8d2000640 1 -- 172.21.15.97:0/3787679655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc031990 con 0x7fe8d40cbf20 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.706+0000 7fe8d2000640 1 -- 172.21.15.97:0/3787679655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc031d20 con 0x7fe8d40cbf20 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 -- 172.21.15.97:0/3787679655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 msgr2=0x7fe8d40722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/3787679655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d40722c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40711b0 tx=0x7fe8bc02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 -- 172.21.15.97:0/3787679655 shutdown_connections 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/3787679655 >> v2:172.21.15.188:3300/0 conn(0x7fe8d40abcf0 0x7fe8d40ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/3787679655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d40722c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 -- 172.21.15.97:0/3787679655 >> 172.21.15.97:0/3787679655 conn(0x7fe8d40b0a10 msgr2=0x7fe8d40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 -- 172.21.15.97:0/3787679655 shutdown_connections 2024-01-31T05:01:46.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.707+0000 7fe8d9e00640 1 -- 172.21.15.97:0/3787679655 wait complete. 2024-01-31T05:01:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.709+0000 7fe8d9e00640 1 Processor -- start 2024-01-31T05:01:46.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.709+0000 7fe8d9e00640 1 -- start start 2024-01-31T05:01:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d40abcf0 0x7fe8d415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8d9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8d415b470 0x7fe8d40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8d9e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d40b3dd0 con 0x7fe8d40cbf20 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8d9e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415bfb0 con 0x7fe8d40abcf0 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8d9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415c290 con 0x7fe8d415b470 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33430/0 (socket says 172.21.15.97:33430) 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 -- 172.21.15.97:0/1715002014 learned_addr learned my addr 172.21.15.97:0/1715002014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 -- 172.21.15.97:0/1715002014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8d415b470 msgr2=0x7fe8d40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8d415b470 0x7fe8d40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 -- 172.21.15.97:0/1715002014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d40abcf0 msgr2=0x7fe8d415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d40abcf0 0x7fe8d415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.710+0000 7fe8cbe00640 1 -- 172.21.15.97:0/1715002014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc009400 con 0x7fe8d40cbf20 2024-01-31T05:01:46.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.711+0000 7fe8cbe00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d414a9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0098c0 tx=0x7fe8bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.711+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc03ba70 con 0x7fe8d40cbf20 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.711+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc03bbf0 con 0x7fe8d40cbf20 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.711+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d415c570 con 0x7fe8d40cbf20 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.711+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0453f0 con 0x7fe8d40cbf20 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.711+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d415cb30 con 0x7fe8d40cbf20 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.712+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d4067140 con 0x7fe8d40cbf20 2024-01-31T05:01:46.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.713+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe8bc04e450 con 0x7fe8d40cbf20 2024-01-31T05:01:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.714+0000 7fe8cb400640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8ac06af00 0x7fe8ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.714+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe8bc0abb00 con 0x7fe8d40cbf20 2024-01-31T05:01:46.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.714+0000 7fe8d2a00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8ac06af00 0x7fe8ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:46.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.715+0000 7fe8d2a00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8ac06af00 0x7fe8ac06d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4001520 tx=0x7fe8c4007910 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:46.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.718+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe8bc0767a0 con 0x7fe8d40cbf20 2024-01-31T05:01:46.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.911+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8d415cd20 con 0x7fe8d40cbf20 2024-01-31T05:01:46.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.912+0000 7fe8cb400640 1 -- 172.21.15.97:0/1715002014 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe8bc035020 con 0x7fe8d40cbf20 2024-01-31T05:01:46.912 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8ac06af00 msgr2=0x7fe8ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8ac06af00 0x7fe8ac06d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4001520 tx=0x7fe8c4007910 comp rx=0 tx=0).stop 2024-01-31T05:01:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 msgr2=0x7fe8d414a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d414a9d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0098c0 tx=0x7fe8bc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 shutdown_connections 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8d415b470 0x7fe8d40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8ac06af00 0x7fe8ac06d3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8d40cbf20 0x7fe8d414a9d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 --2- 172.21.15.97:0/1715002014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8d40abcf0 0x7fe8d415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 >> 172.21.15.97:0/1715002014 conn(0x7fe8d40b0a10 msgr2=0x7fe8d40cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 shutdown_connections 2024-01-31T05:01:46.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:46.915+0000 7fe8d9e00640 1 -- 172.21.15.97:0/1715002014 wait complete. 2024-01-31T05:01:46.959 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-01-31T05:01:46.959 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-31T05:01:46.959 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:47.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.170+0000 7f2493600640 1 Processor -- start 2024-01-31T05:01:47.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.170+0000 7f2493600640 1 -- start start 2024-01-31T05:01:47.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f2493600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f248c0ccd00 0x7f248c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f2493600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c06d1a0 con 0x7f248c15b400 2024-01-31T05:01:47.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f2493600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c06c600 con 0x7f248c0cb810 2024-01-31T05:01:47.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f2493600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0cc960 con 0x7f248c0ccd00 2024-01-31T05:01:47.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f248be00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f248c15b400 0x7f248c06d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36792/0 (socket says 172.21.15.97:36792) 2024-01-31T05:01:47.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f248be00640 1 -- 172.21.15.97:0/3354832683 learned_addr learned my addr 172.21.15.97:0/3354832683 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:47.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.172+0000 7f2491600640 1 --2- 172.21.15.97:0/3354832683 >> v2:172.21.15.97:3300/0 conn(0x7f248c0ccd00 0x7f248c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.173+0000 7f248b400640 1 -- 172.21.15.97:0/3354832683 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173872581 0 0) 0x7f248c06d1a0 con 0x7f248c15b400 2024-01-31T05:01:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.173+0000 7f248b400640 1 -- 172.21.15.97:0/3354832683 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f246c003520 con 0x7f248c15b400 2024-01-31T05:01:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.173+0000 7f2491600640 1 -- 172.21.15.97:0/3354832683 >> v1:172.21.15.188:6789/0 conn(0x7f248c0cb810 legacy=0x7f248c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.173+0000 7f2491600640 1 -- 172.21.15.97:0/3354832683 >> v1:172.21.15.97:6789/0 conn(0x7f248c15b400 legacy=0x7f248c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.173+0000 7f2491600640 1 -- 172.21.15.97:0/3354832683 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0cd6d0 con 0x7f248c0ccd00 2024-01-31T05:01:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.174+0000 7f2491600640 1 --2- 172.21.15.97:0/3354832683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c0ccd00 0x7f248c0cd190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2484009d90 tx=0x7f248402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8969008e73a56d2f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.174+0000 7f248b400640 1 -- 172.21.15.97:0/3354832683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248402cc60 con 0x7f248c0ccd00 2024-01-31T05:01:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.174+0000 7f248b400640 1 -- 172.21.15.97:0/3354832683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248402cdc0 con 0x7f248c0ccd00 2024-01-31T05:01:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.174+0000 7f248b400640 1 -- 172.21.15.97:0/3354832683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484034640 con 0x7f248c0ccd00 2024-01-31T05:01:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 -- 172.21.15.97:0/3354832683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c0ccd00 msgr2=0x7f248c0cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 --2- 172.21.15.97:0/3354832683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c0ccd00 0x7f248c0cd190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2484009d90 tx=0x7f248402beb0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 -- 172.21.15.97:0/3354832683 shutdown_connections 2024-01-31T05:01:47.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 --2- 172.21.15.97:0/3354832683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c0ccd00 0x7f248c0cd190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 -- 172.21.15.97:0/3354832683 >> 172.21.15.97:0/3354832683 conn(0x7f248c0b0a10 msgr2=0x7f248c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:47.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 -- 172.21.15.97:0/3354832683 shutdown_connections 2024-01-31T05:01:47.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.175+0000 7f2493600640 1 -- 172.21.15.97:0/3354832683 wait complete. 2024-01-31T05:01:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.177+0000 7f2493600640 1 Processor -- start 2024-01-31T05:01:47.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.177+0000 7f2493600640 1 -- start start 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2493600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f248c0cb810 0x7f248c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2493600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248c0ccd00 0x7f248c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2493600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 0x7f248c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2493600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c159f20 con 0x7f248c15b400 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2493600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0b2560 con 0x7f248c0cb810 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2493600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0b2840 con 0x7f248c0ccd00 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2491600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 0x7f248c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2491600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 0x7f248c0b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33452/0 (socket says 172.21.15.97:33452) 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2491600640 1 -- 172.21.15.97:0/3945855456 learned_addr learned my addr 172.21.15.97:0/3945855456 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:47.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f2490c00640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f248c0cb810 0x7f248c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.178+0000 7f248be00640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248c0ccd00 0x7f248c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2491600640 1 -- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248c0ccd00 msgr2=0x7f248c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2491600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248c0ccd00 0x7f248c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2491600640 1 -- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f248c0cb810 msgr2=0x7f248c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2491600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f248c0cb810 0x7f248c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2491600640 1 -- 172.21.15.97:0/3945855456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2484009a70 con 0x7f248c15b400 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2491600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 0x7f248c0b3600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2484037040 tx=0x7f2484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484040650 con 0x7f248c15b400 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24840407d0 con 0x7f248c15b400 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484040b80 con 0x7f248c15b400 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c0b2b20 con 0x7f248c15b400 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.179+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c0b30e0 con 0x7f248c15b400 2024-01-31T05:01:47.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.181+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c0677c0 con 0x7f248c15b400 2024-01-31T05:01:47.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.181+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2484040ce0 con 0x7f248c15b400 2024-01-31T05:01:47.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.182+0000 7f2488c00640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f246406af50 0x7f246406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.182+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f24840ac1e0 con 0x7f248c15b400 2024-01-31T05:01:47.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.183+0000 7f2490c00640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f246406af50 0x7f246406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.183+0000 7f2490c00640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f246406af50 0x7f246406d410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f247c004ea0 tx=0x7f247c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:47.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.187+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2484076dc0 con 0x7f248c15b400 2024-01-31T05:01:47.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.384+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f248c0ad980 con 0x7f248c15b400 2024-01-31T05:01:47.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.387+0000 7f2488c00640 1 -- 172.21.15.97:0/3945855456 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f2484009450 con 0x7f248c15b400 2024-01-31T05:01:47.387 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:47.387 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:47.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.389+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f246406af50 msgr2=0x7f246406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.389+0000 7f2493600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f246406af50 0x7f246406d410 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f247c004ea0 tx=0x7f247c009040 comp rx=0 tx=0).stop 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.389+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 msgr2=0x7f248c0b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.389+0000 7f2493600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 0x7f248c0b3600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2484037040 tx=0x7f2484004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 shutdown_connections 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f246406af50 0x7f246406d410 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f248c15b400 0x7f248c0b3600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f248c0ccd00 0x7f248c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 --2- 172.21.15.97:0/3945855456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f248c0cb810 0x7f248c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 >> 172.21.15.97:0/3945855456 conn(0x7f248c0b0a10 msgr2=0x7f248c1566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 shutdown_connections 2024-01-31T05:01:47.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.390+0000 7f2493600640 1 -- 172.21.15.97:0/3945855456 wait complete. 2024-01-31T05:01:47.410 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:47.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.624+0000 7f403d600640 1 Processor -- start 2024-01-31T05:01:47.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.624+0000 7f403d600640 1 -- start start 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f403d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40380ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f403d600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4038161020 0x7f40381614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f403d600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403815db40 con 0x7f403806d580 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f403d600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403815e7c0 con 0x7f4038161020 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f403d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403815de20 con 0x7f403815d690 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f4037400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4038161020 0x7f40381614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f4037400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4038161020 0x7f40381614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33468/0 (socket says 172.21.15.97:33468) 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f4037400640 1 -- 172.21.15.97:0/1770852272 learned_addr learned my addr 172.21.15.97:0/1770852272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:47.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.626+0000 7f4036a00640 1 --1- 172.21.15.97:0/1770852272 >> v1:172.21.15.97:6789/0 conn(0x7f403806d580 0x7f403806d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36800/0 (socket says 172.21.15.97:36800) 2024-01-31T05:01:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.627+0000 7f402de00640 1 --2- 172.21.15.97:0/1770852272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40380ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.627+0000 7f4037400640 1 -- 172.21.15.97:0/1770852272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 msgr2=0x7f40380ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.627+0000 7f4037400640 1 --2- 172.21.15.97:0/1770852272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40380ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.627+0000 7f4037400640 1 -- 172.21.15.97:0/1770852272 >> v1:172.21.15.97:6789/0 conn(0x7f403806d580 legacy=0x7f403806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.627+0000 7f4037400640 1 -- 172.21.15.97:0/1770852272 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403815e100 con 0x7f4038161020 2024-01-31T05:01:47.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.627+0000 7f4036000640 1 -- 172.21.15.97:0/1770852272 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269093232 0 0) 0x7f403815db40 con 0x7f403806d580 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.628+0000 7f4037400640 1 --2- 172.21.15.97:0/1770852272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4038161020 0x7f40381614d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4030009e50 tx=0x7f403002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf52054d666f960d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.628+0000 7f4036000640 1 -- 172.21.15.97:0/1770852272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403002cb00 con 0x7f4038161020 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.628+0000 7f4036000640 1 -- 172.21.15.97:0/1770852272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403002cc60 con 0x7f4038161020 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f4036000640 1 -- 172.21.15.97:0/1770852272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300343f0 con 0x7f4038161020 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 -- 172.21.15.97:0/1770852272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4038161020 msgr2=0x7f40381614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 --2- 172.21.15.97:0/1770852272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4038161020 0x7f40381614d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4030009e50 tx=0x7f403002bf60 comp rx=0 tx=0).stop 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 -- 172.21.15.97:0/1770852272 shutdown_connections 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 --2- 172.21.15.97:0/1770852272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4038161020 0x7f40381614d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 --2- 172.21.15.97:0/1770852272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40380ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 -- 172.21.15.97:0/1770852272 >> 172.21.15.97:0/1770852272 conn(0x7f40380b0a10 msgr2=0x7f40380b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 -- 172.21.15.97:0/1770852272 shutdown_connections 2024-01-31T05:01:47.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.629+0000 7f403d600640 1 -- 172.21.15.97:0/1770852272 wait complete. 2024-01-31T05:01:47.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.631+0000 7f403d600640 1 Processor -- start 2024-01-31T05:01:47.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.631+0000 7f403d600640 1 -- start start 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f403d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 0x7f4038070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f403d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40381f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f403d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4038161020 0x7f403809a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f403d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40381f2fa0 con 0x7f403806d580 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f403d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403809bbf0 con 0x7f4038161020 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f403d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403809ad60 con 0x7f403815d690 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f4036a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 0x7f4038070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f4036a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 0x7f4038070e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33476/0 (socket says 172.21.15.97:33476) 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.632+0000 7f4036a00640 1 -- 172.21.15.97:0/2647220440 learned_addr learned my addr 172.21.15.97:0/2647220440 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4037400640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4038161020 0x7f403809a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f402de00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40381f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4036a00640 1 -- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 msgr2=0x7f40381f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4036a00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40381f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4036a00640 1 -- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4038161020 msgr2=0x7f403809a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4036a00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4038161020 0x7f403809a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4036a00640 1 -- 172.21.15.97:0/2647220440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009b30 con 0x7f403806d580 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f402de00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40381f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f4036a00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 0x7f4038070e30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f402800c430 tx=0x7f402800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280074e0 con 0x7f403806d580 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028007640 con 0x7f403806d580 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280059e0 con 0x7f403806d580 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403809afe0 con 0x7f403806d580 2024-01-31T05:01:47.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.633+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403809b5a0 con 0x7f403806d580 2024-01-31T05:01:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.634+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40380b25e0 con 0x7f403806d580 2024-01-31T05:01:47.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.636+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4028005c00 con 0x7f403806d580 2024-01-31T05:01:47.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.636+0000 7f402f400640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f401006aed0 0x7f401006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:47.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.636+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4028088ff0 con 0x7f403806d580 2024-01-31T05:01:47.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.637+0000 7f402de00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f401006aed0 0x7f401006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:47.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.637+0000 7f402de00640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f401006aed0 0x7f401006d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40381608b0 tx=0x7f4020008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:47.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.641+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f40280bc190 con 0x7f403806d580 2024-01-31T05:01:47.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.835+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4038067830 con 0x7f403806d580 2024-01-31T05:01:47.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.836+0000 7f402f400640 1 -- 172.21.15.97:0/2647220440 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4028017020 con 0x7f403806d580 2024-01-31T05:01:47.835 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:47.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f401006aed0 msgr2=0x7f401006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f401006aed0 0x7f401006d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f40381608b0 tx=0x7f4020008040 comp rx=0 tx=0).stop 2024-01-31T05:01:47.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 msgr2=0x7f4038070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:47.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 0x7f4038070e30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f402800c430 tx=0x7f402800c8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 shutdown_connections 2024-01-31T05:01:47.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4038161020 0x7f403809a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f403815d690 0x7f40381f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f401006aed0 0x7f401006d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 --2- 172.21.15.97:0/2647220440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f403806d580 0x7f4038070e30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 >> 172.21.15.97:0/2647220440 conn(0x7f40380b0a10 msgr2=0x7f40380cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.838+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 shutdown_connections 2024-01-31T05:01:47.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:47.839+0000 7f403d600640 1 -- 172.21.15.97:0/2647220440 wait complete. 2024-01-31T05:01:47.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:48.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.061+0000 7fce74e00640 1 Processor -- start 2024-01-31T05:01:48.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.062+0000 7fce74e00640 1 -- start start 2024-01-31T05:01:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.063+0000 7fce74e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7015d590 0x7fce7015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.063+0000 7fce74e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce700cff20 0x7fce70161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.063+0000 7fce74e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700cded0 con 0x7fce700cfae0 2024-01-31T05:01:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.063+0000 7fce74e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7015d970 con 0x7fce700cff20 2024-01-31T05:01:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.063+0000 7fce74e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700cf210 con 0x7fce7015d590 2024-01-31T05:01:48.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce700cff20 0x7fce70161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fce700cff20 0x7fce70161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33480/0 (socket says 172.21.15.97:33480) 2024-01-31T05:01:48.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 -- 172.21.15.97:0/4122949038 learned_addr learned my addr 172.21.15.97:0/4122949038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6de00640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7015d590 0x7fce7015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 -- 172.21.15.97:0/4122949038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7015d590 msgr2=0x7fce7015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7015d590 0x7fce7015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 -- 172.21.15.97:0/4122949038 >> v1:172.21.15.188:6789/0 conn(0x7fce700cfae0 legacy=0x7fce70161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6e800640 1 -- 172.21.15.97:0/4122949038 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700ce170 con 0x7fce700cff20 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.064+0000 7fce6de00640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7015d590 0x7fce7015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.065+0000 7fce6e800640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cff20 0x7fce70161790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e50 tx=0x7fce6802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7060b9597ad01be3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.065+0000 7fce6ca00640 1 -- 172.21.15.97:0/4122949038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6802cb00 con 0x7fce700cff20 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce6ca00640 1 -- 172.21.15.97:0/4122949038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6802cc60 con 0x7fce700cff20 2024-01-31T05:01:48.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce6ca00640 1 -- 172.21.15.97:0/4122949038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680343f0 con 0x7fce700cff20 2024-01-31T05:01:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 -- 172.21.15.97:0/4122949038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cff20 msgr2=0x7fce70161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cff20 0x7fce70161790 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e50 tx=0x7fce6802bf60 comp rx=0 tx=0).stop 2024-01-31T05:01:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 -- 172.21.15.97:0/4122949038 shutdown_connections 2024-01-31T05:01:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cff20 0x7fce70161790 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 --2- 172.21.15.97:0/4122949038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7015d590 0x7fce7015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 -- 172.21.15.97:0/4122949038 >> 172.21.15.97:0/4122949038 conn(0x7fce7009c560 msgr2=0x7fce700d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 -- 172.21.15.97:0/4122949038 shutdown_connections 2024-01-31T05:01:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.066+0000 7fce74e00640 1 -- 172.21.15.97:0/4122949038 wait complete. 2024-01-31T05:01:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.068+0000 7fce74e00640 1 Processor -- start 2024-01-31T05:01:48.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.068+0000 7fce74e00640 1 -- start start 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce74e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 0x7fce700b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce74e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce700cff20 0x7fce701f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce74e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7015d590 0x7fce701f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce74e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b15a0 con 0x7fce700cfae0 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce74e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b0730 con 0x7fce7015d590 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce74e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b0990 con 0x7fce700cff20 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 0x7fce700b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce6de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 0x7fce700b01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33492/0 (socket says 172.21.15.97:33492) 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.069+0000 7fce6de00640 1 -- 172.21.15.97:0/2656544736 learned_addr learned my addr 172.21.15.97:0/2656544736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6e800640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7015d590 0x7fce701f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6d400640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce700cff20 0x7fce701f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6de00640 1 -- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce700cff20 msgr2=0x7fce701f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6de00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce700cff20 0x7fce701f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6de00640 1 -- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7015d590 msgr2=0x7fce701f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6de00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7015d590 0x7fce701f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6de00640 1 -- 172.21.15.97:0/2656544736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009b30 con 0x7fce700cfae0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6d400640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce700cff20 0x7fce701f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6e800640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7015d590 0x7fce701f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce6de00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 0x7fce700b01f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c7f0 tx=0x7fce6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60007820 con 0x7fce700cfae0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60007980 con 0x7fce700cfae0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60007cf0 con 0x7fce700cfae0 2024-01-31T05:01:48.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.070+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce700b0c70 con 0x7fce700cfae0 2024-01-31T05:01:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.071+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce700b1230 con 0x7fce700cfae0 2024-01-31T05:01:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.072+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce700b13c0 con 0x7fce700cfae0 2024-01-31T05:01:48.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.073+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fce600113f0 con 0x7fce700cfae0 2024-01-31T05:01:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.074+0000 7fce66000640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fce4806aed0 0x7fce4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.074+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fce60089c50 con 0x7fce700cfae0 2024-01-31T05:01:48.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.074+0000 7fce6d400640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fce4806aed0 0x7fce4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.075+0000 7fce6d400640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fce4806aed0 0x7fce4806d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce7015bcc0 tx=0x7fce58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.078+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fce6008f020 con 0x7fce700cfae0 2024-01-31T05:01:48.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.272+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce70067830 con 0x7fce700cfae0 2024-01-31T05:01:48.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.273+0000 7fce66000640 1 -- 172.21.15.97:0/2656544736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce60054860 con 0x7fce700cfae0 2024-01-31T05:01:48.272 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.275+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fce4806aed0 msgr2=0x7fce4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.275+0000 7fce74e00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fce4806aed0 0x7fce4806d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fce7015bcc0 tx=0x7fce58009040 comp rx=0 tx=0).stop 2024-01-31T05:01:48.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.275+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 msgr2=0x7fce700b01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.275+0000 7fce74e00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 0x7fce700b01f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c7f0 tx=0x7fce6000ccb0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 shutdown_connections 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7015d590 0x7fce701f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce700cff20 0x7fce701f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fce4806aed0 0x7fce4806d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 --2- 172.21.15.97:0/2656544736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce700cfae0 0x7fce700b01f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 >> 172.21.15.97:0/2656544736 conn(0x7fce7009c560 msgr2=0x7fce700b63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 shutdown_connections 2024-01-31T05:01:48.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.276+0000 7fce74e00640 1 -- 172.21.15.97:0/2656544736 wait complete. 2024-01-31T05:01:48.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:48.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.505+0000 7f8de0e00640 1 Processor -- start 2024-01-31T05:01:48.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.505+0000 7f8de0e00640 1 -- start start 2024-01-31T05:01:48.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.506+0000 7f8de0e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc15b470 0x7f8ddc0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.506+0000 7f8de0e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ddc0abcf0 0x7f8ddc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.506+0000 7f8de0e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc06cd20 con 0x7f8ddc0cb810 2024-01-31T05:01:48.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.506+0000 7f8de0e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0ce970 con 0x7f8ddc0abcf0 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8de0e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc15b850 con 0x7f8ddc15b470 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ddc0abcf0 0x7f8ddc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dda800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8ddc0cb810 0x7f8ddc0ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36804/0 (socket says 172.21.15.97:36804) 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dda800640 1 -- 172.21.15.97:0/3053853694 learned_addr learned my addr 172.21.15.97:0/3053853694 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd9e00640 1 --2- 172.21.15.97:0/3053853694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc15b470 0x7f8ddc0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3053853694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc15b470 msgr2=0x7f8ddc0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd3e00640 1 --2- 172.21.15.97:0/3053853694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc15b470 0x7f8ddc0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3053853694 >> v1:172.21.15.97:6789/0 conn(0x7f8ddc0cb810 legacy=0x7f8ddc0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3053853694 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0ceb60 con 0x7f8ddc0abcf0 2024-01-31T05:01:48.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.507+0000 7f8dd9400640 1 -- 172.21.15.97:0/3053853694 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247900021 0 0) 0x7f8ddc06cd20 con 0x7f8ddc0cb810 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.508+0000 7f8dd3e00640 1 --2- 172.21.15.97:0/3053853694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0abcf0 0x7f8ddc0ce430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40097d0 tx=0x7f8dc402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff86d0bddb5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8dd9400640 1 -- 172.21.15.97:0/3053853694 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4031800 con 0x7f8ddc0abcf0 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8dd9400640 1 -- 172.21.15.97:0/3053853694 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4031960 con 0x7f8ddc0abcf0 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8dd9400640 1 -- 172.21.15.97:0/3053853694 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4031cf0 con 0x7f8ddc0abcf0 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8de0e00640 1 -- 172.21.15.97:0/3053853694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0abcf0 msgr2=0x7f8ddc0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3053853694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0abcf0 0x7f8ddc0ce430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40097d0 tx=0x7f8dc402b8a0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8de0e00640 1 -- 172.21.15.97:0/3053853694 shutdown_connections 2024-01-31T05:01:48.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3053853694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0abcf0 0x7f8ddc0ce430 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3053853694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc15b470 0x7f8ddc0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.509+0000 7f8de0e00640 1 -- 172.21.15.97:0/3053853694 >> 172.21.15.97:0/3053853694 conn(0x7f8ddc0b0a10 msgr2=0x7f8ddc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:48.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.510+0000 7f8de0e00640 1 -- 172.21.15.97:0/3053853694 shutdown_connections 2024-01-31T05:01:48.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.510+0000 7f8de0e00640 1 -- 172.21.15.97:0/3053853694 wait complete. 2024-01-31T05:01:48.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.511+0000 7f8de0e00640 1 Processor -- start 2024-01-31T05:01:48.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.511+0000 7f8de0e00640 1 -- start start 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.512+0000 7f8de0e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc0abcf0 0x7f8ddc1488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.512+0000 7f8de0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 0x7f8ddc15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.512+0000 7f8de0e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ddc15b470 0x7f8ddc156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.512+0000 7f8de0e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc146090 con 0x7f8ddc0cb810 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.512+0000 7f8de0e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc145480 con 0x7f8ddc15b470 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.512+0000 7f8de0e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc145760 con 0x7f8ddc0abcf0 2024-01-31T05:01:48.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 0x7f8ddc15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 0x7f8ddc15e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33502/0 (socket says 172.21.15.97:33502) 2024-01-31T05:01:48.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3338449414 learned_addr learned my addr 172.21.15.97:0/3338449414 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd9e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc0abcf0 0x7f8ddc1488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc0abcf0 msgr2=0x7f8ddc1488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dda800640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ddc15b470 0x7f8ddc156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc0abcf0 0x7f8ddc1488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ddc15b470 msgr2=0x7f8ddc156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ddc15b470 0x7f8ddc156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dd3e00640 1 -- 172.21.15.97:0/3338449414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc4009400 con 0x7f8ddc0cb810 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.513+0000 7f8dda800640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ddc15b470 0x7f8ddc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:48.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.514+0000 7f8dd3e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 0x7f8ddc15e1e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40097d0 tx=0x7f8dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.514+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc403bd30 con 0x7f8ddc0cb810 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.514+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc403be90 con 0x7f8ddc0cb810 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.514+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4045630 con 0x7f8ddc0cb810 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.514+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc145a40 con 0x7f8ddc0cb810 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.514+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc15e720 con 0x7f8ddc0cb810 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.515+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc146240 con 0x7f8ddc0cb810 2024-01-31T05:01:48.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.516+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8dc404e450 con 0x7f8ddc0cb810 2024-01-31T05:01:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.517+0000 7f8dd2000640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8db406af00 0x7f8db406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.517+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8dc402e070 con 0x7f8ddc0cb810 2024-01-31T05:01:48.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.517+0000 7f8dd9e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8db406af00 0x7f8db406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.518+0000 7f8dd9e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8db406af00 0x7f8db406d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0096d0 tx=0x7f8dcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.522+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8ddc146240 con 0x7f8ddc0cb810 2024-01-31T05:01:48.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.707+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ddc146240 con 0x7f8ddc0cb810 2024-01-31T05:01:48.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.708+0000 7f8dd2000640 1 -- 172.21.15.97:0/3338449414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ddc146240 con 0x7f8ddc0cb810 2024-01-31T05:01:48.707 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8db406af00 msgr2=0x7f8db406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8db406af00 0x7f8db406d3c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc0096d0 tx=0x7f8dcc008040 comp rx=0 tx=0).stop 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 msgr2=0x7f8ddc15e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 0x7f8ddc15e1e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40097d0 tx=0x7f8dc40046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 shutdown_connections 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ddc15b470 0x7f8ddc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8db406af00 0x7f8db406d3c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ddc0cb810 0x7f8ddc15e1e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 --2- 172.21.15.97:0/3338449414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ddc0abcf0 0x7f8ddc1488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 >> 172.21.15.97:0/3338449414 conn(0x7f8ddc0b0a10 msgr2=0x7f8ddc06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 shutdown_connections 2024-01-31T05:01:48.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.710+0000 7f8de0e00640 1 -- 172.21.15.97:0/3338449414 wait complete. 2024-01-31T05:01:48.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:48.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.936+0000 7fafb2e00640 1 Processor -- start 2024-01-31T05:01:48.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.936+0000 7fafb2e00640 1 -- start start 2024-01-31T05:01:48.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafb2e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac15b470 0x7fafac0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafb2e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafac0abcf0 0x7fafac0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafb2e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac06cd20 con 0x7fafac0cb810 2024-01-31T05:01:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafb2e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafac0ce970 con 0x7fafac0abcf0 2024-01-31T05:01:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafb2e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac15b850 con 0x7fafac15b470 2024-01-31T05:01:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafa3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafac0abcf0 0x7fafac0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafa3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fafac0abcf0 0x7fafac0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33508/0 (socket says 172.21.15.97:33508) 2024-01-31T05:01:48.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafa3200640 1 -- 172.21.15.97:0/1446732085 learned_addr learned my addr 172.21.15.97:0/1446732085 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafabe00640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac15b470 0x7fafac0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.938+0000 7fafa3200640 1 -- 172.21.15.97:0/1446732085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac15b470 msgr2=0x7fafac0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.939+0000 7fafa3200640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac15b470 0x7fafac0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.939+0000 7fafa3200640 1 -- 172.21.15.97:0/1446732085 >> v1:172.21.15.97:6789/0 conn(0x7fafac0cb810 legacy=0x7fafac0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.939+0000 7fafa3200640 1 -- 172.21.15.97:0/1446732085 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0ceb60 con 0x7fafac0abcf0 2024-01-31T05:01:48.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.939+0000 7fafabe00640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac15b470 0x7fafac0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:48.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.940+0000 7fafa3200640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac0abcf0 0x7fafac0ce430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf940098a0 tx=0x7faf9402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6177213222a810e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.940+0000 7fafab400640 1 -- 172.21.15.97:0/1446732085 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94030840 con 0x7fafac0abcf0 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.940+0000 7fafab400640 1 -- 172.21.15.97:0/1446732085 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf940309a0 con 0x7fafac0abcf0 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.940+0000 7fafab400640 1 -- 172.21.15.97:0/1446732085 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94030d30 con 0x7fafac0abcf0 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 -- 172.21.15.97:0/1446732085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac0abcf0 msgr2=0x7fafac0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac0abcf0 0x7fafac0ce430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf940098a0 tx=0x7faf9402b980 comp rx=0 tx=0).stop 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 -- 172.21.15.97:0/1446732085 shutdown_connections 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac0abcf0 0x7fafac0ce430 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 --2- 172.21.15.97:0/1446732085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac15b470 0x7fafac0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 -- 172.21.15.97:0/1446732085 >> 172.21.15.97:0/1446732085 conn(0x7fafac0b0a10 msgr2=0x7fafac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 -- 172.21.15.97:0/1446732085 shutdown_connections 2024-01-31T05:01:48.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.941+0000 7fafb2e00640 1 -- 172.21.15.97:0/1446732085 wait complete. 2024-01-31T05:01:48.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.942+0000 7fafb2e00640 1 Processor -- start 2024-01-31T05:01:48.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.943+0000 7fafb2e00640 1 -- start start 2024-01-31T05:01:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.943+0000 7fafb2e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac0abcf0 0x7fafac147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.943+0000 7fafb2e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafac0cb810 0x7fafac1448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.943+0000 7fafb2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 0x7fafac0ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac1481c0 con 0x7fafac15b470 2024-01-31T05:01:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb2e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac145510 con 0x7fafac0cb810 2024-01-31T05:01:48.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb2e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac1457f0 con 0x7fafac0abcf0 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 0x7fafac0ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 0x7fafac0ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33510/0 (socket says 172.21.15.97:33510) 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 -- 172.21.15.97:0/989371220 learned_addr learned my addr 172.21.15.97:0/989371220 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 -- 172.21.15.97:0/989371220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac0abcf0 msgr2=0x7fafac147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac0abcf0 0x7fafac147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 -- 172.21.15.97:0/989371220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafac0cb810 msgr2=0x7fafac1448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafac0cb810 0x7fafac1448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.944+0000 7fafb0e00640 1 -- 172.21.15.97:0/989371220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf940094d0 con 0x7fafac15b470 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.945+0000 7fafb0e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 0x7fafac0ad3d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fafa400c710 tx=0x7fafa400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.945+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4006d50 con 0x7fafac15b470 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.945+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa400ce20 con 0x7fafac15b470 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.945+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4003bb0 con 0x7fafac15b470 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.945+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac145ad0 con 0x7fafac15b470 2024-01-31T05:01:48.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.945+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac146090 con 0x7fafac15b470 2024-01-31T05:01:48.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.946+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac067830 con 0x7fafac15b470 2024-01-31T05:01:48.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.947+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fafa4003d10 con 0x7fafac15b470 2024-01-31T05:01:48.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.948+0000 7fafa8c00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf8406af00 0x7faf8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:48.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.948+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fafa4089490 con 0x7fafac15b470 2024-01-31T05:01:48.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.948+0000 7fafabe00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf8406af00 0x7faf8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:48.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.949+0000 7fafabe00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf8406af00 0x7faf8406d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00a870 tx=0x7faf9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:48.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:48.952+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fafa4057b30 con 0x7fafac15b470 2024-01-31T05:01:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.148+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fafac0b19a0 con 0x7fafac15b470 2024-01-31T05:01:49.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.149+0000 7fafa8c00640 1 -- 172.21.15.97:0/989371220 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fafa40576c0 con 0x7fafac15b470 2024-01-31T05:01:49.148 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.151+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf8406af00 msgr2=0x7faf8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.151+0000 7fafb2e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf8406af00 0x7faf8406d3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00a870 tx=0x7faf9c008040 comp rx=0 tx=0).stop 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.151+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 msgr2=0x7fafac0ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.151+0000 7fafb2e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 0x7fafac0ad3d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fafa400c710 tx=0x7fafa400cbd0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 shutdown_connections 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf8406af00 0x7faf8406d3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fafac15b470 0x7fafac0ad3d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafac0cb810 0x7fafac1448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 --2- 172.21.15.97:0/989371220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafac0abcf0 0x7fafac147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 >> 172.21.15.97:0/989371220 conn(0x7fafac0b0a10 msgr2=0x7fafac06d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:49.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 shutdown_connections 2024-01-31T05:01:49.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.152+0000 7fafb2e00640 1 -- 172.21.15.97:0/989371220 wait complete. 2024-01-31T05:01:49.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-01-31T05:01:49.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.367+0000 7f82a9c00640 1 Processor -- start 2024-01-31T05:01:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.367+0000 7f82a9c00640 1 -- start start 2024-01-31T05:01:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a9c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f82a40cb810 0x7f82a40cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a9c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82a415b400 0x7f82a406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a9c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a406d1a0 con 0x7f82a406db70 2024-01-31T05:01:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a9c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a40cc960 con 0x7f82a40cb810 2024-01-31T05:01:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a9c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a406c600 con 0x7f82a415b400 2024-01-31T05:01:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a2a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f82a40cb810 0x7f82a40cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a3400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f82a406db70 0x7f82a40ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36832/0 (socket says 172.21.15.97:36832) 2024-01-31T05:01:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a3400640 1 -- 172.21.15.97:0/3601980148 learned_addr learned my addr 172.21.15.97:0/3601980148 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a2000640 1 --2- 172.21.15.97:0/3601980148 >> v2:172.21.15.188:3300/0 conn(0x7f82a415b400 0x7f82a406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a1600640 1 -- 172.21.15.97:0/3601980148 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343105448 0 0) 0x7f82a406d1a0 con 0x7f82a406db70 2024-01-31T05:01:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.369+0000 7f82a1600640 1 -- 172.21.15.97:0/3601980148 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8284003520 con 0x7f82a406db70 2024-01-31T05:01:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.370+0000 7f82a2a00640 1 -- 172.21.15.97:0/3601980148 >> v2:172.21.15.188:3300/0 conn(0x7f82a415b400 msgr2=0x7f82a406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.370+0000 7f82a2a00640 1 --2- 172.21.15.97:0/3601980148 >> v2:172.21.15.188:3300/0 conn(0x7f82a415b400 0x7f82a406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.370+0000 7f82a2a00640 1 -- 172.21.15.97:0/3601980148 >> v1:172.21.15.97:6789/0 conn(0x7f82a406db70 legacy=0x7f82a40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.370+0000 7f82a2a00640 1 -- 172.21.15.97:0/3601980148 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40cc130 con 0x7f82a40cb810 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.371+0000 7f82a2a00640 1 --2- 172.21.15.97:0/3601980148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a40cb810 0x7f82a40cbbf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f82a40ae830 tx=0x7f829402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecc466cef1683c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.371+0000 7f82a1600640 1 -- 172.21.15.97:0/3601980148 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294031830 con 0x7f82a40cb810 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.371+0000 7f82a1600640 1 -- 172.21.15.97:0/3601980148 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8294031990 con 0x7f82a40cb810 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.371+0000 7f82a1600640 1 -- 172.21.15.97:0/3601980148 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294031d20 con 0x7f82a40cb810 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 -- 172.21.15.97:0/3601980148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a40cb810 msgr2=0x7f82a40cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 --2- 172.21.15.97:0/3601980148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a40cb810 0x7f82a40cbbf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f82a40ae830 tx=0x7f829402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 -- 172.21.15.97:0/3601980148 shutdown_connections 2024-01-31T05:01:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 --2- 172.21.15.97:0/3601980148 >> v2:172.21.15.188:3300/0 conn(0x7f82a415b400 0x7f82a406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 --2- 172.21.15.97:0/3601980148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a40cb810 0x7f82a40cbbf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 -- 172.21.15.97:0/3601980148 >> 172.21.15.97:0/3601980148 conn(0x7f82a40b0a10 msgr2=0x7f82a40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 -- 172.21.15.97:0/3601980148 shutdown_connections 2024-01-31T05:01:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.372+0000 7f82a9c00640 1 -- 172.21.15.97:0/3601980148 wait complete. 2024-01-31T05:01:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.373+0000 7f82a9c00640 1 Processor -- start 2024-01-31T05:01:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.374+0000 7f82a9c00640 1 -- start start 2024-01-31T05:01:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.374+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a406db70 0x7f82a414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.374+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40cb810 0x7f82a41f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.374+0000 7f82a9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 0x7f82a40afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.374+0000 7f82a9c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40b2dd0 con 0x7f82a415b400 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a414feb0 con 0x7f82a40cb810 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40c8ef0 con 0x7f82a406db70 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 0x7f82a40afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 0x7f82a40afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33538/0 (socket says 172.21.15.97:33538) 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 -- 172.21.15.97:0/2889537227 learned_addr learned my addr 172.21.15.97:0/2889537227 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a2a00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a406db70 0x7f82a414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 -- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a406db70 msgr2=0x7f82a414f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a406db70 0x7f82a414f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 -- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40cb810 msgr2=0x7f82a41f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40cb810 0x7f82a41f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 -- 172.21.15.97:0/2889537227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294009400 con 0x7f82a415b400 2024-01-31T05:01:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.375+0000 7f82a3400640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 0x7f82a40afba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f829c00cbb0 tx=0x7f829c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.376+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00ad40 con 0x7f82a415b400 2024-01-31T05:01:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.376+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c012400 con 0x7f82a415b400 2024-01-31T05:01:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.376+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00aea0 con 0x7f82a415b400 2024-01-31T05:01:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.376+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a41f7280 con 0x7f82a415b400 2024-01-31T05:01:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.376+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a41f3580 con 0x7f82a415b400 2024-01-31T05:01:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.377+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a40c90c0 con 0x7f82a415b400 2024-01-31T05:01:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.378+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f829c007070 con 0x7f82a415b400 2024-01-31T05:01:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.378+0000 7f829aa00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f827c06aed0 0x7f827c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.379+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f829c089850 con 0x7f82a415b400 2024-01-31T05:01:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.379+0000 7f82a2a00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f827c06aed0 0x7f827c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.379+0000 7f82a2a00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f827c06aed0 0x7f827c06d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8294036000 tx=0x7f8294008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:49.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.383+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f829c054470 con 0x7f82a415b400 2024-01-31T05:01:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.631+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f82a4155330 con 0x7f82a415b400 2024-01-31T05:01:49.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.887+0000 7f829aa00640 1 -- 172.21.15.97:0/2889537227 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v76) v1 ==== 150+0+0 (secure 0 0 0) 0x7f829c057a90 con 0x7f82a415b400 2024-01-31T05:01:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.891+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f827c06aed0 msgr2=0x7f827c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.891+0000 7f82a9c00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f827c06aed0 0x7f827c06d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8294036000 tx=0x7f8294008030 comp rx=0 tx=0).stop 2024-01-31T05:01:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.891+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 msgr2=0x7f82a40afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:49.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.891+0000 7f82a9c00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 0x7f82a40afba0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f829c00cbb0 tx=0x7f829c00a430 comp rx=0 tx=0).stop 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 shutdown_connections 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f827c06aed0 0x7f827c06d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a415b400 0x7f82a40afba0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a40cb810 0x7f82a41f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 --2- 172.21.15.97:0/2889537227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a406db70 0x7f82a414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 >> 172.21.15.97:0/2889537227 conn(0x7f82a40b0a10 msgr2=0x7f82a4152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 shutdown_connections 2024-01-31T05:01:49.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:49.892+0000 7f82a9c00640 1 -- 172.21.15.97:0/2889537227 wait complete. 2024-01-31T05:01:49.913 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-31T05:01:50.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.123+0000 7f488f400640 1 Processor -- start 2024-01-31T05:01:50.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.124+0000 7f488f400640 1 -- start start 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f488f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48880ce000 0x7f48880cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f488f400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48880cd4f0 con 0x7f48880b2570 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f488f400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48880cb760 con 0x7f48880d1b90 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f488f400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488806ce00 con 0x7f48880ce000 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f487fc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48880ce000 0x7f48880cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f488ca00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f48880b2570 0x7f48880cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36840/0 (socket says 172.21.15.97:36840) 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f487fc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48880ce000 0x7f48880cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33552/0 (socket says 172.21.15.97:33552) 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f488ca00640 1 -- 172.21.15.97:0/1743909035 learned_addr learned my addr 172.21.15.97:0/1743909035 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:50.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.125+0000 7f487fc00640 1 -- 172.21.15.97:0/1743909035 >> v1:172.21.15.188:6789/0 conn(0x7f48880d1b90 legacy=0x7f488815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.126+0000 7f487fc00640 1 -- 172.21.15.97:0/1743909035 >> v1:172.21.15.97:6789/0 conn(0x7f48880b2570 legacy=0x7f48880cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.126+0000 7f487fc00640 1 -- 172.21.15.97:0/1743909035 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48880b2950 con 0x7f48880ce000 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.126+0000 7f4887e00640 1 -- 172.21.15.97:0/1743909035 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464998876 0 0) 0x7f48880cd4f0 con 0x7f48880b2570 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.127+0000 7f487fc00640 1 --2- 172.21.15.97:0/1743909035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880ce000 0x7f48880cf3e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f48700097a0 tx=0x7f487002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b6edba988a9217 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.127+0000 7f4887e00640 1 -- 172.21.15.97:0/1743909035 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870031830 con 0x7f48880ce000 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.127+0000 7f4887e00640 1 -- 172.21.15.97:0/1743909035 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4870031990 con 0x7f48880ce000 2024-01-31T05:01:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.127+0000 7f4887e00640 1 -- 172.21.15.97:0/1743909035 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870031d20 con 0x7f48880ce000 2024-01-31T05:01:50.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.127+0000 7f488f400640 1 -- 172.21.15.97:0/1743909035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880ce000 msgr2=0x7f48880cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.127+0000 7f488f400640 1 --2- 172.21.15.97:0/1743909035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880ce000 0x7f48880cf3e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f48700097a0 tx=0x7f487002b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.128+0000 7f488f400640 1 -- 172.21.15.97:0/1743909035 shutdown_connections 2024-01-31T05:01:50.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.128+0000 7f488f400640 1 --2- 172.21.15.97:0/1743909035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880ce000 0x7f48880cf3e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.128+0000 7f488f400640 1 -- 172.21.15.97:0/1743909035 >> 172.21.15.97:0/1743909035 conn(0x7f48880d1070 msgr2=0x7f48880d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:50.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.128+0000 7f488f400640 1 -- 172.21.15.97:0/1743909035 shutdown_connections 2024-01-31T05:01:50.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.128+0000 7f488f400640 1 -- 172.21.15.97:0/1743909035 wait complete. 2024-01-31T05:01:50.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.129+0000 7f488f400640 1 Processor -- start 2024-01-31T05:01:50.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 -- start start 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48880b2570 0x7f48881e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880ce000 0x7f48881e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 0x7f48881e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48881e05d0 con 0x7f48880d1b90 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48880c82e0 con 0x7f48880ce000 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.130+0000 7f488f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48880c85c0 con 0x7f48880b2570 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 0x7f48881e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 0x7f48881e5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33566/0 (socket says 172.21.15.97:33566) 2024-01-31T05:01:50.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 -- 172.21.15.97:0/3203998077 learned_addr learned my addr 172.21.15.97:0/3203998077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:50.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 -- 172.21.15.97:0/3203998077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48880b2570 msgr2=0x7f48881e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48880b2570 0x7f48881e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 -- 172.21.15.97:0/3203998077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880ce000 msgr2=0x7f48881e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880ce000 0x7f48881e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 -- 172.21.15.97:0/3203998077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009400 con 0x7f48880d1b90 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.131+0000 7f488d400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 0x7f48881e5510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f488000b490 tx=0x7f488000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.132+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880011830 con 0x7f48880d1b90 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.132+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48800119b0 con 0x7f48880d1b90 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.132+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880011d60 con 0x7f48880d1b90 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.132+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48880c88a0 con 0x7f48880d1b90 2024-01-31T05:01:50.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.132+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48880c8e60 con 0x7f48880d1b90 2024-01-31T05:01:50.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.133+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4888067040 con 0x7f48880d1b90 2024-01-31T05:01:50.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.134+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f488001a400 con 0x7f48880d1b90 2024-01-31T05:01:50.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.135+0000 7f4885600640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f486006aed0 0x7f486006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.135+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4880089b10 con 0x7f48880d1b90 2024-01-31T05:01:50.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.135+0000 7f488ca00640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f486006aed0 0x7f486006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.136+0000 7f488ca00640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f486006aed0 0x7f486006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4878001560 tx=0x7f4878007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:50.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.139+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f48800545f0 con 0x7f48880d1b90 2024-01-31T05:01:50.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.333+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f48880c9050 con 0x7f48880d1b90 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.337+0000 7f4885600640 1 -- 172.21.15.97:0/3203998077 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4880015020 con 0x7f48880d1b90 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f486006aed0 msgr2=0x7f486006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f486006aed0 0x7f486006d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4878001560 tx=0x7f4878007ca0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 msgr2=0x7f48881e5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 0x7f48881e5510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f488000b490 tx=0x7f488000b950 comp rx=0 tx=0).stop 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 shutdown_connections 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f486006aed0 0x7f486006d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48880d1b90 0x7f48881e5510 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48880ce000 0x7f48881e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 --2- 172.21.15.97:0/3203998077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48880b2570 0x7f48881e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 >> 172.21.15.97:0/3203998077 conn(0x7f48880d1070 msgr2=0x7f488806ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 shutdown_connections 2024-01-31T05:01:50.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.341+0000 7f488f400640 1 -- 172.21.15.97:0/3203998077 wait complete. 2024-01-31T05:01:50.359 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-31T05:01:50.359 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:01:50.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:01:50.586 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-31T05:01:50.586 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:01:50.586 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:50.645 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:01:50.645 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:50.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.891+0000 7fd851e00640 1 Processor -- start 2024-01-31T05:01:50.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.892+0000 7fd851e00640 1 -- start start 2024-01-31T05:01:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd851e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 0x7fd84c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd851e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c0d1b20 con 0x7fd84c0d3bb0 2024-01-31T05:01:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd851e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c0d1e00 con 0x7fd84c0b3e90 2024-01-31T05:01:50.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd851e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0d20e0 con 0x7fd84c0cfca0 2024-01-31T05:01:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd841e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd84c0d3bb0 0x7fd84c0b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36844/0 (socket says 172.21.15.97:36844) 2024-01-31T05:01:50.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd841e00640 1 -- 172.21.15.97:0/2802256848 learned_addr learned my addr 172.21.15.97:0/2802256848 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:50.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd84aa00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 0x7fd84c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348353768 0 0) 0x7fd84c0d1b20 con 0x7fd84c0d3bb0 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.894+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd82c003520 con 0x7fd84c0d3bb0 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1803145167 0 0) 0x7fd82c003520 con 0x7fd84c0d3bb0 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 msgr2=0x7fd84c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 0x7fd84c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 >> v1:172.21.15.188:6789/0 conn(0x7fd84c0b3e90 legacy=0x7fd84c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0d4120 con 0x7fd84c0d3bb0 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fd834001670 con 0x7fd84c0d3bb0 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 >> v1:172.21.15.97:6789/0 conn(0x7fd84c0d3bb0 legacy=0x7fd84c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c0039a0 0x7fd82c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c0047c0 0x7fd82c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 0x7fd82c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0d20e0 con 0x7fd82c0039a0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c00ba60 con 0x7fd82c0047c0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c00bc20 con 0x7fd84c0cfca0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.895+0000 7fd84aa00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c0039a0 0x7fd82c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd84b400640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 0x7fd82c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd841e00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c0047c0 0x7fd82c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd84aa00640 1 -- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 msgr2=0x7fd82c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd84aa00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd84c0cfca0 0x7fd82c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd84aa00640 1 -- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c0047c0 msgr2=0x7fd82c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd84aa00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c0047c0 0x7fd82c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd84aa00640 1 -- 172.21.15.97:0/2802256848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c003520 con 0x7fd82c0039a0 2024-01-31T05:01:50.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.896+0000 7fd841e00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd82c0047c0 0x7fd82c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.897+0000 7fd84aa00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c0039a0 0x7fd82c003dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009f90 tx=0x7fd82c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d53d797c1902e35 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.897+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c02cbb0 con 0x7fd82c0039a0 2024-01-31T05:01:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.897+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00f040 con 0x7fd82c0039a0 2024-01-31T05:01:50.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.897+0000 7fd84a000640 1 -- 172.21.15.97:0/2802256848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c034440 con 0x7fd82c0039a0 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd851e00640 1 -- 172.21.15.97:0/2802256848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c0039a0 msgr2=0x7fd82c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd851e00640 1 --2- 172.21.15.97:0/2802256848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd82c0039a0 0x7fd82c003dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009f90 tx=0x7fd82c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd84b400640 1 -- 172.21.15.97:0/2802256848 reap_dead start 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd851e00640 1 -- 172.21.15.97:0/2802256848 shutdown_connections 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd851e00640 1 -- 172.21.15.97:0/2802256848 >> 172.21.15.97:0/2802256848 conn(0x7fd84c09c570 msgr2=0x7fd84c0d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd851e00640 1 -- 172.21.15.97:0/2802256848 shutdown_connections 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.898+0000 7fd851e00640 1 -- 172.21.15.97:0/2802256848 wait complete. 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.900+0000 7fd851e00640 1 Processor -- start 2024-01-31T05:01:50.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.900+0000 7fd851e00640 1 -- start start 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd851e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82c0039a0 0x7fd84c1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd851e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd84c15aee0 0x7fd84c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd851e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 0x7fd84c1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd851e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0cbfa0 con 0x7fd84c0b26e0 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd851e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0b2c30 con 0x7fd84c15aee0 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd851e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0cb670 con 0x7fd82c0039a0 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd84b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 0x7fd84c1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd84b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 0x7fd84c1513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33572/0 (socket says 172.21.15.97:33572) 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd84b400640 1 -- 172.21.15.97:0/3929876370 learned_addr learned my addr 172.21.15.97:0/3929876370 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd84aa00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82c0039a0 0x7fd84c1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.901+0000 7fd841e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd84c15aee0 0x7fd84c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd84b400640 1 -- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82c0039a0 msgr2=0x7fd84c1f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd84b400640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82c0039a0 0x7fd84c1f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd84b400640 1 -- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd84c15aee0 msgr2=0x7fd84c0b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd84b400640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd84c15aee0 0x7fd84c0b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd84b400640 1 -- 172.21.15.97:0/3929876370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0cb950 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd84b400640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 0x7fd84c1513f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd84c06b1b0 tx=0x7fd84400d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84400b070 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84400de60 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c0cbc30 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844003810 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.902+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c0b10e0 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.904+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c0cc150 con 0x7fd84c0b26e0 2024-01-31T05:01:50.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.904+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd844003970 con 0x7fd84c0b26e0 2024-01-31T05:01:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.905+0000 7fd843e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd82406aed0 0x7fd82406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.905+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd844088d10 con 0x7fd84c0b26e0 2024-01-31T05:01:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.905+0000 7fd84aa00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd82406aed0 0x7fd82406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:50.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.906+0000 7fd84aa00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd82406aed0 0x7fd82406d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd83c036000 tx=0x7fd83c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:50.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:50.910+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd84c0cc150 con 0x7fd84c0b26e0 2024-01-31T05:01:51.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.106+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd84c0b10e0 con 0x7fd84c0b26e0 2024-01-31T05:01:51.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.108+0000 7fd843e00640 1 -- 172.21.15.97:0/3929876370 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7fd84c0b10e0 con 0x7fd84c0b26e0 2024-01-31T05:01:51.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:51.108 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd82406aed0 msgr2=0x7fd82406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd82406aed0 0x7fd82406d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd83c036000 tx=0x7fd83c02e000 comp rx=0 tx=0).stop 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 msgr2=0x7fd84c1513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 0x7fd84c1513f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd84c06b1b0 tx=0x7fd84400d660 comp rx=0 tx=0).stop 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 shutdown_connections 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd82406aed0 0x7fd82406d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd84c0b26e0 0x7fd84c1513f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd84c15aee0 0x7fd84c0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 --2- 172.21.15.97:0/3929876370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd82c0039a0 0x7fd84c1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 >> 172.21.15.97:0/3929876370 conn(0x7fd84c09c570 msgr2=0x7fd84c0cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 shutdown_connections 2024-01-31T05:01:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.111+0000 7fd851e00640 1 -- 172.21.15.97:0/3929876370 wait complete. 2024-01-31T05:01:51.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:51.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:51.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.331+0000 7f71dd200640 1 Processor -- start 2024-01-31T05:01:51.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.332+0000 7f71dd200640 1 -- start start 2024-01-31T05:01:51.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.333+0000 7f71dd200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71d80b2570 0x7f71d80cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.333+0000 7f71dd200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.333+0000 7f71dd200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d80cb760 con 0x7f71d80cde30 2024-01-31T05:01:51.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.333+0000 7f71dd200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d80ce2c0 con 0x7f71d80b2570 2024-01-31T05:01:51.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.333+0000 7f71dd200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d806ce00 con 0x7f71d80d1b90 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.334+0000 7f71d5e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71d80b2570 0x7f71d80cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.334+0000 7f71d5e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71d80b2570 0x7f71d80cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:60110/0 (socket says 172.21.15.97:60110) 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.334+0000 7f71d5e00640 1 -- 172.21.15.97:0/1298432173 learned_addr learned my addr 172.21.15.97:0/1298432173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.334+0000 7f71cd200640 1 --2- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.335+0000 7f71d5e00640 1 -- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 msgr2=0x7f71d80cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.335+0000 7f71d5e00640 1 --2- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.335+0000 7f71d5e00640 1 -- 172.21.15.97:0/1298432173 >> v1:172.21.15.188:6789/0 conn(0x7f71d80cde30 legacy=0x7f71d815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.335+0000 7f71d5e00640 1 -- 172.21.15.97:0/1298432173 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80cd4f0 con 0x7f71d80b2570 2024-01-31T05:01:51.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.336+0000 7f71d5e00640 1 --2- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80b2570 0x7f71d80cf3e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71c80098a0 tx=0x7f71c802b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881c3ba14aaef6b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:51.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.336+0000 7f71d5400640 1 -- 172.21.15.97:0/1298432173 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8030830 con 0x7f71d80b2570 2024-01-31T05:01:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.336+0000 7f71d5400640 1 -- 172.21.15.97:0/1298432173 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c8030990 con 0x7f71d80b2570 2024-01-31T05:01:51.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.336+0000 7f71d5400640 1 -- 172.21.15.97:0/1298432173 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8030d20 con 0x7f71d80b2570 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 -- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80b2570 msgr2=0x7f71d80cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 --2- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80b2570 0x7f71d80cf3e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71c80098a0 tx=0x7f71c802b970 comp rx=0 tx=0).stop 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 -- 172.21.15.97:0/1298432173 shutdown_connections 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 --2- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 --2- 172.21.15.97:0/1298432173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80b2570 0x7f71d80cf3e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 -- 172.21.15.97:0/1298432173 >> 172.21.15.97:0/1298432173 conn(0x7f71d80d1070 msgr2=0x7f71d80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 -- 172.21.15.97:0/1298432173 shutdown_connections 2024-01-31T05:01:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.337+0000 7f71dd200640 1 -- 172.21.15.97:0/1298432173 wait complete. 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.339+0000 7f71dd200640 1 Processor -- start 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.339+0000 7f71dd200640 1 -- start start 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71dd200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 0x7f71d8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71dd200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80cde30 0x7f71d814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71dd200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71dd200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80c9d70 con 0x7f71d80b2570 2024-01-31T05:01:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71dd200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80c75c0 con 0x7f71d80cde30 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71dd200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80c78a0 con 0x7f71d80d1b90 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71d5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 0x7f71d8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71d5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 0x7f71d8158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33586/0 (socket says 172.21.15.97:33586) 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71d5e00640 1 -- 172.21.15.97:0/1395940645 learned_addr learned my addr 172.21.15.97:0/1395940645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71cd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80cde30 0x7f71d814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71d5e00640 1 -- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 msgr2=0x7f71d80c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.340+0000 7f71d6800640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71d5e00640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71d5e00640 1 -- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80cde30 msgr2=0x7f71d814f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71d5e00640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80cde30 0x7f71d814f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71d5e00640 1 -- 172.21.15.97:0/1395940645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80094d0 con 0x7f71d80b2570 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71cd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80cde30 0x7f71d814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71d5e00640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 0x7f71d8158d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71c80098a0 tx=0x7f71c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8047070 con 0x7f71d80b2570 2024-01-31T05:01:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c8039660 con 0x7f71d80b2570 2024-01-31T05:01:51.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d80ca010 con 0x7f71d80b2570 2024-01-31T05:01:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c80399f0 con 0x7f71d80b2570 2024-01-31T05:01:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.341+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d814f900 con 0x7f71d80b2570 2024-01-31T05:01:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.343+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d8067830 con 0x7f71d80b2570 2024-01-31T05:01:51.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.343+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f71c8039b50 con 0x7f71d80b2570 2024-01-31T05:01:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.344+0000 7f71cea00640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f71b006aed0 0x7f71b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.344+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f71c803e070 con 0x7f71d80b2570 2024-01-31T05:01:51.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.345+0000 7f71cd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f71b006aed0 0x7f71b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.345+0000 7f71cd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f71b006aed0 0x7f71b006d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71d815c130 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:51.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.349+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f71c8076920 con 0x7f71d80b2570 2024-01-31T05:01:51.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.542+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71d80b01d0 con 0x7f71d80b2570 2024-01-31T05:01:51.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.543+0000 7f71cea00640 1 -- 172.21.15.97:0/1395940645 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71c8079f40 con 0x7f71d80b2570 2024-01-31T05:01:51.543 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f71b006aed0 msgr2=0x7f71b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f71b006aed0 0x7f71b006d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71d815c130 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 msgr2=0x7f71d8158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 0x7f71d8158d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71c80098a0 tx=0x7f71c8004730 comp rx=0 tx=0).stop 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 shutdown_connections 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71d80d1b90 0x7f71d80c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71d80cde30 0x7f71d814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f71b006aed0 0x7f71b006d390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 --2- 172.21.15.97:0/1395940645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71d80b2570 0x7f71d8158d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 >> 172.21.15.97:0/1395940645 conn(0x7f71d80d1070 msgr2=0x7f71d80b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 shutdown_connections 2024-01-31T05:01:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.546+0000 7f71dd200640 1 -- 172.21.15.97:0/1395940645 wait complete. 2024-01-31T05:01:51.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:51.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.769+0000 7f9eddc00640 1 Processor -- start 2024-01-31T05:01:51.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.770+0000 7f9eddc00640 1 -- start start 2024-01-31T05:01:51.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.771+0000 7f9eddc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ed815b080 0x7f9ed80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.771+0000 7f9eddc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 0x7f9ed80cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.771+0000 7f9eddc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ed80ce4b0 0x7f9ed815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.771+0000 7f9eddc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80b41c0 con 0x7f9ed815b080 2024-01-31T05:01:51.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.771+0000 7f9eddc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80ce920 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.771+0000 7f9eddc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80b3f60 con 0x7f9ed80ae360 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ed815b080 0x7f9ed80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ed815b080 0x7f9ed80b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33596/0 (socket says 172.21.15.97:33596) 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 -- 172.21.15.97:0/2887166887 learned_addr learned my addr 172.21.15.97:0/2887166887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 -- 172.21.15.97:0/2887166887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 msgr2=0x7f9ed80cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 --2- 172.21.15.97:0/2887166887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 0x7f9ed80cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 -- 172.21.15.97:0/2887166887 >> v2:172.21.15.188:3300/0 conn(0x7f9ed80ce4b0 msgr2=0x7f9ed815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 --2- 172.21.15.97:0/2887166887 >> v2:172.21.15.188:3300/0 conn(0x7f9ed80ce4b0 0x7f9ed815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.772+0000 7f9ed7e00640 1 -- 172.21.15.97:0/2887166887 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed815c540 con 0x7f9ed815b080 2024-01-31T05:01:51.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.773+0000 7f9ed7e00640 1 --2- 172.21.15.97:0/2887166887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed815b080 0x7f9ed80b3a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80cfd20 tx=0x7f9ec802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de561d4a55e50b96 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:51.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.773+0000 7f9ed6a00640 1 -- 172.21.15.97:0/2887166887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8031640 con 0x7f9ed815b080 2024-01-31T05:01:51.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.773+0000 7f9ed6a00640 1 -- 172.21.15.97:0/2887166887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec80317a0 con 0x7f9ed815b080 2024-01-31T05:01:51.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.773+0000 7f9ed6a00640 1 -- 172.21.15.97:0/2887166887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8031b30 con 0x7f9ed815b080 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 -- 172.21.15.97:0/2887166887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed815b080 msgr2=0x7f9ed80b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 --2- 172.21.15.97:0/2887166887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed815b080 0x7f9ed80b3a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80cfd20 tx=0x7f9ec802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 -- 172.21.15.97:0/2887166887 shutdown_connections 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 --2- 172.21.15.97:0/2887166887 >> v2:172.21.15.188:3300/0 conn(0x7f9ed80ce4b0 0x7f9ed815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 --2- 172.21.15.97:0/2887166887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 0x7f9ed80cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 --2- 172.21.15.97:0/2887166887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed815b080 0x7f9ed80b3a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 -- 172.21.15.97:0/2887166887 >> 172.21.15.97:0/2887166887 conn(0x7f9ed806ad40 msgr2=0x7f9ed80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 -- 172.21.15.97:0/2887166887 shutdown_connections 2024-01-31T05:01:51.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.774+0000 7f9eddc00640 1 -- 172.21.15.97:0/2887166887 wait complete. 2024-01-31T05:01:51.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.776+0000 7f9eddc00640 1 Processor -- start 2024-01-31T05:01:51.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.776+0000 7f9eddc00640 1 -- start start 2024-01-31T05:01:51.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9eddc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 0x7f9ed814bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9eddc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 0x7f9ed80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9eddc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed815b080 0x7f9ed80afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9eddc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8071390 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9eddc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed809bf60 con 0x7f9ed815b080 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9eddc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8070380 con 0x7f9ed80ae360 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.777+0000 7f9ed7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 0x7f9ed80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 0x7f9ed80af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33606/0 (socket says 172.21.15.97:33606) 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 -- 172.21.15.97:0/4028982594 learned_addr learned my addr 172.21.15.97:0/4028982594 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9edd200640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed815b080 0x7f9ed80afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 -- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 msgr2=0x7f9ed814bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 0x7f9ed814bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 -- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed815b080 msgr2=0x7f9ed80afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed815b080 0x7f9ed80afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 -- 172.21.15.97:0/4028982594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80094d0 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ed7400640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 0x7f9ed80af8d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000b4a0 tx=0x7f9ec000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0010610 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.778+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0010770 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.779+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed80705d0 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.779+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0010b20 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.779+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed8070b90 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.780+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed80676b0 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.781+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9ec0010c80 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.782+0000 7f9ecfe00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9eb006aed0 0x7f9eb006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.782+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9ec0089580 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.782+0000 7f9ed7e00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9eb006aed0 0x7f9eb006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:51.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.783+0000 7f9ed7e00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9eb006aed0 0x7f9eb006d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8036000 tx=0x7f9ec8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:51.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.786+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9ec0054100 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.979+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ed80b3230 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.980+0000 7f9ecfe00640 1 -- 172.21.15.97:0/4028982594 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ec0057720 con 0x7f9ed80ce4b0 2024-01-31T05:01:51.980 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:51.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9eb006aed0 msgr2=0x7f9eb006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9eb006aed0 0x7f9eb006d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8036000 tx=0x7f9ec8030000 comp rx=0 tx=0).stop 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 msgr2=0x7f9ed80af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 0x7f9ed80af8d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000b4a0 tx=0x7f9ec000b960 comp rx=0 tx=0).stop 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 shutdown_connections 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ed815b080 0x7f9ed80afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9eb006aed0 0x7f9eb006d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed80ce4b0 0x7f9ed80af8d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 --2- 172.21.15.97:0/4028982594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ed80ae360 0x7f9ed814bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 >> 172.21.15.97:0/4028982594 conn(0x7f9ed806ad40 msgr2=0x7f9ed81555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 shutdown_connections 2024-01-31T05:01:51.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:51.983+0000 7f9eddc00640 1 -- 172.21.15.97:0/4028982594 wait complete. 2024-01-31T05:01:52.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.199+0000 7f97f1400640 1 Processor -- start 2024-01-31T05:01:52.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.199+0000 7f97f1400640 1 -- start start 2024-01-31T05:01:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.200+0000 7f97f1400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97ec15b080 0x7f97ec0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.200+0000 7f97f1400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0b3f60 con 0x7f97ec0ae360 2024-01-31T05:01:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97f1400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec15c540 con 0x7f97ec0ce680 2024-01-31T05:01:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97f1400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0b41c0 con 0x7f97ec15b080 2024-01-31T05:01:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97e3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f97ec0ae360 0x7f97ec0cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36846/0 (socket says 172.21.15.97:36846) 2024-01-31T05:01:52.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97e3e00640 1 -- 172.21.15.97:0/4246998575 learned_addr learned my addr 172.21.15.97:0/4246998575 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622602762 0 0) 0x7f97ec0b3f60 con 0x7f97ec0ae360 2024-01-31T05:01:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97ebe00640 1 --2- 172.21.15.97:0/4246998575 >> v2:172.21.15.188:3300/0 conn(0x7f97ec15b080 0x7f97ec0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97cc003520 con 0x7f97ec0ae360 2024-01-31T05:01:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787688293 0 0) 0x7f97cc003520 con 0x7f97ec0ae360 2024-01-31T05:01:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 >> v2:172.21.15.188:3300/0 conn(0x7f97ec15b080 msgr2=0x7f97ec0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:52.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97eb400640 1 --2- 172.21.15.97:0/4246998575 >> v2:172.21.15.188:3300/0 conn(0x7f97ec15b080 0x7f97ec0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.201+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 >> v1:172.21.15.188:6789/0 conn(0x7f97ec0ce680 legacy=0x7f97ec0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec15b780 con 0x7f97ec0ae360 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f97d4001670 con 0x7f97ec0ae360 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 >> v1:172.21.15.97:6789/0 conn(0x7f97ec0ae360 legacy=0x7f97ec0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97cc003a30 0x7f97cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97cc0047c0 0x7f97cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0b41c0 con 0x7f97cc0052a0 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc00c1a0 con 0x7f97cc0047c0 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc00c360 con 0x7f97cc003a30 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97f0a00640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97f0a00640 1 -- 172.21.15.97:0/4246998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97cc003a30 msgr2=0x7f97cc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97f0a00640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97cc003a30 0x7f97cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97f0a00640 1 -- 172.21.15.97:0/4246998575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97cc0047c0 msgr2=0x7f97cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:52.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97f0a00640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97cc0047c0 0x7f97cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.202+0000 7f97f0a00640 1 -- 172.21.15.97:0/4246998575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc003520 con 0x7f97cc0052a0 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.203+0000 7f97f0a00640 1 -- 172.21.15.97:0/4246998575 reap_dead start 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.203+0000 7f97f0a00640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97cc00b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0b1cc0 tx=0x7f97cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ceb0f6c04defb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4003b70 con 0x7f97cc0052a0 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4003cd0 con 0x7f97cc0052a0 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97eb400640 1 -- 172.21.15.97:0/4246998575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4005310 con 0x7f97cc0052a0 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97f1400640 1 -- 172.21.15.97:0/4246998575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 msgr2=0x7f97cc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97f1400640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97cc00b980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0b1cc0 tx=0x7f97cc0057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97f1400640 1 -- 172.21.15.97:0/4246998575 shutdown_connections 2024-01-31T05:01:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97f1400640 1 --2- 172.21.15.97:0/4246998575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97cc00b980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97f1400640 1 -- 172.21.15.97:0/4246998575 >> 172.21.15.97:0/4246998575 conn(0x7f97ec06ad40 msgr2=0x7f97ec0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.204+0000 7f97f1400640 1 -- 172.21.15.97:0/4246998575 shutdown_connections 2024-01-31T05:01:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.205+0000 7f97f1400640 1 -- 172.21.15.97:0/4246998575 wait complete. 2024-01-31T05:01:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.206+0000 7f97f1400640 1 Processor -- start 2024-01-31T05:01:52.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.206+0000 7f97f1400640 1 -- start start 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.207+0000 7f97f1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97ec1f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.207+0000 7f97f1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97ec09aa10 0x7f97ec06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.207+0000 7f97f1400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97ec157700 0x7f97ec06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.207+0000 7f97f1400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec09af10 con 0x7f97cc0052a0 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.207+0000 7f97f1400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec09b1f0 con 0x7f97ec157700 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.207+0000 7f97f1400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec09b4d0 con 0x7f97ec09aa10 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97ec1f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97ec1f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33624/0 (socket says 172.21.15.97:33624) 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 -- 172.21.15.97:0/1509600523 learned_addr learned my addr 172.21.15.97:0/1509600523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 -- 172.21.15.97:0/1509600523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97ec09aa10 msgr2=0x7f97ec06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97ec09aa10 0x7f97ec06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 -- 172.21.15.97:0/1509600523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97ec157700 msgr2=0x7f97ec06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97ec157700 0x7f97ec06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 -- 172.21.15.97:0/1509600523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec09b7b0 con 0x7f97cc0052a0 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97ebe00640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97ec1f2460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d800 tx=0x7f97dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.208+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00b6c0 con 0x7f97cc0052a0 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.209+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc00b840 con 0x7f97cc0052a0 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.209+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc00bbf0 con 0x7f97cc0052a0 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.209+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec09ba90 con 0x7f97cc0052a0 2024-01-31T05:01:52.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.209+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec09c4b0 con 0x7f97cc0052a0 2024-01-31T05:01:52.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.210+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec0676b0 con 0x7f97cc0052a0 2024-01-31T05:01:52.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.211+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f97dc00bd50 con 0x7f97cc0052a0 2024-01-31T05:01:52.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.211+0000 7f97e8c00640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97c406aed0 0x7f97c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.212+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f97dc089e40 con 0x7f97cc0052a0 2024-01-31T05:01:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.212+0000 7f97e3e00640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97c406aed0 0x7f97c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.212+0000 7f97e3e00640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97c406aed0 0x7f97c406d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97ec1f5f60 tx=0x7f97d4001540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:52.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.216+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f97dc054960 con 0x7f97cc0052a0 2024-01-31T05:01:52.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.408+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97ec070630 con 0x7f97cc0052a0 2024-01-31T05:01:52.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.409+0000 7f97e8c00640 1 -- 172.21.15.97:0/1509600523 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97dc0180a0 con 0x7f97cc0052a0 2024-01-31T05:01:52.408 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:52.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.411+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97c406aed0 msgr2=0x7f97c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.411+0000 7f97f1400640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97c406aed0 0x7f97c406d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f97ec1f5f60 tx=0x7f97d4001540 comp rx=0 tx=0).stop 2024-01-31T05:01:52.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.411+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 msgr2=0x7f97ec1f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.411+0000 7f97f1400640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97ec1f2460 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00d800 tx=0x7f97dc00dcc0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 shutdown_connections 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97ec157700 0x7f97ec06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97ec09aa10 0x7f97ec06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f97c406aed0 0x7f97c406d390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 --2- 172.21.15.97:0/1509600523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97cc0052a0 0x7f97ec1f2460 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 >> 172.21.15.97:0/1509600523 conn(0x7f97ec06ad40 msgr2=0x7f97ec15c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 shutdown_connections 2024-01-31T05:01:52.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.412+0000 7f97f1400640 1 -- 172.21.15.97:0/1509600523 wait complete. 2024-01-31T05:01:52.430 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.634+0000 7fad6dc00640 1 Processor -- start 2024-01-31T05:01:52.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.634+0000 7fad6dc00640 1 -- start start 2024-01-31T05:01:52.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad6dc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad68161b10 0x7fad680b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad6dc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad680cb7d0 0x7fad680cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad6dc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad68161fc0 con 0x7fad680ce1e0 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad6dc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad68162c40 con 0x7fad68161b10 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad6dc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad681622a0 con 0x7fad680cb7d0 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad66a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fad680ce1e0 0x7fad680ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36856/0 (socket says 172.21.15.97:36856) 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad66a00640 1 -- 172.21.15.97:0/297222375 learned_addr learned my addr 172.21.15.97:0/297222375 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:52.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.636+0000 7fad5fe00640 1 --2- 172.21.15.97:0/297222375 >> v2:172.21.15.97:3300/0 conn(0x7fad68161b10 0x7fad680b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad67400640 1 --2- 172.21.15.97:0/297222375 >> v2:172.21.15.188:3300/0 conn(0x7fad680cb7d0 0x7fad680cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad66000640 1 -- 172.21.15.97:0/297222375 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231929192 0 0) 0x7fad68161fc0 con 0x7fad680ce1e0 2024-01-31T05:01:52.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad66000640 1 -- 172.21.15.97:0/297222375 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad48003520 con 0x7fad680ce1e0 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad5fe00640 1 -- 172.21.15.97:0/297222375 >> v2:172.21.15.188:3300/0 conn(0x7fad680cb7d0 msgr2=0x7fad680cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad5fe00640 1 --2- 172.21.15.97:0/297222375 >> v2:172.21.15.188:3300/0 conn(0x7fad680cb7d0 0x7fad680cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad5fe00640 1 -- 172.21.15.97:0/297222375 >> v1:172.21.15.97:6789/0 conn(0x7fad680ce1e0 legacy=0x7fad680ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.637+0000 7fad5fe00640 1 -- 172.21.15.97:0/297222375 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68162580 con 0x7fad68161b10 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.638+0000 7fad5fe00640 1 --2- 172.21.15.97:0/297222375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad680b2fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad680ade50 tx=0x7fad5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5a543c773b1183 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.638+0000 7fad66000640 1 -- 172.21.15.97:0/297222375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5002e070 con 0x7fad68161b10 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.638+0000 7fad66000640 1 -- 172.21.15.97:0/297222375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad500315a0 con 0x7fad68161b10 2024-01-31T05:01:52.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.638+0000 7fad66000640 1 -- 172.21.15.97:0/297222375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50031910 con 0x7fad68161b10 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 -- 172.21.15.97:0/297222375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 msgr2=0x7fad680b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 --2- 172.21.15.97:0/297222375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad680b2fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fad680ade50 tx=0x7fad5002b840 comp rx=0 tx=0).stop 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 -- 172.21.15.97:0/297222375 shutdown_connections 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 --2- 172.21.15.97:0/297222375 >> v2:172.21.15.188:3300/0 conn(0x7fad680cb7d0 0x7fad680cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 --2- 172.21.15.97:0/297222375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad680b2fc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 -- 172.21.15.97:0/297222375 >> 172.21.15.97:0/297222375 conn(0x7fad680b0a10 msgr2=0x7fad680b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 -- 172.21.15.97:0/297222375 shutdown_connections 2024-01-31T05:01:52.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.639+0000 7fad6dc00640 1 -- 172.21.15.97:0/297222375 wait complete. 2024-01-31T05:01:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.640+0000 7fad6dc00640 1 Processor -- start 2024-01-31T05:01:52.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.641+0000 7fad6dc00640 1 -- start start 2024-01-31T05:01:52.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.641+0000 7fad6dc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad680cb7d0 0x7fad681f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad6dc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad680ce1e0 0x7fad681f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad6dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad6815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad6dc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6815ce00 con 0x7fad68161b10 2024-01-31T05:01:52.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad6dc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad681f8130 con 0x7fad680cb7d0 2024-01-31T05:01:52.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad6dc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6815c840 con 0x7fad680ce1e0 2024-01-31T05:01:52.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad6815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad6815b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33636/0 (socket says 172.21.15.97:33636) 2024-01-31T05:01:52.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 -- 172.21.15.97:0/4110793398 learned_addr learned my addr 172.21.15.97:0/4110793398 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 -- 172.21.15.97:0/4110793398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad680ce1e0 msgr2=0x7fad681f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad680ce1e0 0x7fad681f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 -- 172.21.15.97:0/4110793398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad680cb7d0 msgr2=0x7fad681f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad680cb7d0 0x7fad681f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 -- 172.21.15.97:0/4110793398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad50009400 con 0x7fad68161b10 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.642+0000 7fad67400640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad6815b9a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad60007e90 tx=0x7fad6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.643+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60014420 con 0x7fad68161b10 2024-01-31T05:01:52.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.643+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad60014580 con 0x7fad68161b10 2024-01-31T05:01:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.643+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60014930 con 0x7fad68161b10 2024-01-31T05:01:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.643+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6815cb20 con 0x7fad68161b10 2024-01-31T05:01:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.643+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad681f76d0 con 0x7fad68161b10 2024-01-31T05:01:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.644+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad68067830 con 0x7fad68161b10 2024-01-31T05:01:52.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.645+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fad60014a90 con 0x7fad68161b10 2024-01-31T05:01:52.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.646+0000 7fad5ea00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fad4006aed0 0x7fad4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:52.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.646+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fad6008a3e0 con 0x7fad68161b10 2024-01-31T05:01:52.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.646+0000 7fad66a00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fad4006aed0 0x7fad4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:52.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.647+0000 7fad66a00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fad4006aed0 0x7fad4006d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad58007fd0 tx=0x7fad580077c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.651+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fad60054ec0 con 0x7fad68161b10 2024-01-31T05:01:52.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.844+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fad680cc6c0 con 0x7fad68161b10 2024-01-31T05:01:52.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.845+0000 7fad5ea00640 1 -- 172.21.15.97:0/4110793398 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fad600584e0 con 0x7fad68161b10 2024-01-31T05:01:52.844 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fad4006aed0 msgr2=0x7fad4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fad4006aed0 0x7fad4006d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad58007fd0 tx=0x7fad580077c0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 msgr2=0x7fad6815b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad6815b9a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fad60007e90 tx=0x7fad6000e4b0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 shutdown_connections 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fad4006aed0 0x7fad4006d390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad68161b10 0x7fad6815b9a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.847+0000 7fad6dc00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad680ce1e0 0x7fad681f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.848+0000 7fad6dc00640 1 --2- 172.21.15.97:0/4110793398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad680cb7d0 0x7fad681f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.848+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 >> 172.21.15.97:0/4110793398 conn(0x7fad680b0a10 msgr2=0x7fad680b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.848+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 shutdown_connections 2024-01-31T05:01:52.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:52.848+0000 7fad6dc00640 1 -- 172.21.15.97:0/4110793398 wait complete. 2024-01-31T05:01:52.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:01:53.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.074+0000 7fa534000640 1 Processor -- start 2024-01-31T05:01:53.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.074+0000 7fa534000640 1 -- start start 2024-01-31T05:01:53.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.076+0000 7fa534000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa52c0b2570 0x7fa52c0cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.076+0000 7fa534000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa52c0ce000 0x7fa52c15cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.076+0000 7fa534000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0cb760 con 0x7fa52c0d1b90 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.076+0000 7fa534000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c06ce00 con 0x7fa52c0b2570 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.076+0000 7fa534000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0cd4f0 con 0x7fa52c0ce000 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa52c0b2570 0x7fa52c0cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa52c0b2570 0x7fa52c0cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33646/0 (socket says 172.21.15.97:33646) 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 -- 172.21.15.97:0/158037528 learned_addr learned my addr 172.21.15.97:0/158037528 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa532000640 1 --2- 172.21.15.97:0/158037528 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0ce000 0x7fa52c15cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 -- 172.21.15.97:0/158037528 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0ce000 msgr2=0x7fa52c15cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 --2- 172.21.15.97:0/158037528 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0ce000 0x7fa52c15cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 -- 172.21.15.97:0/158037528 >> v1:172.21.15.97:6789/0 conn(0x7fa52c0d1b90 legacy=0x7fa52c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa52be00640 1 -- 172.21.15.97:0/158037528 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160033519 0 0) 0x7fa52c0cb760 con 0x7fa52c0d1b90 2024-01-31T05:01:53.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.077+0000 7fa531600640 1 -- 172.21.15.97:0/158037528 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0b2950 con 0x7fa52c0b2570 2024-01-31T05:01:53.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.078+0000 7fa531600640 1 --2- 172.21.15.97:0/158037528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0b2570 0x7fa52c0cf3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa52c066e10 tx=0x7fa51c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead91f08fda91ba server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.078+0000 7fa52be00640 1 -- 172.21.15.97:0/158037528 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c02cb20 con 0x7fa52c0b2570 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa52be00640 1 -- 172.21.15.97:0/158037528 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c02cc80 con 0x7fa52c0b2570 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa52be00640 1 -- 172.21.15.97:0/158037528 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0343f0 con 0x7fa52c0b2570 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 -- 172.21.15.97:0/158037528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0b2570 msgr2=0x7fa52c0cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 --2- 172.21.15.97:0/158037528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0b2570 0x7fa52c0cf3e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa52c066e10 tx=0x7fa51c02bf80 comp rx=0 tx=0).stop 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 -- 172.21.15.97:0/158037528 shutdown_connections 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 --2- 172.21.15.97:0/158037528 >> v2:172.21.15.188:3300/0 conn(0x7fa52c0ce000 0x7fa52c15cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 --2- 172.21.15.97:0/158037528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0b2570 0x7fa52c0cf3e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 -- 172.21.15.97:0/158037528 >> 172.21.15.97:0/158037528 conn(0x7fa52c0d1070 msgr2=0x7fa52c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 -- 172.21.15.97:0/158037528 shutdown_connections 2024-01-31T05:01:53.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.079+0000 7fa534000640 1 -- 172.21.15.97:0/158037528 wait complete. 2024-01-31T05:01:53.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.081+0000 7fa534000640 1 Processor -- start 2024-01-31T05:01:53.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.081+0000 7fa534000640 1 -- start start 2024-01-31T05:01:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.082+0000 7fa534000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0b2570 0x7fa52c0c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.082+0000 7fa534000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 0x7fa52c1e7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.082+0000 7fa534000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0d1b90 0x7fa52c1d8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.082+0000 7fa534000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0c6680 con 0x7fa52c0ce000 2024-01-31T05:01:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.082+0000 7fa534000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c1e73c0 con 0x7fa52c0d1b90 2024-01-31T05:01:53.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.082+0000 7fa534000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c1e7d90 con 0x7fa52c0b2570 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 0x7fa52c1e7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 0x7fa52c1e7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33656/0 (socket says 172.21.15.97:33656) 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 -- 172.21.15.97:0/1615199634 learned_addr learned my addr 172.21.15.97:0/1615199634 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa532000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0d1b90 0x7fa52c1d8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa531600640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0b2570 0x7fa52c0c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 -- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0b2570 msgr2=0x7fa52c0c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0b2570 0x7fa52c0c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 -- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0d1b90 msgr2=0x7fa52c1d8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0d1b90 0x7fa52c1d8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 -- 172.21.15.97:0/1615199634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c009b30 con 0x7fa52c0ce000 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.083+0000 7fa530c00640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 0x7fa52c1e7850 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c4b0 tx=0x7fa51400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.084+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514005480 con 0x7fa52c0ce000 2024-01-31T05:01:53.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.084+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5140035f0 con 0x7fa52c0ce000 2024-01-31T05:01:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.084+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140039e0 con 0x7fa52c0ce000 2024-01-31T05:01:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.084+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c1e64f0 con 0x7fa52c0ce000 2024-01-31T05:01:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.084+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c1e6a90 con 0x7fa52c0ce000 2024-01-31T05:01:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.085+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c067830 con 0x7fa52c0ce000 2024-01-31T05:01:53.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.086+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa514003b40 con 0x7fa52c0ce000 2024-01-31T05:01:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.087+0000 7fa529600640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa50406aed0 0x7fa50406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.087+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa514089040 con 0x7fa52c0ce000 2024-01-31T05:01:53.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.087+0000 7fa531600640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa50406aed0 0x7fa50406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.088+0000 7fa531600640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa50406aed0 0x7fa50406d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa51c035000 tx=0x7fa51c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:53.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.091+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa5140bc190 con 0x7fa52c0ce000 2024-01-31T05:01:53.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.285+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa52c06cf80 con 0x7fa52c0ce000 2024-01-31T05:01:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.286+0000 7fa529600640 1 -- 172.21.15.97:0/1615199634 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa514017070 con 0x7fa52c0ce000 2024-01-31T05:01:53.285 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:01:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa50406aed0 msgr2=0x7fa50406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa50406aed0 0x7fa50406d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa51c035000 tx=0x7fa51c030000 comp rx=0 tx=0).stop 2024-01-31T05:01:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 msgr2=0x7fa52c1e7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 0x7fa52c1e7850 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c4b0 tx=0x7fa51400c970 comp rx=0 tx=0).stop 2024-01-31T05:01:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 shutdown_connections 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c0d1b90 0x7fa52c1d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa50406aed0 0x7fa50406d390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa52c0ce000 0x7fa52c1e7850 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.288+0000 7fa534000640 1 --2- 172.21.15.97:0/1615199634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa52c0b2570 0x7fa52c0c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.289+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 >> 172.21.15.97:0/1615199634 conn(0x7fa52c0d1070 msgr2=0x7fa52c06faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.289+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 shutdown_connections 2024-01-31T05:01:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.289+0000 7fa534000640 1 -- 172.21.15.97:0/1615199634 wait complete. 2024-01-31T05:01:53.305 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:01:53.305 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:53.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:53.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.607+0000 7fe915400640 1 Processor -- start 2024-01-31T05:01:53.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.607+0000 7fe915400640 1 -- start start 2024-01-31T05:01:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe915400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9101616d0 0x7fe9100cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe915400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9100abcf0 con 0x7fe9100ce430 2024-01-31T05:01:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe915400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91006ca40 con 0x7fe91015d520 2024-01-31T05:01:53.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe915400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91015d900 con 0x7fe9101616d0 2024-01-31T05:01:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe90de00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9101616d0 0x7fe9100cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe90de00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9101616d0 0x7fe9100cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33662/0 (socket says 172.21.15.97:33662) 2024-01-31T05:01:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe90de00640 1 -- 172.21.15.97:0/3022513036 learned_addr learned my addr 172.21.15.97:0/3022513036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.609+0000 7fe90d400640 1 -- 172.21.15.97:0/3022513036 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392279314 0 0) 0x7fe9100abcf0 con 0x7fe9100ce430 2024-01-31T05:01:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.610+0000 7fe90d400640 1 -- 172.21.15.97:0/3022513036 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f0003520 con 0x7fe9100ce430 2024-01-31T05:01:53.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.610+0000 7fe90de00640 1 -- 172.21.15.97:0/3022513036 >> v1:172.21.15.188:6789/0 conn(0x7fe91015d520 legacy=0x7fe910160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.610+0000 7fe90de00640 1 -- 172.21.15.97:0/3022513036 >> v1:172.21.15.97:6789/0 conn(0x7fe9100ce430 legacy=0x7fe91006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.610+0000 7fe90de00640 1 -- 172.21.15.97:0/3022513036 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910161e20 con 0x7fe9101616d0 2024-01-31T05:01:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.611+0000 7fe90de00640 1 --2- 172.21.15.97:0/3022513036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9101616d0 0x7fe9100cd710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009870 tx=0x7fe8f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bcb2beb0037a69 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.611+0000 7fe90d400640 1 -- 172.21.15.97:0/3022513036 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8030860 con 0x7fe9101616d0 2024-01-31T05:01:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.611+0000 7fe90d400640 1 -- 172.21.15.97:0/3022513036 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f80309c0 con 0x7fe9101616d0 2024-01-31T05:01:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.611+0000 7fe90d400640 1 -- 172.21.15.97:0/3022513036 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8030d50 con 0x7fe9101616d0 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 -- 172.21.15.97:0/3022513036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9101616d0 msgr2=0x7fe9100cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 --2- 172.21.15.97:0/3022513036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9101616d0 0x7fe9100cd710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009870 tx=0x7fe8f802b9a0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 -- 172.21.15.97:0/3022513036 shutdown_connections 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 --2- 172.21.15.97:0/3022513036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9101616d0 0x7fe9100cd710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 -- 172.21.15.97:0/3022513036 >> 172.21.15.97:0/3022513036 conn(0x7fe9100b0a10 msgr2=0x7fe9100b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 -- 172.21.15.97:0/3022513036 shutdown_connections 2024-01-31T05:01:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.612+0000 7fe915400640 1 -- 172.21.15.97:0/3022513036 wait complete. 2024-01-31T05:01:53.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.614+0000 7fe915400640 1 Processor -- start 2024-01-31T05:01:53.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.614+0000 7fe915400640 1 -- start start 2024-01-31T05:01:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe915400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9100ce430 0x7fe910160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe915400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 0x7fe910151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe915400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9101616d0 0x7fe910151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe915400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910070ac0 con 0x7fe91015d520 2024-01-31T05:01:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe915400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe910070420 con 0x7fe9101616d0 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe915400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91015bff0 con 0x7fe9100ce430 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 0x7fe910151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 0x7fe910151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33670/0 (socket says 172.21.15.97:33670) 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 -- 172.21.15.97:0/4067886834 learned_addr learned my addr 172.21.15.97:0/4067886834 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 -- 172.21.15.97:0/4067886834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9100ce430 msgr2=0x7fe910160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9100ce430 0x7fe910160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 -- 172.21.15.97:0/4067886834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9101616d0 msgr2=0x7fe910151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9101616d0 0x7fe910151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.615+0000 7fe90de00640 1 -- 172.21.15.97:0/4067886834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f80094d0 con 0x7fe91015d520 2024-01-31T05:01:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.616+0000 7fe90de00640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 0x7fe910151580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80090b0 tx=0x7fe8f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.616+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f803e070 con 0x7fe91015d520 2024-01-31T05:01:53.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.616+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8039800 con 0x7fe91015d520 2024-01-31T05:01:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.616+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8039bb0 con 0x7fe91015d520 2024-01-31T05:01:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.616+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91015c2d0 con 0x7fe91015d520 2024-01-31T05:01:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.616+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9101f2f10 con 0x7fe91015d520 2024-01-31T05:01:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.617+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9101f3160 con 0x7fe91015d520 2024-01-31T05:01:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.618+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe8f8042070 con 0x7fe91015d520 2024-01-31T05:01:53.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.619+0000 7fe906a00640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8e806aed0 0x7fe8e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:53.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.619+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe8f80ab6d0 con 0x7fe91015d520 2024-01-31T05:01:53.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.619+0000 7fe90e800640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8e806aed0 0x7fe8e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:53.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.620+0000 7fe90e800640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8e806aed0 0x7fe8e806d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe900003cc0 tx=0x7fe9000079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:53.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.623+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe8f80761b0 con 0x7fe91015d520 2024-01-31T05:01:53.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.817+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9100b1130 con 0x7fe91015d520 2024-01-31T05:01:53.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.819+0000 7fe906a00640 1 -- 172.21.15.97:0/4067886834 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7fe8f80797d0 con 0x7fe91015d520 2024-01-31T05:01:53.819 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:53.819 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8e806aed0 msgr2=0x7fe8e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8e806aed0 0x7fe8e806d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe900003cc0 tx=0x7fe9000079b0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 msgr2=0x7fe910151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 0x7fe910151580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80090b0 tx=0x7fe8f80046c0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 shutdown_connections 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9101616d0 0x7fe910151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe8e806aed0 0x7fe8e806d390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe91015d520 0x7fe910151580 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 --2- 172.21.15.97:0/4067886834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9100ce430 0x7fe910160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 >> 172.21.15.97:0/4067886834 conn(0x7fe9100b0a10 msgr2=0x7fe9100b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 shutdown_connections 2024-01-31T05:01:53.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:53.822+0000 7fe915400640 1 -- 172.21.15.97:0/4067886834 wait complete. 2024-01-31T05:01:53.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:53.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.050+0000 7f22db800640 1 Processor -- start 2024-01-31T05:01:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.050+0000 7f22db800640 1 -- start start 2024-01-31T05:01:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22db800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 0x7f22d40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22db800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d406cd20 con 0x7f22d415b470 2024-01-31T05:01:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22db800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d40ceb60 con 0x7f22d40abcf0 2024-01-31T05:01:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22db800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d415b850 con 0x7f22d40cbf20 2024-01-31T05:01:54.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22d8e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f22d415b470 0x7f22d40cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36886/0 (socket says 172.21.15.97:36886) 2024-01-31T05:01:54.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22d8e00640 1 -- 172.21.15.97:0/1509162134 learned_addr learned my addr 172.21.15.97:0/1509162134 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:54.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22cbc00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 0x7f22d40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.052+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560499529 0 0) 0x7f22d406cd20 con 0x7f22d415b470 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b4003520 con 0x7f22d415b470 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2615616237 0 0) 0x7f22b4003520 con 0x7f22d415b470 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 msgr2=0x7f22d40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 0x7f22d40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 >> v1:172.21.15.188:6789/0 conn(0x7f22d40abcf0 legacy=0x7f22d406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d406c3f0 con 0x7f22d415b470 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f22c4001b70 con 0x7f22d415b470 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 >> v1:172.21.15.97:6789/0 conn(0x7f22d415b470 legacy=0x7f22d40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b40039a0 0x7f22b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40047c0 0x7f22b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 0x7f22b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d415b850 con 0x7f22b40039a0 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b400ba60 con 0x7f22b40047c0 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b400bc20 con 0x7f22d40cbf20 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.053+0000 7f22d8e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b40039a0 0x7f22b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.054+0000 7f22d8e00640 1 -- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 msgr2=0x7f22b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.054+0000 7f22d8e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d40cbf20 0x7f22b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.054+0000 7f22d8e00640 1 -- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40047c0 msgr2=0x7f22b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.054+0000 7f22d8e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40047c0 0x7f22b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.054+0000 7f22d8e00640 1 -- 172.21.15.97:0/1509162134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b4003520 con 0x7f22b40039a0 2024-01-31T05:01:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.055+0000 7f22d8e00640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b40039a0 0x7f22b4003dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c410 tx=0x7f22b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ae4b8709094a8f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.055+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c4005d20 con 0x7f22b40039a0 2024-01-31T05:01:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.055+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4005e80 con 0x7f22b40039a0 2024-01-31T05:01:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.055+0000 7f22d3e00640 1 -- 172.21.15.97:0/1509162134 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c402cd10 con 0x7f22b40039a0 2024-01-31T05:01:54.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22db800640 1 -- 172.21.15.97:0/1509162134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b40039a0 msgr2=0x7f22b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22db800640 1 --2- 172.21.15.97:0/1509162134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22b40039a0 0x7f22b4003dc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22c400c410 tx=0x7f22b40057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22d9800640 1 -- 172.21.15.97:0/1509162134 reap_dead start 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22db800640 1 -- 172.21.15.97:0/1509162134 shutdown_connections 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22db800640 1 -- 172.21.15.97:0/1509162134 >> 172.21.15.97:0/1509162134 conn(0x7f22d40b0a10 msgr2=0x7f22d40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22db800640 1 -- 172.21.15.97:0/1509162134 shutdown_connections 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.056+0000 7f22db800640 1 -- 172.21.15.97:0/1509162134 wait complete. 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.058+0000 7f22db800640 1 Processor -- start 2024-01-31T05:01:54.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.058+0000 7f22db800640 1 -- start start 2024-01-31T05:01:54.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22db800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40039a0 0x7f22d415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22db800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 0x7f22d41506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22db800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d4151d20 0x7f22d4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22db800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4154b50 con 0x7f22d41546f0 2024-01-31T05:01:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22db800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4150c20 con 0x7f22b40039a0 2024-01-31T05:01:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22db800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d41525d0 con 0x7f22d4151d20 2024-01-31T05:01:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 0x7f22d41506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 0x7f22d41506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33694/0 (socket says 172.21.15.97:33694) 2024-01-31T05:01:54.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 -- 172.21.15.97:0/4282948404 learned_addr learned my addr 172.21.15.97:0/4282948404 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 -- 172.21.15.97:0/4282948404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d4151d20 msgr2=0x7f22d4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d4151d20 0x7f22d4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 -- 172.21.15.97:0/4282948404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40039a0 msgr2=0x7f22d415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40039a0 0x7f22d415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.059+0000 7f22cbc00640 1 -- 172.21.15.97:0/4282948404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d41528b0 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.060+0000 7f22cbc00640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 0x7f22d41506e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00e930 tx=0x7f22bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.060+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc009680 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.060+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc00c940 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.060+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc00cd10 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.060+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d4152b90 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.060+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d415bf70 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.061+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4067830 con 0x7f22d41546f0 2024-01-31T05:01:54.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.062+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f22bc013470 con 0x7f22d41546f0 2024-01-31T05:01:54.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.063+0000 7f22d1600640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f22ac06af00 0x7f22ac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.063+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f22bc089ab0 con 0x7f22d41546f0 2024-01-31T05:01:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.063+0000 7f22d8e00640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f22ac06af00 0x7f22ac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.064+0000 7f22d8e00640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f22ac06af00 0x7f22ac06d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034000 tx=0x7f22c4007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.067+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f22bc054590 con 0x7f22d41546f0 2024-01-31T05:01:54.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.259+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22d4150f30 con 0x7f22d41546f0 2024-01-31T05:01:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.261+0000 7f22d1600640 1 -- 172.21.15.97:0/4282948404 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22bc057bb0 con 0x7f22d41546f0 2024-01-31T05:01:54.260 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:54.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f22ac06af00 msgr2=0x7f22ac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f22ac06af00 0x7f22ac06d3c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22c4034000 tx=0x7f22c4007000 comp rx=0 tx=0).stop 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 msgr2=0x7f22d41506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 0x7f22d41506e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22bc00e930 tx=0x7f22bc00edf0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 shutdown_connections 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d4151d20 0x7f22d4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f22ac06af00 0x7f22ac06d3c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22d41546f0 0x7f22d41506e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 --2- 172.21.15.97:0/4282948404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22b40039a0 0x7f22d415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.263+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 >> 172.21.15.97:0/4282948404 conn(0x7f22d40b0a10 msgr2=0x7f22d40cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.264+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 shutdown_connections 2024-01-31T05:01:54.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.264+0000 7f22db800640 1 -- 172.21.15.97:0/4282948404 wait complete. 2024-01-31T05:01:54.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:54.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.495+0000 7fd47e600640 1 Processor -- start 2024-01-31T05:01:54.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.495+0000 7fd47e600640 1 -- start start 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.496+0000 7fd47e600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4780cfca0 0x7fd4780b3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.496+0000 7fd47e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd4780ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.496+0000 7fd47e600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4780d1b20 con 0x7fd4780d3bb0 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.496+0000 7fd47e600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4780d1e00 con 0x7fd4780cfca0 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.496+0000 7fd47e600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780d20e0 con 0x7fd4780b3e90 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.497+0000 7fd476a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd4780d3bb0 0x7fd4780b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36896/0 (socket says 172.21.15.97:36896) 2024-01-31T05:01:54.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.497+0000 7fd476a00640 1 -- 172.21.15.97:0/4093955405 learned_addr learned my addr 172.21.15.97:0/4093955405 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:54.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.497+0000 7fd477e00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd4780ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.497+0000 7fd477400640 1 --2- 172.21.15.97:0/4093955405 >> v2:172.21.15.188:3300/0 conn(0x7fd4780cfca0 0x7fd4780b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.497+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33170850 0 0) 0x7fd4780d1b20 con 0x7fd4780d3bb0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.497+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd458003520 con 0x7fd4780d3bb0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2178448140 0 0) 0x7fd458003520 con 0x7fd4780d3bb0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 msgr2=0x7fd4780ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd4780ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 >> v2:172.21.15.188:3300/0 conn(0x7fd4780cfca0 msgr2=0x7fd4780b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 --2- 172.21.15.97:0/4093955405 >> v2:172.21.15.188:3300/0 conn(0x7fd4780cfca0 0x7fd4780b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780d4120 con 0x7fd4780d3bb0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fd460001670 con 0x7fd4780d3bb0 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 >> v1:172.21.15.97:6789/0 conn(0x7fd4780d3bb0 legacy=0x7fd4780b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd458003970 0x7fd458003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4580047c0 0x7fd458004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd458005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780d1e00 con 0x7fd4580047c0 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780d20e0 con 0x7fd458003970 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45800b980 con 0x7fd4780b3e90 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd476a00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4580047c0 0x7fd458004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd477e00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd458005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.498+0000 7fd477400640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd458003970 0x7fd458003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd476a00640 1 -- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 msgr2=0x7fd458005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd476a00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd458005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd476a00640 1 -- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd458003970 msgr2=0x7fd458003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd476a00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd458003970 0x7fd458003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd476a00640 1 -- 172.21.15.97:0/4093955405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd458003520 con 0x7fd4580047c0 2024-01-31T05:01:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd477e00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4780b3e90 0x7fd458005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:54.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.499+0000 7fd477400640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd458003970 0x7fd458003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd476a00640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4580047c0 0x7fd458004c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c240 tx=0x7fd4580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257fde35c5ac45ba server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46002cb00 con 0x7fd4580047c0 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46002cc60 con 0x7fd4580047c0 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd476000640 1 -- 172.21.15.97:0/4093955405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460005af0 con 0x7fd4580047c0 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd47e600640 1 -- 172.21.15.97:0/4093955405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4580047c0 msgr2=0x7fd458004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd47e600640 1 --2- 172.21.15.97:0/4093955405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4580047c0 0x7fd458004c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c240 tx=0x7fd4580057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.500+0000 7fd477e00640 1 -- 172.21.15.97:0/4093955405 reap_dead start 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.501+0000 7fd47e600640 1 -- 172.21.15.97:0/4093955405 shutdown_connections 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.501+0000 7fd47e600640 1 -- 172.21.15.97:0/4093955405 >> 172.21.15.97:0/4093955405 conn(0x7fd47809c570 msgr2=0x7fd4780d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:54.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.501+0000 7fd47e600640 1 -- 172.21.15.97:0/4093955405 shutdown_connections 2024-01-31T05:01:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.501+0000 7fd47e600640 1 -- 172.21.15.97:0/4093955405 wait complete. 2024-01-31T05:01:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.502+0000 7fd47e600640 1 Processor -- start 2024-01-31T05:01:54.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 -- start start 2024-01-31T05:01:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4580047c0 0x7fd47814f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 0x7fd47815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd47815f3e0 0x7fd47815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47815ecc0 con 0x7fd47815bc10 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47815fda0 con 0x7fd4580047c0 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.503+0000 7fd47e600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47815e040 con 0x7fd47815f3e0 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 0x7fd47815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 0x7fd47815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33716/0 (socket says 172.21.15.97:33716) 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 -- 172.21.15.97:0/3393682978 learned_addr learned my addr 172.21.15.97:0/3393682978 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd477e00640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd47815f3e0 0x7fd47815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd477400640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4580047c0 0x7fd47814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 -- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd47815f3e0 msgr2=0x7fd47815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd47815f3e0 0x7fd47815f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 -- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4580047c0 msgr2=0x7fd47814f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4580047c0 0x7fd47814f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 -- 172.21.15.97:0/3393682978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47815e320 con 0x7fd47815bc10 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.504+0000 7fd476a00640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 0x7fd47815d810 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c240 tx=0x7fd460001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.505+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46003c040 con 0x7fd47815bc10 2024-01-31T05:01:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.505+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460004bf0 con 0x7fd47815bc10 2024-01-31T05:01:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.505+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460007070 con 0x7fd47815bc10 2024-01-31T05:01:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.505+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47815e600 con 0x7fd47815bc10 2024-01-31T05:01:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.505+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47815dd50 con 0x7fd47815bc10 2024-01-31T05:01:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.506+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd478067830 con 0x7fd47815bc10 2024-01-31T05:01:54.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.507+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd460032070 con 0x7fd47815bc10 2024-01-31T05:01:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.508+0000 7fd46f400640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd45006aed0 0x7fd45006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.508+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd4600aa620 con 0x7fd47815bc10 2024-01-31T05:01:54.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.508+0000 7fd477400640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd45006aed0 0x7fd45006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.509+0000 7fd477400640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd45006aed0 0x7fd45006d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd468009a40 tx=0x7fd468008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.512+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd460075290 con 0x7fd47815bc10 2024-01-31T05:01:54.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.710+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd478160a70 con 0x7fd47815bc10 2024-01-31T05:01:54.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.711+0000 7fd46f400640 1 -- 172.21.15.97:0/3393682978 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4600788b0 con 0x7fd47815bc10 2024-01-31T05:01:54.711 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd45006aed0 msgr2=0x7fd45006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd45006aed0 0x7fd45006d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd468009a40 tx=0x7fd468008040 comp rx=0 tx=0).stop 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 msgr2=0x7fd47815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 0x7fd47815d810 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c240 tx=0x7fd460001a00 comp rx=0 tx=0).stop 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 shutdown_connections 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd47815f3e0 0x7fd47815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd45006aed0 0x7fd45006d390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47815bc10 0x7fd47815d810 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 --2- 172.21.15.97:0/3393682978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4580047c0 0x7fd47814f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 >> 172.21.15.97:0/3393682978 conn(0x7fd47809c570 msgr2=0x7fd4780b8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:54.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.714+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 shutdown_connections 2024-01-31T05:01:54.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.715+0000 7fd47e600640 1 -- 172.21.15.97:0/3393682978 wait complete. 2024-01-31T05:01:54.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.948+0000 7fc8af800640 1 Processor -- start 2024-01-31T05:01:54.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.948+0000 7fc8af800640 1 -- start start 2024-01-31T05:01:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8af800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8a815b470 0x7fc8a80722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8af800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a80cbf20 0x7fc8a80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8af800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a806cd20 con 0x7fc8a80abcf0 2024-01-31T05:01:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8af800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a80ce970 con 0x7fc8a815b470 2024-01-31T05:01:54.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8af800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a815b850 con 0x7fc8a80cbf20 2024-01-31T05:01:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8ace00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8a815b470 0x7fc8a80722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8ad800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a80cbf20 0x7fc8a80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8ace00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8a815b470 0x7fc8a80722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:37770/0 (socket says 172.21.15.97:37770) 2024-01-31T05:01:54.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.950+0000 7fc8ace00640 1 -- 172.21.15.97:0/794703088 learned_addr learned my addr 172.21.15.97:0/794703088 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.951+0000 7fc8ace00640 1 -- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a80cbf20 msgr2=0x7fc8a80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.951+0000 7fc8ace00640 1 --2- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a80cbf20 0x7fc8a80ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.951+0000 7fc8ace00640 1 -- 172.21.15.97:0/794703088 >> v1:172.21.15.188:6789/0 conn(0x7fc8a80abcf0 legacy=0x7fc8a80cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.951+0000 7fc8ace00640 1 -- 172.21.15.97:0/794703088 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80ceb60 con 0x7fc8a815b470 2024-01-31T05:01:54.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.953+0000 7fc8ace00640 1 --2- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a815b470 0x7fc8a80722c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc898009880 tx=0x7fc89802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bebc0ace5be59c07 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.953+0000 7fc8a7400640 1 -- 172.21.15.97:0/794703088 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898004230 con 0x7fc8a815b470 2024-01-31T05:01:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8a7400640 1 -- 172.21.15.97:0/794703088 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898004390 con 0x7fc8a815b470 2024-01-31T05:01:54.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8a7400640 1 -- 172.21.15.97:0/794703088 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898003810 con 0x7fc8a815b470 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 -- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a815b470 msgr2=0x7fc8a80722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 --2- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a815b470 0x7fc8a80722c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc898009880 tx=0x7fc89802b960 comp rx=0 tx=0).stop 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 -- 172.21.15.97:0/794703088 shutdown_connections 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 --2- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a80cbf20 0x7fc8a80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 --2- 172.21.15.97:0/794703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a815b470 0x7fc8a80722c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 -- 172.21.15.97:0/794703088 >> 172.21.15.97:0/794703088 conn(0x7fc8a80b0a10 msgr2=0x7fc8a80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 -- 172.21.15.97:0/794703088 shutdown_connections 2024-01-31T05:01:54.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.954+0000 7fc8af800640 1 -- 172.21.15.97:0/794703088 wait complete. 2024-01-31T05:01:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.956+0000 7fc8af800640 1 Processor -- start 2024-01-31T05:01:54.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.956+0000 7fc8af800640 1 -- start start 2024-01-31T05:01:54.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.957+0000 7fc8af800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a80abcf0 0x7fc8a8144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.957+0000 7fc8af800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 0x7fc8a8159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.957+0000 7fc8af800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a815b470 0x7fc8a815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.957+0000 7fc8af800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8145a60 con 0x7fc8a80cbf20 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.957+0000 7fc8af800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8148010 con 0x7fc8a80abcf0 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.957+0000 7fc8af800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8145130 con 0x7fc8a815b470 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 0x7fc8a8159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 0x7fc8a8159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39836/0 (socket says 172.21.15.97:39836) 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 -- 172.21.15.97:0/647626576 learned_addr learned my addr 172.21.15.97:0/647626576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8ace00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a80abcf0 0x7fc8a8144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8ad800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a815b470 0x7fc8a815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 -- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a815b470 msgr2=0x7fc8a815a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a815b470 0x7fc8a815a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 -- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a80abcf0 msgr2=0x7fc8a8144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:54.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a80abcf0 0x7fc8a8144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:54.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 -- 172.21.15.97:0/647626576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898009460 con 0x7fc8a80cbf20 2024-01-31T05:01:54.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.958+0000 7fc8a7e00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 0x7fc8a8159c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c440 tx=0x7fc890001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.959+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900052d0 con 0x7fc8a80cbf20 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.959+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890005430 con 0x7fc8a80cbf20 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.959+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890007260 con 0x7fc8a80cbf20 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.959+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a8145410 con 0x7fc8a80cbf20 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.959+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81e39b0 con 0x7fc8a80cbf20 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.960+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a806d190 con 0x7fc8a80cbf20 2024-01-31T05:01:54.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.961+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc8900073c0 con 0x7fc8a80cbf20 2024-01-31T05:01:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.962+0000 7fc8a4c00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88006aed0 0x7fc88006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.962+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc890088eb0 con 0x7fc8a80cbf20 2024-01-31T05:01:54.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.962+0000 7fc8ace00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88006aed0 0x7fc88006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:54.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.963+0000 7fc8ace00640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88006aed0 0x7fc88006d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc898035000 tx=0x7fc898008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:54.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:54.966+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc8a806d190 con 0x7fc8a80cbf20 2024-01-31T05:01:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.158+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8a806d190 con 0x7fc8a80cbf20 2024-01-31T05:01:55.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.159+0000 7fc8a4c00640 1 -- 172.21.15.97:0/647626576 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8a806d190 con 0x7fc8a80cbf20 2024-01-31T05:01:55.158 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88006aed0 msgr2=0x7fc88006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88006aed0 0x7fc88006d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc898035000 tx=0x7fc898008030 comp rx=0 tx=0).stop 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 msgr2=0x7fc8a8159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 0x7fc8a8159c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc89000c440 tx=0x7fc890001e70 comp rx=0 tx=0).stop 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 shutdown_connections 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8a815b470 0x7fc8a815a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88006aed0 0x7fc88006d390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8a80cbf20 0x7fc8a8159c40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 --2- 172.21.15.97:0/647626576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8a80abcf0 0x7fc8a8144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 >> 172.21.15.97:0/647626576 conn(0x7fc8a80b0a10 msgr2=0x7fc8a815bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.161+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 shutdown_connections 2024-01-31T05:01:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.162+0000 7fc8af800640 1 -- 172.21.15.97:0/647626576 wait complete. 2024-01-31T05:01:55.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:55.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.389+0000 7f405cc00640 1 Processor -- start 2024-01-31T05:01:55.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.389+0000 7f405cc00640 1 -- start start 2024-01-31T05:01:55.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f405cc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f40581663a0 0x7f4058168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f405cc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40580ceb40 con 0x7f405815d690 2024-01-31T05:01:55.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f405cc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405815db40 con 0x7f405806d580 2024-01-31T05:01:55.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f405cc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405815e7c0 con 0x7f40581663a0 2024-01-31T05:01:55.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f404d200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f405815d690 0x7f40580ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38678/0 (socket says 172.21.15.97:38678) 2024-01-31T05:01:55.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f404d200640 1 -- 172.21.15.97:0/78371988 learned_addr learned my addr 172.21.15.97:0/78371988 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.391+0000 7f4056800640 1 --2- 172.21.15.97:0/78371988 >> v2:172.21.15.188:3300/0 conn(0x7f40581663a0 0x7f4058168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301731300 0 0) 0x7f40580ceb40 con 0x7f405815d690 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4038003520 con 0x7f405815d690 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980243181 0 0) 0x7f4038003520 con 0x7f405815d690 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 >> v2:172.21.15.188:3300/0 conn(0x7f40581663a0 msgr2=0x7f4058168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 --2- 172.21.15.97:0/78371988 >> v2:172.21.15.188:3300/0 conn(0x7f40581663a0 0x7f4058168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 >> v1:172.21.15.188:6789/0 conn(0x7f405806d580 legacy=0x7f405806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405815de20 con 0x7f405815d690 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f4040001670 con 0x7f405815d690 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 >> v1:172.21.15.97:6789/0 conn(0x7f405815d690 legacy=0x7f40580ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f4038003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40380047c0 0x7f4038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4038005180 0x7f403800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405815e7c0 con 0x7f40380039c0 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403800c080 con 0x7f40380047c0 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403800c240 con 0x7f4038005180 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.392+0000 7f4055e00640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f4038003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f404d200640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40380047c0 0x7f4038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4056800640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4038005180 0x7f403800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4055e00640 1 -- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4038005180 msgr2=0x7f403800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4055e00640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4038005180 0x7f403800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4055e00640 1 -- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40380047c0 msgr2=0x7f4038004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4055e00640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40380047c0 0x7f4038004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4055e00640 1 -- 172.21.15.97:0/78371988 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038003520 con 0x7f40380039c0 2024-01-31T05:01:55.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.393+0000 7f4056800640 1 -- 172.21.15.97:0/78371988 reap_dead start 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.394+0000 7f4055e00640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f4038003de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4048001e70 tx=0x7f40380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919a9b41b425c654 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.394+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048003bc0 con 0x7f40380039c0 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.394+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048003d20 con 0x7f40380039c0 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.394+0000 7f4055400640 1 -- 172.21.15.97:0/78371988 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048005310 con 0x7f40380039c0 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 -- 172.21.15.97:0/78371988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 msgr2=0x7f4038003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f4038003de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4048001e70 tx=0x7f40380057f0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 -- 172.21.15.97:0/78371988 shutdown_connections 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 --2- 172.21.15.97:0/78371988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f4038003de0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 -- 172.21.15.97:0/78371988 >> 172.21.15.97:0/78371988 conn(0x7f40580b0a10 msgr2=0x7f40580b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 -- 172.21.15.97:0/78371988 shutdown_connections 2024-01-31T05:01:55.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.395+0000 7f405cc00640 1 -- 172.21.15.97:0/78371988 wait complete. 2024-01-31T05:01:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.397+0000 7f405cc00640 1 Processor -- start 2024-01-31T05:01:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.397+0000 7f405cc00640 1 -- start start 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f405cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f40580abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f405cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40580ad700 0x7f405815c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f405cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f405814d710 0x7f4058070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f405cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40580ad2e0 con 0x7f40380039c0 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f405cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4058071b70 con 0x7f40580ad700 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f405cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40580ac3c0 con 0x7f405814d710 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f40580abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f40580abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39866/0 (socket says 172.21.15.97:39866) 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 -- 172.21.15.97:0/3195885075 learned_addr learned my addr 172.21.15.97:0/3195885075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 -- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f405814d710 msgr2=0x7f4058070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f404d200640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40580ad700 0x7f405815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f405814d710 0x7f4058070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 -- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40580ad700 msgr2=0x7f405815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.398+0000 7f4055e00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40580ad700 0x7f405815c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f4055e00640 1 -- 172.21.15.97:0/3195885075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580ac660 con 0x7f40380039c0 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f4055e00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f40580abe80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4048001270 tx=0x7f40480014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404803e070 con 0x7f40380039c0 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048007e00 con 0x7f40380039c0 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048005830 con 0x7f40380039c0 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40580ac940 con 0x7f40380039c0 2024-01-31T05:01:55.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.399+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40580acf00 con 0x7f40380039c0 2024-01-31T05:01:55.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.400+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40580ad550 con 0x7f40380039c0 2024-01-31T05:01:55.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.401+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4048007040 con 0x7f40380039c0 2024-01-31T05:01:55.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.402+0000 7f404ea00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f403006af00 0x7f403006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.402+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f40480ab080 con 0x7f40380039c0 2024-01-31T05:01:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.402+0000 7f404d200640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f403006af00 0x7f403006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.403+0000 7f404d200640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f403006af00 0x7f403006d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4058070540 tx=0x7f4040000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:55.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.406+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4048075db0 con 0x7f40380039c0 2024-01-31T05:01:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.596+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4058067830 con 0x7f40380039c0 2024-01-31T05:01:55.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.597+0000 7f404ea00640 1 -- 172.21.15.97:0/3195885075 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f40480470a0 con 0x7f40380039c0 2024-01-31T05:01:55.597 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f403006af00 msgr2=0x7f403006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f403006af00 0x7f403006d3c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4058070540 tx=0x7f4040000ed0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 msgr2=0x7f40580abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f40580abe80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4048001270 tx=0x7f40480014f0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 shutdown_connections 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f405814d710 0x7f4058070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40580ad700 0x7f405815c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f403006af00 0x7f403006d3c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 --2- 172.21.15.97:0/3195885075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40380039c0 0x7f40580abe80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 >> 172.21.15.97:0/3195885075 conn(0x7f40580b0a10 msgr2=0x7f40580af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 shutdown_connections 2024-01-31T05:01:55.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.600+0000 7f405cc00640 1 -- 172.21.15.97:0/3195885075 wait complete. 2024-01-31T05:01:55.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:01:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.829+0000 7f6835400640 1 Processor -- start 2024-01-31T05:01:55.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.829+0000 7f6835400640 1 -- start start 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f6835400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f683006d580 0x7f683006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f6835400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6830161020 0x7f68301614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f6835400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300ceb40 con 0x7f683015d690 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f6835400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683015e7c0 con 0x7f6830161020 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f6835400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683006dea0 con 0x7f683006d580 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f682f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6830161020 0x7f68301614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f682f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6830161020 0x7f68301614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39878/0 (socket says 172.21.15.97:39878) 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f682f200640 1 -- 172.21.15.97:0/225440228 learned_addr learned my addr 172.21.15.97:0/225440228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.831+0000 7f682e800640 1 --2- 172.21.15.97:0/225440228 >> v2:172.21.15.188:3300/0 conn(0x7f683006d580 0x7f683006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.832+0000 7f682f200640 1 -- 172.21.15.97:0/225440228 >> v2:172.21.15.188:3300/0 conn(0x7f683006d580 msgr2=0x7f683006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.832+0000 7f682f200640 1 --2- 172.21.15.97:0/225440228 >> v2:172.21.15.188:3300/0 conn(0x7f683006d580 0x7f683006d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.832+0000 7f682f200640 1 -- 172.21.15.97:0/225440228 >> v1:172.21.15.188:6789/0 conn(0x7f683015d690 legacy=0x7f68300ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.832+0000 7f682f200640 1 -- 172.21.15.97:0/225440228 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683015db40 con 0x7f6830161020 2024-01-31T05:01:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.833+0000 7f682f200640 1 --2- 172.21.15.97:0/225440228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301614b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6830072e60 tx=0x7f682802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc08bc2dd2d5c3f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:55.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.833+0000 7f682d400640 1 -- 172.21.15.97:0/225440228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682802ca80 con 0x7f6830161020 2024-01-31T05:01:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.833+0000 7f682d400640 1 -- 172.21.15.97:0/225440228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682802cbe0 con 0x7f6830161020 2024-01-31T05:01:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.833+0000 7f682d400640 1 -- 172.21.15.97:0/225440228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280343f0 con 0x7f6830161020 2024-01-31T05:01:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 -- 172.21.15.97:0/225440228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 msgr2=0x7f68301614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 --2- 172.21.15.97:0/225440228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301614b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6830072e60 tx=0x7f682802bee0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 -- 172.21.15.97:0/225440228 shutdown_connections 2024-01-31T05:01:55.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 --2- 172.21.15.97:0/225440228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301614b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 --2- 172.21.15.97:0/225440228 >> v2:172.21.15.188:3300/0 conn(0x7f683006d580 0x7f683006d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 -- 172.21.15.97:0/225440228 >> 172.21.15.97:0/225440228 conn(0x7f68300b0a10 msgr2=0x7f68300b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:55.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 -- 172.21.15.97:0/225440228 shutdown_connections 2024-01-31T05:01:55.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.834+0000 7f6835400640 1 -- 172.21.15.97:0/225440228 wait complete. 2024-01-31T05:01:55.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.836+0000 7f6835400640 1 Processor -- start 2024-01-31T05:01:55.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.836+0000 7f6835400640 1 -- start start 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f6835400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f683006d580 0x7f68301f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f6835400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f683015d690 0x7f6830147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f6835400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f6835400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830148c80 con 0x7f6830161020 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f6835400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301f3fb0 con 0x7f683006d580 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f6835400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68301f3970 con 0x7f683015d690 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f682f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f682f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39890/0 (socket says 172.21.15.97:39890) 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f682f200640 1 -- 172.21.15.97:0/2916403985 learned_addr learned my addr 172.21.15.97:0/2916403985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f682e800640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f683006d580 0x7f68301f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.837+0000 7f682de00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f683015d690 0x7f6830147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682f200640 1 -- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f683015d690 msgr2=0x7f6830147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682f200640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f683015d690 0x7f6830147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682f200640 1 -- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f683006d580 msgr2=0x7f68301f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682f200640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f683006d580 0x7f68301f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682f200640 1 -- 172.21.15.97:0/2916403985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009b30 con 0x7f6830161020 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682de00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f683015d690 0x7f6830147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f682f200640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301451c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6828009f40 tx=0x7f6828004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828009510 con 0x7f6830161020 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682803e450 con 0x7f6830161020 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682803e7e0 con 0x7f6830161020 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f6835400640 1 -- 172.21.15.97:0/2916403985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301486c0 con 0x7f6830161020 2024-01-31T05:01:55.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.838+0000 7f6835400640 1 -- 172.21.15.97:0/2916403985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301f3bd0 con 0x7f6830161020 2024-01-31T05:01:55.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.840+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68301f40f0 con 0x7f6830161020 2024-01-31T05:01:55.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.840+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f682802cd50 con 0x7f6830161020 2024-01-31T05:01:55.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.841+0000 7f6826a00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f680806aed0 0x7f680806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.841+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f68280ac7a0 con 0x7f6830161020 2024-01-31T05:01:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.841+0000 7f682e800640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f680806aed0 0x7f680806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:55.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.842+0000 7f682e800640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f680806aed0 0x7f680806d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f682000b9f0 tx=0x7f6820008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:55.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:55.845+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f68301f40f0 con 0x7f6830161020 2024-01-31T05:01:56.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.052+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68301f40f0 con 0x7f6830161020 2024-01-31T05:01:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.052+0000 7f6826a00640 1 -- 172.21.15.97:0/2916403985 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68301f40f0 con 0x7f6830161020 2024-01-31T05:01:56.052 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f680806aed0 msgr2=0x7f680806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f680806aed0 0x7f680806d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f682000b9f0 tx=0x7f6820008040 comp rx=0 tx=0).stop 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 msgr2=0x7f68301451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301451c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6828009f40 tx=0x7f6828004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 shutdown_connections 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f680806aed0 0x7f680806d390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6830161020 0x7f68301451c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f683015d690 0x7f6830147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 --2- 172.21.15.97:0/2916403985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f683006d580 0x7f68301f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 >> 172.21.15.97:0/2916403985 conn(0x7f68300b0a10 msgr2=0x7f68300aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 shutdown_connections 2024-01-31T05:01:56.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.055+0000 7f681fe00640 1 -- 172.21.15.97:0/2916403985 wait complete. 2024-01-31T05:01:56.073 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:01:56.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:56.074 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:01:56.166 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:01:56.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:01:56.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:01:56.166 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:01:56.214 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:01:56.214 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:56.271 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:01:56.271 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:56.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:56.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.539+0000 7f6ea4000640 1 Processor -- start 2024-01-31T05:01:56.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.540+0000 7f6ea4000640 1 -- start start 2024-01-31T05:01:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.541+0000 7f6ea4000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.541+0000 7f6ea4000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e9c06db70 0x7f6e9c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.541+0000 7f6ea4000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c06d1a0 con 0x7f6e9c0cb810 2024-01-31T05:01:56.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.541+0000 7f6ea4000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0cc960 con 0x7f6e9c06db70 2024-01-31T05:01:56.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.541+0000 7f6ea4000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c06c600 con 0x7f6e9c15b400 2024-01-31T05:01:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.542+0000 7f6ea0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.542+0000 7f6ea1600640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f6e9c0cb810 0x7f6e9c0cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:47220/0 (socket says 172.21.15.97:47220) 2024-01-31T05:01:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.542+0000 7f6ea1600640 1 -- 172.21.15.97:0/2934798221 learned_addr learned my addr 172.21.15.97:0/2934798221 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:56.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.542+0000 7f6ea2000640 1 --2- 172.21.15.97:0/2934798221 >> v2:172.21.15.188:3300/0 conn(0x7f6e9c06db70 0x7f6e9c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.543+0000 7f6ea0c00640 1 -- 172.21.15.97:0/2934798221 >> v2:172.21.15.188:3300/0 conn(0x7f6e9c06db70 msgr2=0x7f6e9c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.543+0000 7f6ea0c00640 1 --2- 172.21.15.97:0/2934798221 >> v2:172.21.15.188:3300/0 conn(0x7f6e9c06db70 0x7f6e9c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.543+0000 7f6ea0c00640 1 -- 172.21.15.97:0/2934798221 >> v1:172.21.15.188:6789/0 conn(0x7f6e9c0cb810 legacy=0x7f6e9c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.543+0000 7f6ea0c00640 1 -- 172.21.15.97:0/2934798221 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c06c870 con 0x7f6e9c15b400 2024-01-31T05:01:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.544+0000 7f6ea0c00640 1 --2- 172.21.15.97:0/2934798221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c06d630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e840097a0 tx=0x7f6e8402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9567892b2c5fce server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6e9be00640 1 -- 172.21.15.97:0/2934798221 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84031830 con 0x7f6e9c15b400 2024-01-31T05:01:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6e9be00640 1 -- 172.21.15.97:0/2934798221 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e84031990 con 0x7f6e9c15b400 2024-01-31T05:01:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6e9be00640 1 -- 172.21.15.97:0/2934798221 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84031d20 con 0x7f6e9c15b400 2024-01-31T05:01:56.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6ea4000640 1 -- 172.21.15.97:0/2934798221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 msgr2=0x7f6e9c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6ea4000640 1 --2- 172.21.15.97:0/2934798221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c06d630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e840097a0 tx=0x7f6e8402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6ea4000640 1 -- 172.21.15.97:0/2934798221 shutdown_connections 2024-01-31T05:01:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6ea4000640 1 --2- 172.21.15.97:0/2934798221 >> v2:172.21.15.188:3300/0 conn(0x7f6e9c06db70 0x7f6e9c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6ea4000640 1 --2- 172.21.15.97:0/2934798221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c06d630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.545+0000 7f6ea4000640 1 -- 172.21.15.97:0/2934798221 >> 172.21.15.97:0/2934798221 conn(0x7f6e9c0b0a10 msgr2=0x7f6e9c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:56.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.546+0000 7f6ea4000640 1 -- 172.21.15.97:0/2934798221 shutdown_connections 2024-01-31T05:01:56.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.546+0000 7f6ea4000640 1 -- 172.21.15.97:0/2934798221 wait complete. 2024-01-31T05:01:56.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.547+0000 7f6ea4000640 1 Processor -- start 2024-01-31T05:01:56.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.547+0000 7f6ea4000640 1 -- start start 2024-01-31T05:01:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.548+0000 7f6ea4000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 0x7f6e9c14ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.548+0000 7f6ea4000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e9c0cb810 0x7f6e9c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.548+0000 7f6ea4000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.548+0000 7f6ea4000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c14f170 con 0x7f6e9c06db70 2024-01-31T05:01:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.548+0000 7f6ea4000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0b1e10 con 0x7f6e9c0cb810 2024-01-31T05:01:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.548+0000 7f6ea4000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c14f2e0 con 0x7f6e9c15b400 2024-01-31T05:01:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 0x7f6e9c14ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 0x7f6e9c14ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39900/0 (socket says 172.21.15.97:39900) 2024-01-31T05:01:56.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 -- 172.21.15.97:0/576024680 learned_addr learned my addr 172.21.15.97:0/576024680 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea2000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea0c00640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e9c0cb810 0x7f6e9c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 -- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 msgr2=0x7f6e9c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c0cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 -- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e9c0cb810 msgr2=0x7f6e9c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e9c0cb810 0x7f6e9c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 -- 172.21.15.97:0/576024680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e84009400 con 0x7f6e9c06db70 2024-01-31T05:01:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.549+0000 7f6ea1600640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 0x7f6e9c14ec30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c4a0 tx=0x7f6e8c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.550+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c005ce0 con 0x7f6e9c06db70 2024-01-31T05:01:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.550+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c005e40 con 0x7f6e9c06db70 2024-01-31T05:01:56.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.550+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c007270 con 0x7f6e9c06db70 2024-01-31T05:01:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.550+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c14f5a0 con 0x7f6e9c06db70 2024-01-31T05:01:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.550+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c14fb60 con 0x7f6e9c06db70 2024-01-31T05:01:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.551+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c0677c0 con 0x7f6e9c06db70 2024-01-31T05:01:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.552+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6e8c007490 con 0x7f6e9c06db70 2024-01-31T05:01:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.553+0000 7f6e99600640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e7406aed0 0x7f6e7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.553+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6e8c088ef0 con 0x7f6e9c06db70 2024-01-31T05:01:56.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.554+0000 7f6ea0c00640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e7406aed0 0x7f6e7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.554+0000 7f6ea0c00640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e7406aed0 0x7f6e7406d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e84036000 tx=0x7f6e8403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:56.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.558+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6e8c053a80 con 0x7f6e9c06db70 2024-01-31T05:01:56.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.753+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e9c156df0 con 0x7f6e9c06db70 2024-01-31T05:01:56.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.755+0000 7f6e99600640 1 -- 172.21.15.97:0/576024680 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f6e8c017020 con 0x7f6e9c06db70 2024-01-31T05:01:56.755 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:56.755 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e7406aed0 msgr2=0x7f6e7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e7406aed0 0x7f6e7406d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e84036000 tx=0x7f6e8403c040 comp rx=0 tx=0).stop 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 msgr2=0x7f6e9c14ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 0x7f6e9c14ec30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c4a0 tx=0x7f6e8c00c960 comp rx=0 tx=0).stop 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 shutdown_connections 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e9c15b400 0x7f6e9c0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e9c0cb810 0x7f6e9c0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e7406aed0 0x7f6e7406d390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 --2- 172.21.15.97:0/576024680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e9c06db70 0x7f6e9c14ec30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 >> 172.21.15.97:0/576024680 conn(0x7f6e9c0b0a10 msgr2=0x7f6e9c0ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:56.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 shutdown_connections 2024-01-31T05:01:56.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.758+0000 7f6ea4000640 1 -- 172.21.15.97:0/576024680 wait complete. 2024-01-31T05:01:56.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:56.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:56.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.997+0000 7ff65ea00640 1 Processor -- start 2024-01-31T05:01:56.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.997+0000 7ff65ea00640 1 -- start start 2024-01-31T05:01:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.998+0000 7ff65ea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6581599a0 0x7ff6580ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff65ea00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff658159d80 con 0x7ff6580ccd00 2024-01-31T05:01:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff65ea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6580cc5e0 con 0x7ff6580adea0 2024-01-31T05:01:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff65ea00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65815e210 con 0x7ff6581599a0 2024-01-31T05:01:56.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff657e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6581599a0 0x7ff6580ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff65ca00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6580ccd00 0x7ff6580cd150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38684/0 (socket says 172.21.15.97:38684) 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff657e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6581599a0 0x7ff6580ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39916/0 (socket says 172.21.15.97:39916) 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff65ca00640 1 -- 172.21.15.97:0/275614590 learned_addr learned my addr 172.21.15.97:0/275614590 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff657e00640 1 -- 172.21.15.97:0/275614590 >> v1:172.21.15.188:6789/0 conn(0x7ff6580adea0 legacy=0x7ff6580b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff657e00640 1 -- 172.21.15.97:0/275614590 >> v1:172.21.15.97:6789/0 conn(0x7ff6580ccd00 legacy=0x7ff6580cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff657e00640 1 -- 172.21.15.97:0/275614590 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580cc880 con 0x7ff6581599a0 2024-01-31T05:01:56.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:56.999+0000 7ff657400640 1 -- 172.21.15.97:0/275614590 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154787787 0 0) 0x7ff658159d80 con 0x7ff6580ccd00 2024-01-31T05:01:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.000+0000 7ff657e00640 1 --2- 172.21.15.97:0/275614590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6581599a0 0x7ff6580ae3f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff64800a930 tx=0x7ff64802ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b120d6a70292aa server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff657400640 1 -- 172.21.15.97:0/275614590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64802daa0 con 0x7ff6581599a0 2024-01-31T05:01:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff657400640 1 -- 172.21.15.97:0/275614590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64802dc00 con 0x7ff6581599a0 2024-01-31T05:01:57.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff657400640 1 -- 172.21.15.97:0/275614590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648035440 con 0x7ff6581599a0 2024-01-31T05:01:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff65ea00640 1 -- 172.21.15.97:0/275614590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6581599a0 msgr2=0x7ff6580ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff65ea00640 1 --2- 172.21.15.97:0/275614590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6581599a0 0x7ff6580ae3f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff64800a930 tx=0x7ff64802ccf0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff65ea00640 1 -- 172.21.15.97:0/275614590 shutdown_connections 2024-01-31T05:01:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff65ea00640 1 --2- 172.21.15.97:0/275614590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6581599a0 0x7ff6580ae3f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.001+0000 7ff65ea00640 1 -- 172.21.15.97:0/275614590 >> 172.21.15.97:0/275614590 conn(0x7ff6580b0a10 msgr2=0x7ff6580b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:57.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.002+0000 7ff65ea00640 1 -- 172.21.15.97:0/275614590 shutdown_connections 2024-01-31T05:01:57.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.002+0000 7ff65ea00640 1 -- 172.21.15.97:0/275614590 wait complete. 2024-01-31T05:01:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.003+0000 7ff65ea00640 1 Processor -- start 2024-01-31T05:01:57.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.004+0000 7ff65ea00640 1 -- start start 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.004+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6580adea0 0x7ff658071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.004+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 0x7ff6580ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.004+0000 7ff65ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6581599a0 0x7ff6580aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff65ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580b00b0 con 0x7ff6580ccd00 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff65ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580713f0 con 0x7ff6581599a0 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff65ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580707e0 con 0x7ff6580adea0 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 0x7ff6580ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 0x7ff6580ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39922/0 (socket says 172.21.15.97:39922) 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 -- 172.21.15.97:0/2696665648 learned_addr learned my addr 172.21.15.97:0/2696665648 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 -- 172.21.15.97:0/2696665648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6580adea0 msgr2=0x7ff658071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6580adea0 0x7ff658071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 -- 172.21.15.97:0/2696665648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6581599a0 msgr2=0x7ff6580aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6581599a0 0x7ff6580aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 -- 172.21.15.97:0/2696665648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64800a540 con 0x7ff6580ccd00 2024-01-31T05:01:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.005+0000 7ff64f200640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 0x7ff6580ae960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c430 tx=0x7ff64000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.006+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640011790 con 0x7ff6580ccd00 2024-01-31T05:01:57.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.006+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6400118f0 con 0x7ff6580ccd00 2024-01-31T05:01:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.006+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640011ce0 con 0x7ff6580ccd00 2024-01-31T05:01:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.006+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff658070ac0 con 0x7ff6580ccd00 2024-01-31T05:01:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.006+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff658071080 con 0x7ff6580ccd00 2024-01-31T05:01:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.007+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6580b23d0 con 0x7ff6580ccd00 2024-01-31T05:01:57.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.008+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff64001c460 con 0x7ff6580ccd00 2024-01-31T05:01:57.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.009+0000 7ff654c00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff63006af00 0x7ff63006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.009+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff640089880 con 0x7ff6580ccd00 2024-01-31T05:01:57.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.010+0000 7ff657e00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff63006af00 0x7ff63006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.010+0000 7ff657e00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff63006af00 0x7ff63006d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff648037000 tx=0x7ff64802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.014+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff640054470 con 0x7ff6580ccd00 2024-01-31T05:01:57.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.207+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff658071860 con 0x7ff6580ccd00 2024-01-31T05:01:57.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.208+0000 7ff654c00640 1 -- 172.21.15.97:0/2696665648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff640087e60 con 0x7ff6580ccd00 2024-01-31T05:01:57.208 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff63006af00 msgr2=0x7ff63006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff63006af00 0x7ff63006d3c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff648037000 tx=0x7ff64802f000 comp rx=0 tx=0).stop 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 msgr2=0x7ff6580ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 0x7ff6580ae960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c430 tx=0x7ff64000c8f0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 shutdown_connections 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6581599a0 0x7ff6580aeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff63006af00 0x7ff63006d3c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6580ccd00 0x7ff6580ae960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 --2- 172.21.15.97:0/2696665648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6580adea0 0x7ff658071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 >> 172.21.15.97:0/2696665648 conn(0x7ff6580b0a10 msgr2=0x7ff65814f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 shutdown_connections 2024-01-31T05:01:57.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.211+0000 7ff65ea00640 1 -- 172.21.15.97:0/2696665648 wait complete. 2024-01-31T05:01:57.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:01:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.443+0000 7f9fa0600640 1 Processor -- start 2024-01-31T05:01:57.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.443+0000 7f9fa0600640 1 -- start start 2024-01-31T05:01:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9fa0600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f980b2570 0x7f9f980cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9fa0600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980d1b90 0x7f9f980cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9fa0600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f980cde30 0x7f9f9815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9fa0600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980ce2c0 con 0x7f9f980b2570 2024-01-31T05:01:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9fa0600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9806ce00 con 0x7f9f980cde30 2024-01-31T05:01:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9fa0600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980cb760 con 0x7f9f980d1b90 2024-01-31T05:01:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9f9dc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f980b2570 0x7f9f980cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9f9dc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f980b2570 0x7f9f980cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39932/0 (socket says 172.21.15.97:39932) 2024-01-31T05:01:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9f9dc00640 1 -- 172.21.15.97:0/4056016013 learned_addr learned my addr 172.21.15.97:0/4056016013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9f9d200640 1 --2- 172.21.15.97:0/4056016013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980d1b90 0x7f9f980cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.445+0000 7f9f9dc00640 1 -- 172.21.15.97:0/4056016013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980d1b90 msgr2=0x7f9f980cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.446+0000 7f9f9dc00640 1 --2- 172.21.15.97:0/4056016013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980d1b90 0x7f9f980cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.446+0000 7f9f9e600640 1 --2- 172.21.15.97:0/4056016013 >> v2:172.21.15.188:3300/0 conn(0x7f9f980cde30 0x7f9f9815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.446+0000 7f9f9dc00640 1 -- 172.21.15.97:0/4056016013 >> v2:172.21.15.188:3300/0 conn(0x7f9f980cde30 msgr2=0x7f9f9815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.446+0000 7f9f9dc00640 1 --2- 172.21.15.97:0/4056016013 >> v2:172.21.15.188:3300/0 conn(0x7f9f980cde30 0x7f9f9815cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.446+0000 7f9f9dc00640 1 -- 172.21.15.97:0/4056016013 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980b2950 con 0x7f9f980b2570 2024-01-31T05:01:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.446+0000 7f9f9e600640 1 --2- 172.21.15.97:0/4056016013 >> v2:172.21.15.188:3300/0 conn(0x7f9f980cde30 0x7f9f9815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.447+0000 7f9f9dc00640 1 --2- 172.21.15.97:0/4056016013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f980cf3e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f980cee50 tx=0x7f9f8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30a9c0d8f234ad server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.447+0000 7f9f97e00640 1 -- 172.21.15.97:0/4056016013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8802ca80 con 0x7f9f980b2570 2024-01-31T05:01:57.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.447+0000 7f9f97e00640 1 -- 172.21.15.97:0/4056016013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8802cbe0 con 0x7f9f980b2570 2024-01-31T05:01:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.447+0000 7f9f97e00640 1 -- 172.21.15.97:0/4056016013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880343f0 con 0x7f9f980b2570 2024-01-31T05:01:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 -- 172.21.15.97:0/4056016013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 msgr2=0x7f9f980cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 --2- 172.21.15.97:0/4056016013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f980cf3e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f980cee50 tx=0x7f9f8802bee0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 -- 172.21.15.97:0/4056016013 shutdown_connections 2024-01-31T05:01:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 --2- 172.21.15.97:0/4056016013 >> v2:172.21.15.188:3300/0 conn(0x7f9f980cde30 0x7f9f9815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 --2- 172.21.15.97:0/4056016013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980d1b90 0x7f9f980cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 --2- 172.21.15.97:0/4056016013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f980cf3e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 -- 172.21.15.97:0/4056016013 >> 172.21.15.97:0/4056016013 conn(0x7f9f980d1070 msgr2=0x7f9f980d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 -- 172.21.15.97:0/4056016013 shutdown_connections 2024-01-31T05:01:57.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.448+0000 7f9fa0600640 1 -- 172.21.15.97:0/4056016013 wait complete. 2024-01-31T05:01:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.449+0000 7f9fa0600640 1 Processor -- start 2024-01-31T05:01:57.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.450+0000 7f9fa0600640 1 -- start start 2024-01-31T05:01:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9fa0600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f9809adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9fa0600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980cde30 0x7f9f98150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9fa0600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f980d1b90 0x7f9f9814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9fa0600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9809b2f0 con 0x7f9f980b2570 2024-01-31T05:01:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9fa0600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980c5280 con 0x7f9f980d1b90 2024-01-31T05:01:57.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9fa0600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9814e9c0 con 0x7f9f980cde30 2024-01-31T05:01:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f9809adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f9809adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39946/0 (socket says 172.21.15.97:39946) 2024-01-31T05:01:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 -- 172.21.15.97:0/2141039586 learned_addr learned my addr 172.21.15.97:0/2141039586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9d200640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980cde30 0x7f9f98150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 -- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980cde30 msgr2=0x7f9f98150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980cde30 0x7f9f98150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 -- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f980d1b90 msgr2=0x7f9f9814dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f980d1b90 0x7f9f9814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.451+0000 7f9f9dc00640 1 -- 172.21.15.97:0/2141039586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88009b30 con 0x7f9f980b2570 2024-01-31T05:01:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.452+0000 7f9f9dc00640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f9809adb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009e50 tx=0x7f9f88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.452+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8803e450 con 0x7f9f980b2570 2024-01-31T05:01:57.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.452+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8803e5f0 con 0x7f9f980b2570 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.452+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8803e9a0 con 0x7f9f980b2570 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.452+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9814eca0 con 0x7f9f980b2570 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.452+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9815dba0 con 0x7f9f980b2570 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.453+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9814f1c0 con 0x7f9f980b2570 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.454+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9f8802cd50 con 0x7f9f980b2570 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.455+0000 7f9f95600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f7006aed0 0x7f9f7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.455+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f880ac7a0 con 0x7f9f980b2570 2024-01-31T05:01:57.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.455+0000 7f9f9d200640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f7006aed0 0x7f9f7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.456+0000 7f9f9d200640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f7006aed0 0x7f9f7006d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.460+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9f9814f1c0 con 0x7f9f980b2570 2024-01-31T05:01:57.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.654+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f9814f1c0 con 0x7f9f980b2570 2024-01-31T05:01:57.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.656+0000 7f9f95600640 1 -- 172.21.15.97:0/2141039586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f9814f1c0 con 0x7f9f980b2570 2024-01-31T05:01:57.655 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f7006aed0 msgr2=0x7f9f7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f7006aed0 0x7f9f7006d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 msgr2=0x7f9f9809adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f9809adb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009e50 tx=0x7f9f88004a40 comp rx=0 tx=0).stop 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 shutdown_connections 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f980d1b90 0x7f9f9814dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f980cde30 0x7f9f98150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f7006aed0 0x7f9f7006d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 --2- 172.21.15.97:0/2141039586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f980b2570 0x7f9f9809adb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.658+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 >> 172.21.15.97:0/2141039586 conn(0x7f9f980d1070 msgr2=0x7f9f980cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.659+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 shutdown_connections 2024-01-31T05:01:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.659+0000 7f9fa0600640 1 -- 172.21.15.97:0/2141039586 wait complete. 2024-01-31T05:01:57.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:01:57.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.890+0000 7fe496e00640 1 Processor -- start 2024-01-31T05:01:57.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.890+0000 7fe496e00640 1 -- start start 2024-01-31T05:01:57.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe496e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe49015b470 0x7fe4900722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe496e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4900abcf0 0x7fe4900ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe496e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49006cd20 con 0x7fe4900cb810 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe496e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900ce970 con 0x7fe49015b470 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe496e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49015b850 con 0x7fe4900abcf0 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe48fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe49015b470 0x7fe4900722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe48fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe49015b470 0x7fe4900722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39948/0 (socket says 172.21.15.97:39948) 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe48fe00640 1 -- 172.21.15.97:0/864728730 learned_addr learned my addr 172.21.15.97:0/864728730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:57.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe494e00640 1 --1- 172.21.15.97:0/864728730 >> v1:172.21.15.97:6789/0 conn(0x7fe4900cb810 0x7fe4900ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38696/0 (socket says 172.21.15.97:38696) 2024-01-31T05:01:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.892+0000 7fe48f400640 1 --2- 172.21.15.97:0/864728730 >> v2:172.21.15.188:3300/0 conn(0x7fe4900abcf0 0x7fe4900ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.893+0000 7fe48fe00640 1 -- 172.21.15.97:0/864728730 >> v2:172.21.15.188:3300/0 conn(0x7fe4900abcf0 msgr2=0x7fe4900ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.893+0000 7fe48fe00640 1 --2- 172.21.15.97:0/864728730 >> v2:172.21.15.188:3300/0 conn(0x7fe4900abcf0 0x7fe4900ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.893+0000 7fe48fe00640 1 -- 172.21.15.97:0/864728730 >> v1:172.21.15.97:6789/0 conn(0x7fe4900cb810 legacy=0x7fe4900ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.893+0000 7fe48fe00640 1 -- 172.21.15.97:0/864728730 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900ceb60 con 0x7fe49015b470 2024-01-31T05:01:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.894+0000 7fe48fe00640 1 --2- 172.21.15.97:0/864728730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe4900722c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe490072d90 tx=0x7fe48002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbd1e4e752f4a19 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.894+0000 7fe48ea00640 1 -- 172.21.15.97:0/864728730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480030840 con 0x7fe49015b470 2024-01-31T05:01:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.894+0000 7fe48ea00640 1 -- 172.21.15.97:0/864728730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4800309a0 con 0x7fe49015b470 2024-01-31T05:01:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.894+0000 7fe48ea00640 1 -- 172.21.15.97:0/864728730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480030d30 con 0x7fe49015b470 2024-01-31T05:01:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 -- 172.21.15.97:0/864728730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 msgr2=0x7fe4900722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 --2- 172.21.15.97:0/864728730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe4900722c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe490072d90 tx=0x7fe48002b980 comp rx=0 tx=0).stop 2024-01-31T05:01:57.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 -- 172.21.15.97:0/864728730 shutdown_connections 2024-01-31T05:01:57.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 --2- 172.21.15.97:0/864728730 >> v2:172.21.15.188:3300/0 conn(0x7fe4900abcf0 0x7fe4900ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 --2- 172.21.15.97:0/864728730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe4900722c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 -- 172.21.15.97:0/864728730 >> 172.21.15.97:0/864728730 conn(0x7fe4900b0a10 msgr2=0x7fe4900b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:57.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 -- 172.21.15.97:0/864728730 shutdown_connections 2024-01-31T05:01:57.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.895+0000 7fe496e00640 1 -- 172.21.15.97:0/864728730 wait complete. 2024-01-31T05:01:57.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.896+0000 7fe496e00640 1 Processor -- start 2024-01-31T05:01:57.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 -- start start 2024-01-31T05:01:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4900abcf0 0x7fe490151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4900cb810 0x7fe4901525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe490154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4901520f0 con 0x7fe49015b470 2024-01-31T05:01:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4901551b0 con 0x7fe4900cb810 2024-01-31T05:01:57.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.897+0000 7fe496e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe490155490 con 0x7fe4900abcf0 2024-01-31T05:01:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe490154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe490154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39952/0 (socket says 172.21.15.97:39952) 2024-01-31T05:01:57.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 -- 172.21.15.97:0/3278822612 learned_addr learned my addr 172.21.15.97:0/3278822612 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe48fe00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4900abcf0 0x7fe490151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 -- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4900abcf0 msgr2=0x7fe490151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe48f400640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4900cb810 0x7fe4901525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4900abcf0 0x7fe490151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 -- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4900cb810 msgr2=0x7fe4901525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:57.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4900cb810 0x7fe4901525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe494e00640 1 -- 172.21.15.97:0/3278822612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009470 con 0x7fe49015b470 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.898+0000 7fe48fe00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4900abcf0 0x7fe490151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.899+0000 7fe494e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe490154770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c780 tx=0x7fe48800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.899+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48800ce90 con 0x7fe49015b470 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.899+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48800a640 con 0x7fe49015b470 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.899+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48800a9f0 con 0x7fe49015b470 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.899+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe490155770 con 0x7fe49015b470 2024-01-31T05:01:57.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.899+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe490099d40 con 0x7fe49015b470 2024-01-31T05:01:57.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.900+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe490099f50 con 0x7fe49015b470 2024-01-31T05:01:57.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.901+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe48800ab50 con 0x7fe49015b470 2024-01-31T05:01:57.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.902+0000 7fe487e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe46806aed0 0x7fe46806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:57.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.902+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe488089110 con 0x7fe49015b470 2024-01-31T05:01:57.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.903+0000 7fe48fe00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe46806aed0 0x7fe46806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:57.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.903+0000 7fe48fe00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe46806aed0 0x7fe46806d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe480036000 tx=0x7fe48003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:57.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:57.907+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe488053ca0 con 0x7fe49015b470 2024-01-31T05:01:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.102+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe490067830 con 0x7fe49015b470 2024-01-31T05:01:58.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.103+0000 7fe487e00640 1 -- 172.21.15.97:0/3278822612 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4880572c0 con 0x7fe49015b470 2024-01-31T05:01:58.103 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:01:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.105+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe46806aed0 msgr2=0x7fe46806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.105+0000 7fe496e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe46806aed0 0x7fe46806d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe480036000 tx=0x7fe48003b040 comp rx=0 tx=0).stop 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 msgr2=0x7fe490154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe490154770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe48800c780 tx=0x7fe48800cc40 comp rx=0 tx=0).stop 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 shutdown_connections 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe46806aed0 0x7fe46806d390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe49015b470 0x7fe490154770 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4900cb810 0x7fe4901525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 --2- 172.21.15.97:0/3278822612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4900abcf0 0x7fe490151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 >> 172.21.15.97:0/3278822612 conn(0x7fe4900b0a10 msgr2=0x7fe4901585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 shutdown_connections 2024-01-31T05:01:58.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.106+0000 7fe496e00640 1 -- 172.21.15.97:0/3278822612 wait complete. 2024-01-31T05:01:58.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:01:58.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.356+0000 7f212ac00640 1 Processor -- start 2024-01-31T05:01:58.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.356+0000 7f212ac00640 1 -- start start 2024-01-31T05:01:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.358+0000 7f212ac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21240d0040 0x7f21240b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.358+0000 7f212ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 0x7f21240b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.358+0000 7f212ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240d2190 con 0x7f212415d450 2024-01-31T05:01:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.358+0000 7f212ac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240b3060 con 0x7f21240d0040 2024-01-31T05:01:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.358+0000 7f212ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240b2a30 con 0x7f21240d39f0 2024-01-31T05:01:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.359+0000 7f212a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 0x7f21240b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.359+0000 7f212a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 0x7f21240b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:56810/0 (socket says 172.21.15.97:56810) 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.359+0000 7f212a200640 1 -- 172.21.15.97:0/4190998575 learned_addr learned my addr 172.21.15.97:0/4190998575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.359+0000 7f2128e00640 1 --2- 172.21.15.97:0/4190998575 >> v2:172.21.15.188:3300/0 conn(0x7f21240d0040 0x7f21240b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.360+0000 7f212a200640 1 -- 172.21.15.97:0/4190998575 >> v2:172.21.15.188:3300/0 conn(0x7f21240d0040 msgr2=0x7f21240b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.360+0000 7f212a200640 1 --2- 172.21.15.97:0/4190998575 >> v2:172.21.15.188:3300/0 conn(0x7f21240d0040 0x7f21240b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.360+0000 7f212a200640 1 -- 172.21.15.97:0/4190998575 >> v1:172.21.15.188:6789/0 conn(0x7f212415d450 legacy=0x7f21240cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.360+0000 7f212a200640 1 -- 172.21.15.97:0/4190998575 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240d1c30 con 0x7f21240d39f0 2024-01-31T05:01:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.360+0000 7f2123e00640 1 -- 172.21.15.97:0/4190998575 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174729391 0 0) 0x7f21240d2190 con 0x7f212415d450 2024-01-31T05:01:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.361+0000 7f212a200640 1 --2- 172.21.15.97:0/4190998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 0x7f21240b39c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f211c009870 tx=0x7f211c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78cd038fd4177bc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:58.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.361+0000 7f2123e00640 1 -- 172.21.15.97:0/4190998575 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c030840 con 0x7f21240d39f0 2024-01-31T05:01:58.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.361+0000 7f2123e00640 1 -- 172.21.15.97:0/4190998575 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c0309a0 con 0x7f21240d39f0 2024-01-31T05:01:58.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.361+0000 7f2123e00640 1 -- 172.21.15.97:0/4190998575 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c030d30 con 0x7f21240d39f0 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 -- 172.21.15.97:0/4190998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 msgr2=0x7f21240b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 --2- 172.21.15.97:0/4190998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 0x7f21240b39c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f211c009870 tx=0x7f211c02b980 comp rx=0 tx=0).stop 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 -- 172.21.15.97:0/4190998575 shutdown_connections 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 --2- 172.21.15.97:0/4190998575 >> v2:172.21.15.188:3300/0 conn(0x7f21240d0040 0x7f21240b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 --2- 172.21.15.97:0/4190998575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d39f0 0x7f21240b39c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 -- 172.21.15.97:0/4190998575 >> 172.21.15.97:0/4190998575 conn(0x7f212409c400 msgr2=0x7f21240d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 -- 172.21.15.97:0/4190998575 shutdown_connections 2024-01-31T05:01:58.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.362+0000 7f212ac00640 1 -- 172.21.15.97:0/4190998575 wait complete. 2024-01-31T05:01:58.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.365+0000 7f212ac00640 1 Processor -- start 2024-01-31T05:01:58.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.365+0000 7f212ac00640 1 -- start start 2024-01-31T05:01:58.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d0040 0x7f21240c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240d39f0 0x7f21240ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212ac00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 0x7f21240ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212ac00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21241569c0 con 0x7f212415d450 2024-01-31T05:01:58.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240c7c50 con 0x7f21240d39f0 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240c7f30 con 0x7f21240d0040 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 0x7f21240ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 0x7f21240ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39968/0 (socket says 172.21.15.97:39968) 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f212a200640 1 -- 172.21.15.97:0/2741933091 learned_addr learned my addr 172.21.15.97:0/2741933091 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f2128e00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240d39f0 0x7f21240ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.367+0000 7f2129800640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d0040 0x7f21240c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212a200640 1 -- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d0040 msgr2=0x7f21240c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212a200640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d0040 0x7f21240c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212a200640 1 -- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240d39f0 msgr2=0x7f21240ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212a200640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240d39f0 0x7f21240ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212a200640 1 -- 172.21.15.97:0/2741933091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c0094d0 con 0x7f212415d450 2024-01-31T05:01:58.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212a200640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 0x7f21240ca6e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f211c009870 tx=0x7f211c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c039c90 con 0x7f212415d450 2024-01-31T05:01:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c039e10 con 0x7f212415d450 2024-01-31T05:01:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c043640 con 0x7f212415d450 2024-01-31T05:01:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.368+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21240c8210 con 0x7f212415d450 2024-01-31T05:01:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.369+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21240c87d0 con 0x7f212415d450 2024-01-31T05:01:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.370+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21240676b0 con 0x7f212415d450 2024-01-31T05:01:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.370+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f211c0437a0 con 0x7f212415d450 2024-01-31T05:01:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.371+0000 7f2121600640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f20fc06aed0 0x7f20fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.371+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f211c03e070 con 0x7f212415d450 2024-01-31T05:01:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.372+0000 7f2129800640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f20fc06aed0 0x7f20fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.372+0000 7f2129800640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f20fc06aed0 0x7f20fc06d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2114007fd0 tx=0x7f2114009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.376+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f211c077550 con 0x7f212415d450 2024-01-31T05:01:58.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.579+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f212415cc00 con 0x7f212415d450 2024-01-31T05:01:58.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.580+0000 7f2121600640 1 -- 172.21.15.97:0/2741933091 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f211c07ab70 con 0x7f212415d450 2024-01-31T05:01:58.580 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:01:58.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.583+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f20fc06aed0 msgr2=0x7f20fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.583+0000 7f212ac00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f20fc06aed0 0x7f20fc06d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2114007fd0 tx=0x7f2114009040 comp rx=0 tx=0).stop 2024-01-31T05:01:58.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.583+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 msgr2=0x7f21240ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.583+0000 7f212ac00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 0x7f21240ca6e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f211c009870 tx=0x7f211c004730 comp rx=0 tx=0).stop 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 shutdown_connections 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f20fc06aed0 0x7f20fc06d390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f212415d450 0x7f21240ca6e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21240d39f0 0x7f21240ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 --2- 172.21.15.97:0/2741933091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21240d0040 0x7f21240c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 >> 172.21.15.97:0/2741933091 conn(0x7f212409c400 msgr2=0x7f21240cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 shutdown_connections 2024-01-31T05:01:58.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.584+0000 7f212ac00640 1 -- 172.21.15.97:0/2741933091 wait complete. 2024-01-31T05:01:58.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:01:58.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.806+0000 7f94f4000640 1 Processor -- start 2024-01-31T05:01:58.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.807+0000 7f94f4000640 1 -- start start 2024-01-31T05:01:58.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.808+0000 7f94f4000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f94ec1599a0 0x7f94ec0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.808+0000 7f94f4000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 0x7f94ec0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.808+0000 7f94f4000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0cd6d0 con 0x7f94ec0adea0 2024-01-31T05:01:58.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.808+0000 7f94f4000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec159d80 con 0x7f94ec1599a0 2024-01-31T05:01:58.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.808+0000 7f94f4000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0cc5e0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.809+0000 7f94f2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 0x7f94ec0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.809+0000 7f94f2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 0x7f94ec0cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:56824/0 (socket says 172.21.15.97:56824) 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.809+0000 7f94f2000640 1 -- 172.21.15.97:0/4235066359 learned_addr learned my addr 172.21.15.97:0/4235066359 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.809+0000 7f94f1600640 1 --2- 172.21.15.97:0/4235066359 >> v2:172.21.15.188:3300/0 conn(0x7f94ec1599a0 0x7f94ec0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.810+0000 7f94f2000640 1 -- 172.21.15.97:0/4235066359 >> v2:172.21.15.188:3300/0 conn(0x7f94ec1599a0 msgr2=0x7f94ec0ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.810+0000 7f94f2000640 1 --2- 172.21.15.97:0/4235066359 >> v2:172.21.15.188:3300/0 conn(0x7f94ec1599a0 0x7f94ec0ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.810+0000 7f94f2000640 1 -- 172.21.15.97:0/4235066359 >> v1:172.21.15.188:6789/0 conn(0x7f94ec0adea0 legacy=0x7f94ec0b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.810+0000 7f94f2000640 1 -- 172.21.15.97:0/4235066359 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec15e210 con 0x7f94ec0ccd00 2024-01-31T05:01:58.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.811+0000 7f94f2000640 1 --2- 172.21.15.97:0/4235066359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 0x7f94ec0cd190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009e50 tx=0x7f94e402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a153c2f9fefefcf0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:58.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.812+0000 7f94ebe00640 1 -- 172.21.15.97:0/4235066359 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e402ca70 con 0x7f94ec0ccd00 2024-01-31T05:01:58.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.812+0000 7f94ebe00640 1 -- 172.21.15.97:0/4235066359 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e402cbd0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.812+0000 7f94ebe00640 1 -- 172.21.15.97:0/4235066359 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e40343f0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 -- 172.21.15.97:0/4235066359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 msgr2=0x7f94ec0cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 --2- 172.21.15.97:0/4235066359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 0x7f94ec0cd190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009e50 tx=0x7f94e402bed0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 -- 172.21.15.97:0/4235066359 shutdown_connections 2024-01-31T05:01:58.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 --2- 172.21.15.97:0/4235066359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec0ccd00 0x7f94ec0cd190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 --2- 172.21.15.97:0/4235066359 >> v2:172.21.15.188:3300/0 conn(0x7f94ec1599a0 0x7f94ec0ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 -- 172.21.15.97:0/4235066359 >> 172.21.15.97:0/4235066359 conn(0x7f94ec0b0a10 msgr2=0x7f94ec0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:58.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 -- 172.21.15.97:0/4235066359 shutdown_connections 2024-01-31T05:01:58.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.813+0000 7f94f4000640 1 -- 172.21.15.97:0/4235066359 wait complete. 2024-01-31T05:01:58.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.814+0000 7f94f4000640 1 Processor -- start 2024-01-31T05:01:58.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.815+0000 7f94f4000640 1 -- start start 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.815+0000 7f94f4000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec0adea0 0x7f94ec158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f4000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 0x7f94ec14a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f4000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec1599a0 0x7f94ec0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f4000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0ac500 con 0x7f94ec0ccd00 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f4000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0ac7e0 con 0x7f94ec0adea0 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f4000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0acac0 con 0x7f94ec1599a0 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 0x7f94ec14a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 0x7f94ec14a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39982/0 (socket says 172.21.15.97:39982) 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 -- 172.21.15.97:0/1331130845 learned_addr learned my addr 172.21.15.97:0/1331130845 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f1600640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec0adea0 0x7f94ec158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f2000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec1599a0 0x7f94ec0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 -- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec1599a0 msgr2=0x7f94ec0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec1599a0 0x7f94ec0ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 -- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec0adea0 msgr2=0x7f94ec158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec0adea0 0x7f94ec158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f0c00640 1 -- 172.21.15.97:0/1331130845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4009b30 con 0x7f94ec0ccd00 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.816+0000 7f94f2000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec1599a0 0x7f94ec0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.817+0000 7f94f0c00640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 0x7f94ec14a030 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c3c0 tx=0x7f94d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.817+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d40052d0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.817+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d4005450 con 0x7f94ec0ccd00 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.817+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec0acda0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.817+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4007260 con 0x7f94ec0ccd00 2024-01-31T05:01:58.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.817+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec158970 con 0x7f94ec0ccd00 2024-01-31T05:01:58.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.818+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec0ad2c0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.819+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f94d40073c0 con 0x7f94ec0ccd00 2024-01-31T05:01:58.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.820+0000 7f94e9600640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f94c406aed0 0x7f94c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:58.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.820+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f94d4088d40 con 0x7f94ec0ccd00 2024-01-31T05:01:58.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.820+0000 7f94f1600640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f94c406aed0 0x7f94c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:58.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.821+0000 7f94f1600640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f94c406aed0 0x7f94c406d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009ac0 tx=0x7f94dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:58.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:58.824+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f94ec0ad2c0 con 0x7f94ec0ccd00 2024-01-31T05:01:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.036+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f94ec0ad2c0 con 0x7f94ec0ccd00 2024-01-31T05:01:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.037+0000 7f94e9600640 1 -- 172.21.15.97:0/1331130845 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f94ec0ad2c0 con 0x7f94ec0ccd00 2024-01-31T05:01:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f94c406aed0 msgr2=0x7f94c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f94c406aed0 0x7f94c406d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009ac0 tx=0x7f94dc008040 comp rx=0 tx=0).stop 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 msgr2=0x7f94ec14a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 0x7f94ec14a030 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c3c0 tx=0x7f94d400c880 comp rx=0 tx=0).stop 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 shutdown_connections 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94ec1599a0 0x7f94ec0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f94c406aed0 0x7f94c406d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94ec0ccd00 0x7f94ec14a030 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 --2- 172.21.15.97:0/1331130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94ec0adea0 0x7f94ec158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 >> 172.21.15.97:0/1331130845 conn(0x7f94ec0b0a10 msgr2=0x7f94ec0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 shutdown_connections 2024-01-31T05:01:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.040+0000 7f94f4000640 1 -- 172.21.15.97:0/1331130845 wait complete. 2024-01-31T05:01:59.058 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:01:59.058 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:01:59.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:01:59.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.332+0000 7f6e02600640 1 Processor -- start 2024-01-31T05:01:59.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.332+0000 7f6e02600640 1 -- start start 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.333+0000 7f6e02600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6dfc0c9f40 0x7f6dfc0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.333+0000 7f6e02600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc071e90 0x7f6dfc0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.333+0000 7f6e02600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc15a5c0 con 0x7f6dfc0c9420 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.333+0000 7f6e02600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc0ac400 con 0x7f6dfc0c9f40 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.333+0000 7f6e02600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0cd8d0 con 0x7f6dfc071e90 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6e01200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6dfc0c9f40 0x7f6dfc0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6df3c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6dfc0c9420 0x7f6dfc0abcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38698/0 (socket says 172.21.15.97:38698) 2024-01-31T05:01:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6df3c00640 1 -- 172.21.15.97:0/1754954207 learned_addr learned my addr 172.21.15.97:0/1754954207 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:01:59.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6e01c00640 1 --2- 172.21.15.97:0/1754954207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc071e90 0x7f6dfc0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6e01200640 1 -- 172.21.15.97:0/1754954207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc071e90 msgr2=0x7f6dfc0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:59.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6e01200640 1 --2- 172.21.15.97:0/1754954207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc071e90 0x7f6dfc0cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6e01200640 1 -- 172.21.15.97:0/1754954207 >> v1:172.21.15.97:6789/0 conn(0x7f6dfc0c9420 legacy=0x7f6dfc0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.334+0000 7f6e01200640 1 -- 172.21.15.97:0/1754954207 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc072320 con 0x7f6dfc0c9f40 2024-01-31T05:01:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.335+0000 7f6e01200640 1 --2- 172.21.15.97:0/1754954207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9f40 0x7f6dfc0ca320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097a0 tx=0x7f6dec02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500136d8bf6bfcbe server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.335+0000 7f6dfbe00640 1 -- 172.21.15.97:0/1754954207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec02d040 con 0x7f6dfc0c9f40 2024-01-31T05:01:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.335+0000 7f6dfbe00640 1 -- 172.21.15.97:0/1754954207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec031810 con 0x7f6dfc0c9f40 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.335+0000 7f6dfbe00640 1 -- 172.21.15.97:0/1754954207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec031ba0 con 0x7f6dfc0c9f40 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 -- 172.21.15.97:0/1754954207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9f40 msgr2=0x7f6dfc0ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 --2- 172.21.15.97:0/1754954207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9f40 0x7f6dfc0ca320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097a0 tx=0x7f6dec02b8b0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 -- 172.21.15.97:0/1754954207 shutdown_connections 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 --2- 172.21.15.97:0/1754954207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc071e90 0x7f6dfc0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 --2- 172.21.15.97:0/1754954207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9f40 0x7f6dfc0ca320 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 -- 172.21.15.97:0/1754954207 >> 172.21.15.97:0/1754954207 conn(0x7f6dfc0cec40 msgr2=0x7f6dfc0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 -- 172.21.15.97:0/1754954207 shutdown_connections 2024-01-31T05:01:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.336+0000 7f6e02600640 1 -- 172.21.15.97:0/1754954207 wait complete. 2024-01-31T05:01:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.338+0000 7f6e02600640 1 Processor -- start 2024-01-31T05:01:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.338+0000 7f6e02600640 1 -- start start 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6e02600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc071e90 0x7f6dfc09c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6e02600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 0x7f6dfc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6e02600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc0c9f40 0x7f6dfc152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6e02600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b1ed0 con 0x7f6dfc0c9420 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6e02600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc14f7c0 con 0x7f6dfc071e90 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6e02600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc1527d0 con 0x7f6dfc0c9f40 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6df3c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 0x7f6dfc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6df3c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 0x7f6dfc0b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39998/0 (socket says 172.21.15.97:39998) 2024-01-31T05:01:59.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.339+0000 7f6df3c00640 1 -- 172.21.15.97:0/3398864672 learned_addr learned my addr 172.21.15.97:0/3398864672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df3c00640 1 -- 172.21.15.97:0/3398864672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc0c9f40 msgr2=0x7f6dfc152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df3c00640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc0c9f40 0x7f6dfc152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df3c00640 1 -- 172.21.15.97:0/3398864672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc071e90 msgr2=0x7f6dfc09c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df3c00640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc071e90 0x7f6dfc09c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df3c00640 1 -- 172.21.15.97:0/3398864672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec009400 con 0x7f6dfc0c9420 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df3c00640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 0x7f6dfc0b1990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6de400cb10 tx=0x7f6de400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400ac70 con 0x7f6dfc0c9420 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de400add0 con 0x7f6dfc0c9420 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4012620 con 0x7f6dfc0c9420 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc0b2110 con 0x7f6dfc0c9420 2024-01-31T05:01:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.340+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc0b26d0 con 0x7f6dfc0c9420 2024-01-31T05:01:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.341+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc0676b0 con 0x7f6dfc0c9420 2024-01-31T05:01:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.342+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6de4007070 con 0x7f6dfc0c9420 2024-01-31T05:01:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.343+0000 7f6df9600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6dd406aed0 0x7f6dd406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.343+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6de4089940 con 0x7f6dfc0c9420 2024-01-31T05:01:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.343+0000 7f6e01200640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6dd406aed0 0x7f6dd406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.344+0000 7f6e01200640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6dd406aed0 0x7f6dd406d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6dec036000 tx=0x7f6dec030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:59.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.348+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6de4054530 con 0x7f6dfc0c9420 2024-01-31T05:01:59.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.542+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dfc14f900 con 0x7f6dfc0c9420 2024-01-31T05:01:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.544+0000 7f6df9600640 1 -- 172.21.15.97:0/3398864672 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f6de4017020 con 0x7f6dfc0c9420 2024-01-31T05:01:59.545 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:01:59.545 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:01:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.547+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6dd406aed0 msgr2=0x7f6dd406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.547+0000 7f6e02600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6dd406aed0 0x7f6dd406d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6dec036000 tx=0x7f6dec030000 comp rx=0 tx=0).stop 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.547+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 msgr2=0x7f6dfc0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.547+0000 7f6e02600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 0x7f6dfc0b1990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6de400cb10 tx=0x7f6de400a060 comp rx=0 tx=0).stop 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 shutdown_connections 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6dfc0c9f40 0x7f6dfc152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6dd406aed0 0x7f6dd406d390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dfc0c9420 0x7f6dfc0b1990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 --2- 172.21.15.97:0/3398864672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6dfc071e90 0x7f6dfc09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 >> 172.21.15.97:0/3398864672 conn(0x7f6dfc0cec40 msgr2=0x7f6dfc0cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 shutdown_connections 2024-01-31T05:01:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.548+0000 7f6e02600640 1 -- 172.21.15.97:0/3398864672 wait complete. 2024-01-31T05:01:59.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:01:59.567 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:01:59.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.772+0000 7f8722400640 1 Processor -- start 2024-01-31T05:01:59.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.772+0000 7f8722400640 1 -- start start 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8722400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f871c15a350 0x7f871c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8722400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f871c15a730 0x7f871c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8722400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c06dfc0 0x7f871c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8722400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c0b2c90 con 0x7f871c15a730 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8722400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c15abc0 con 0x7f871c15a350 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8722400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c15aea0 con 0x7f871c06dfc0 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8713e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f871c15a730 0x7f871c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8713e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f871c15a730 0x7f871c06da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40010/0 (socket says 172.21.15.97:40010) 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f8713e00640 1 -- 172.21.15.97:0/3736150369 learned_addr learned my addr 172.21.15.97:0/3736150369 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:59.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f871b400640 1 --2- 172.21.15.97:0/3736150369 >> v2:172.21.15.188:3300/0 conn(0x7f871c15a350 0x7f871c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.774+0000 7f871be00640 1 --2- 172.21.15.97:0/3736150369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c06dfc0 0x7f871c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.775+0000 7f8713e00640 1 -- 172.21.15.97:0/3736150369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c06dfc0 msgr2=0x7f871c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.775+0000 7f8713e00640 1 --2- 172.21.15.97:0/3736150369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c06dfc0 0x7f871c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.775+0000 7f8713e00640 1 -- 172.21.15.97:0/3736150369 >> v2:172.21.15.188:3300/0 conn(0x7f871c15a350 msgr2=0x7f871c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:01:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.775+0000 7f8713e00640 1 --2- 172.21.15.97:0/3736150369 >> v2:172.21.15.188:3300/0 conn(0x7f871c15a350 0x7f871c0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.775+0000 7f8713e00640 1 -- 172.21.15.97:0/3736150369 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c15b180 con 0x7f871c15a730 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.776+0000 7f8713e00640 1 --2- 172.21.15.97:0/3736150369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c15a730 0x7f871c06da80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f871c067370 tx=0x7f870402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d1622be9c4a4ca server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.776+0000 7f871aa00640 1 -- 172.21.15.97:0/3736150369 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704031860 con 0x7f871c15a730 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.776+0000 7f871aa00640 1 -- 172.21.15.97:0/3736150369 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87040319c0 con 0x7f871c15a730 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.776+0000 7f871aa00640 1 -- 172.21.15.97:0/3736150369 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704031d50 con 0x7f871c15a730 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 -- 172.21.15.97:0/3736150369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c15a730 msgr2=0x7f871c06da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 --2- 172.21.15.97:0/3736150369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c15a730 0x7f871c06da80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f871c067370 tx=0x7f870402b900 comp rx=0 tx=0).stop 2024-01-31T05:01:59.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 -- 172.21.15.97:0/3736150369 shutdown_connections 2024-01-31T05:01:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 --2- 172.21.15.97:0/3736150369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c06dfc0 0x7f871c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 --2- 172.21.15.97:0/3736150369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c15a730 0x7f871c06da80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 --2- 172.21.15.97:0/3736150369 >> v2:172.21.15.188:3300/0 conn(0x7f871c15a350 0x7f871c0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 -- 172.21.15.97:0/3736150369 >> 172.21.15.97:0/3736150369 conn(0x7f871c0b0a10 msgr2=0x7f871c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 -- 172.21.15.97:0/3736150369 shutdown_connections 2024-01-31T05:01:59.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.777+0000 7f8722400640 1 -- 172.21.15.97:0/3736150369 wait complete. 2024-01-31T05:01:59.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.778+0000 7f8722400640 1 Processor -- start 2024-01-31T05:01:59.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.779+0000 7f8722400640 1 -- start start 2024-01-31T05:01:59.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.779+0000 7f8722400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 0x7f871c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.779+0000 7f8722400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f871c15a350 0x7f871c09c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.779+0000 7f8722400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c09cae0 0x7f871c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.779+0000 7f8722400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c09b330 con 0x7f871c06dfc0 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f8722400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1543f0 con 0x7f871c15a350 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f8722400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c09a7b0 con 0x7f871c09cae0 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 0x7f871c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 0x7f871c153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40024/0 (socket says 172.21.15.97:40024) 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 -- 172.21.15.97:0/3921577095 learned_addr learned my addr 172.21.15.97:0/3921577095 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 -- 172.21.15.97:0/3921577095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c09cae0 msgr2=0x7f871c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c09cae0 0x7f871c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 -- 172.21.15.97:0/3921577095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f871c15a350 msgr2=0x7f871c09c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f871c15a350 0x7f871c09c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.780+0000 7f871b400640 1 -- 172.21.15.97:0/3921577095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040094d0 con 0x7f871c06dfc0 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.781+0000 7f871b400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 0x7f871c153eb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f870c00d800 tx=0x7f870c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:59.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.781+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00b6c0 con 0x7f871c06dfc0 2024-01-31T05:01:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.781+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c00b820 con 0x7f871c06dfc0 2024-01-31T05:01:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.781+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c09aa90 con 0x7f871c06dfc0 2024-01-31T05:01:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.781+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00bbd0 con 0x7f871c06dfc0 2024-01-31T05:01:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.781+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c09b050 con 0x7f871c06dfc0 2024-01-31T05:01:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.782+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c067830 con 0x7f871c06dfc0 2024-01-31T05:01:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.783+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f870c00bd30 con 0x7f871c06dfc0 2024-01-31T05:01:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.784+0000 7f8713400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86f406af00 0x7f86f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:01:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.784+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f870c089f70 con 0x7f871c06dfc0 2024-01-31T05:01:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.784+0000 7f8713e00640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86f406af00 0x7f86f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:01:59.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.785+0000 7f8713e00640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86f406af00 0x7f86f406d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8704036000 tx=0x7f8704030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:01:59.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.788+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f870c0bd190 con 0x7f871c06dfc0 2024-01-31T05:01:59.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.983+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f871c06cf30 con 0x7f871c06dfc0 2024-01-31T05:01:59.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.984+0000 7f8713400640 1 -- 172.21.15.97:0/3921577095 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f870c054a60 con 0x7f871c06dfc0 2024-01-31T05:01:59.983 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86f406af00 msgr2=0x7f86f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86f406af00 0x7f86f406d3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8704036000 tx=0x7f8704030000 comp rx=0 tx=0).stop 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 msgr2=0x7f871c153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 0x7f871c153eb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f870c00d800 tx=0x7f870c00dcc0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 shutdown_connections 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f871c09cae0 0x7f871c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f871c15a350 0x7f871c09c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f86f406af00 0x7f86f406d3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 --2- 172.21.15.97:0/3921577095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c06dfc0 0x7f871c153eb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:01:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.986+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 >> 172.21.15.97:0/3921577095 conn(0x7f871c0b0a10 msgr2=0x7f871c06c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:01:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.987+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 shutdown_connections 2024-01-31T05:01:59.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:01:59.987+0000 7f8722400640 1 -- 172.21.15.97:0/3921577095 wait complete. 2024-01-31T05:02:00.004 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.211+0000 7f2b87c00640 1 Processor -- start 2024-01-31T05:02:00.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.212+0000 7f2b87c00640 1 -- start start 2024-01-31T05:02:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.213+0000 7f2b87c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b8006d580 0x7f2b8006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.213+0000 7f2b87c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.213+0000 7f2b87c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8006dea0 con 0x7f2b8006c350 2024-01-31T05:02:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.213+0000 7f2b87c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8015e7c0 con 0x7f2b8006d580 2024-01-31T05:02:00.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.213+0000 7f2b87c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8015db40 con 0x7f2b8015d690 2024-01-31T05:02:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b8006d580 0x7f2b8006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2b8006c350 0x7f2b8006c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38706/0 (socket says 172.21.15.97:38706) 2024-01-31T05:02:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b8006d580 0x7f2b8006d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40026/0 (socket says 172.21.15.97:40026) 2024-01-31T05:02:00.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85c00640 1 -- 172.21.15.97:0/1531699356 learned_addr learned my addr 172.21.15.97:0/1531699356 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b7fe00640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85200640 1 -- 172.21.15.97:0/1531699356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 msgr2=0x7f2b800ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85200640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85200640 1 -- 172.21.15.97:0/1531699356 >> v1:172.21.15.97:6789/0 conn(0x7f2b8006c350 legacy=0x7f2b8006c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b7f400640 1 -- 172.21.15.97:0/1531699356 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223180560 0 0) 0x7f2b8006dea0 con 0x7f2b8006c350 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b85200640 1 -- 172.21.15.97:0/1531699356 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8015de20 con 0x7f2b8006d580 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.214+0000 7f2b7fe00640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.215+0000 7f2b85200640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006d580 0x7f2b8006d960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009790 tx=0x7f2b7002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63facdda3dd7bdd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b7f400640 1 -- 172.21.15.97:0/1531699356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7002d040 con 0x7f2b8006d580 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b7f400640 1 -- 172.21.15.97:0/1531699356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70031810 con 0x7f2b8006d580 2024-01-31T05:02:00.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b7f400640 1 -- 172.21.15.97:0/1531699356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70031ba0 con 0x7f2b8006d580 2024-01-31T05:02:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 -- 172.21.15.97:0/1531699356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006d580 msgr2=0x7f2b8006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006d580 0x7f2b8006d960 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009790 tx=0x7f2b7002b8b0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 -- 172.21.15.97:0/1531699356 shutdown_connections 2024-01-31T05:02:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 --2- 172.21.15.97:0/1531699356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006d580 0x7f2b8006d960 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 -- 172.21.15.97:0/1531699356 >> 172.21.15.97:0/1531699356 conn(0x7f2b800b0a10 msgr2=0x7f2b800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.216+0000 7f2b87c00640 1 -- 172.21.15.97:0/1531699356 shutdown_connections 2024-01-31T05:02:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.217+0000 7f2b87c00640 1 -- 172.21.15.97:0/1531699356 wait complete. 2024-01-31T05:02:00.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.218+0000 7f2b87c00640 1 Processor -- start 2024-01-31T05:02:00.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.218+0000 7f2b87c00640 1 -- start start 2024-01-31T05:02:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b87c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 0x7f2b80071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b87c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8006d580 0x7f2b800ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b87c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b87c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8009a490 con 0x7f2b8006c350 2024-01-31T05:02:00.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b87c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80151ca0 con 0x7f2b8006d580 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b87c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b800add80 con 0x7f2b8015d690 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b85200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 0x7f2b80071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.219+0000 7f2b85200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 0x7f2b80071810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40036/0 (socket says 172.21.15.97:40036) 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 -- 172.21.15.97:0/2377409671 learned_addr learned my addr 172.21.15.97:0/2377409671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b7fe00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8006d580 0x7f2b800ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 -- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 msgr2=0x7f2b800ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 -- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8006d580 msgr2=0x7f2b800ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8006d580 0x7f2b800ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 -- 172.21.15.97:0/2377409671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b700093a0 con 0x7f2b8006c350 2024-01-31T05:02:00.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b85200640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 0x7f2b80071810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2b700098b0 tx=0x7f2b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:00.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.220+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7003ad60 con 0x7f2b8006c350 2024-01-31T05:02:00.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.221+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70044400 con 0x7f2b8006c350 2024-01-31T05:02:00.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.221+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700447b0 con 0x7f2b8006c350 2024-01-31T05:02:00.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.221+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b80099af0 con 0x7f2b8006c350 2024-01-31T05:02:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.221+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8009a0b0 con 0x7f2b8006c350 2024-01-31T05:02:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.222+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b80067830 con 0x7f2b8006c350 2024-01-31T05:02:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.223+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2b7004d440 con 0x7f2b8006c350 2024-01-31T05:02:00.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.224+0000 7f2b7cc00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5806aed0 0x7f2b5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.224+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2b700abe70 con 0x7f2b8006c350 2024-01-31T05:02:00.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.224+0000 7f2b7fe00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5806aed0 0x7f2b5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.225+0000 7f2b7fe00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5806aed0 0x7f2b5806d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b8009aff0 tx=0x7f2b68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:00.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.228+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2b700769e0 con 0x7f2b8006c350 2024-01-31T05:02:00.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.420+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b800cc6c0 con 0x7f2b8006c350 2024-01-31T05:02:00.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.420+0000 7f2b7cc00640 1 -- 172.21.15.97:0/2377409671 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b70035020 con 0x7f2b8006c350 2024-01-31T05:02:00.420 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:00.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.423+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5806aed0 msgr2=0x7f2b5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.423+0000 7f2b87c00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5806aed0 0x7f2b5806d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b8009aff0 tx=0x7f2b68008040 comp rx=0 tx=0).stop 2024-01-31T05:02:00.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.423+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 msgr2=0x7f2b80071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.423+0000 7f2b87c00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 0x7f2b80071810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2b700098b0 tx=0x7f2b700046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 shutdown_connections 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b8015d690 0x7f2b800ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8006d580 0x7f2b800ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5806aed0 0x7f2b5806d390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 --2- 172.21.15.97:0/2377409671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b8006c350 0x7f2b80071810 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 >> 172.21.15.97:0/2377409671 conn(0x7f2b800b0a10 msgr2=0x7f2b801616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:00.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 shutdown_connections 2024-01-31T05:02:00.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.424+0000 7f2b87c00640 1 -- 172.21.15.97:0/2377409671 wait complete. 2024-01-31T05:02:00.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:00.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.655+0000 7f1fd3200640 1 Processor -- start 2024-01-31T05:02:00.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.655+0000 7f1fd3200640 1 -- start start 2024-01-31T05:02:00.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 0x7f1fcc164e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd3200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc070420 con 0x7f1fcc06d210 2024-01-31T05:02:00.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd3200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc070700 con 0x7f1fcc15af90 2024-01-31T05:02:00.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15b440 con 0x7f1fcc0ad780 2024-01-31T05:02:00.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd1e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1fcc06d210 0x7f1fcc071860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38708/0 (socket says 172.21.15.97:38708) 2024-01-31T05:02:00.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd1e00640 1 -- 172.21.15.97:0/983162943 learned_addr learned my addr 172.21.15.97:0/983162943 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:00.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.657+0000 7f1fd2800640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 0x7f1fcc164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955840325 0 0) 0x7f1fcc070420 con 0x7f1fcc06d210 2024-01-31T05:02:00.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac003520 con 0x7f1fcc06d210 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4202056530 0 0) 0x7f1fac003520 con 0x7f1fcc06d210 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 msgr2=0x7f1fcc164e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 0x7f1fcc164e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 >> v1:172.21.15.188:6789/0 conn(0x7f1fcc15af90 legacy=0x7f1fcc06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0709e0 con 0x7f1fcc06d210 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f1fbc0019f0 con 0x7f1fcc06d210 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 >> v1:172.21.15.97:6789/0 conn(0x7f1fcc06d210 legacy=0x7f1fcc071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fac0047c0 0x7f1fac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 0x7f1fac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15b440 con 0x7f1fac003a10 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00fec0 con 0x7f1fac0047c0 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.658+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac010080 con 0x7f1fcc0ad780 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd2800640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 0x7f1fac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd1e00640 1 -- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 msgr2=0x7f1fac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ad780 0x7f1fac005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd1e00640 1 -- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fac0047c0 msgr2=0x7f1fac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fac0047c0 0x7f1fac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.659+0000 7f1fd1e00640 1 -- 172.21.15.97:0/983162943 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac003520 con 0x7f1fac003a10 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.660+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fac003e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00a830 tx=0x7f1fac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd47cdeef54a4f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.660+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0072d0 con 0x7f1fac003a10 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.660+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc02f070 con 0x7f1fac003a10 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.660+0000 7f1fd0a00640 1 -- 172.21.15.97:0/983162943 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc007670 con 0x7f1fac003a10 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd3200640 1 -- 172.21.15.97:0/983162943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 msgr2=0x7f1fac003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd3200640 1 --2- 172.21.15.97:0/983162943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fac003e30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00a830 tx=0x7f1fac0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd2800640 1 -- 172.21.15.97:0/983162943 reap_dead start 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd3200640 1 -- 172.21.15.97:0/983162943 shutdown_connections 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd3200640 1 -- 172.21.15.97:0/983162943 >> 172.21.15.97:0/983162943 conn(0x7f1fcc06ad40 msgr2=0x7f1fcc0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd3200640 1 -- 172.21.15.97:0/983162943 shutdown_connections 2024-01-31T05:02:00.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.661+0000 7f1fd3200640 1 -- 172.21.15.97:0/983162943 wait complete. 2024-01-31T05:02:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.663+0000 7f1fd3200640 1 Processor -- start 2024-01-31T05:02:00.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.663+0000 7f1fd3200640 1 -- start start 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fcc0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ae780 0x7f1fcc0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fcc145950 0x7f1fcc0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc156070 con 0x7f1fac003a10 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0cb3c0 con 0x7f1fcc145950 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0aec60 con 0x7f1fcc0ae780 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fcc0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fcc0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40056/0 (socket says 172.21.15.97:40056) 2024-01-31T05:02:00.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.664+0000 7f1fd1e00640 1 -- 172.21.15.97:0/4019068414 learned_addr learned my addr 172.21.15.97:0/4019068414 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:00.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1400640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ae780 0x7f1fcc0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd2800640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fcc145950 0x7f1fcc0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1e00640 1 -- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ae780 msgr2=0x7f1fcc0999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ae780 0x7f1fcc0999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1e00640 1 -- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fcc145950 msgr2=0x7f1fcc0afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fcc145950 0x7f1fcc0afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1e00640 1 -- 172.21.15.97:0/4019068414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0aef40 con 0x7f1fac003a10 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd1e00640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fcc0cae80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc005c20 tx=0x7f1fbc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0074e0 con 0x7f1fac003a10 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc007640 con 0x7f1fac003a10 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0079b0 con 0x7f1fac003a10 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc0af220 con 0x7f1fac003a10 2024-01-31T05:02:00.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.665+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc0af7e0 con 0x7f1fac003a10 2024-01-31T05:02:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.667+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc0af970 con 0x7f1fac003a10 2024-01-31T05:02:00.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.667+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1fbc031030 con 0x7f1fac003a10 2024-01-31T05:02:00.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.668+0000 7f1fca000640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fa406aed0 0x7f1fa406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:00.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.668+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1fbc0ab600 con 0x7f1fac003a10 2024-01-31T05:02:00.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.669+0000 7f1fd1400640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fa406aed0 0x7f1fa406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:00.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.669+0000 7f1fd1400640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fa406aed0 0x7f1fa406d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc15a0b0 tx=0x7f1fb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.673+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1fbc0760e0 con 0x7f1fac003a10 2024-01-31T05:02:00.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.867+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fcc0676b0 con 0x7f1fac003a10 2024-01-31T05:02:00.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.868+0000 7f1fca000640 1 -- 172.21.15.97:0/4019068414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fbc079700 con 0x7f1fac003a10 2024-01-31T05:02:00.877 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:00.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fa406aed0 msgr2=0x7f1fa406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fa406aed0 0x7f1fa406d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc15a0b0 tx=0x7f1fb4009040 comp rx=0 tx=0).stop 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 msgr2=0x7f1fcc0cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fcc0cae80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc005c20 tx=0x7f1fbc001d80 comp rx=0 tx=0).stop 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 shutdown_connections 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fcc145950 0x7f1fcc0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fcc0ae780 0x7f1fcc0999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fa406aed0 0x7f1fa406d390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 --2- 172.21.15.97:0/4019068414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fac003a10 0x7f1fcc0cae80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 >> 172.21.15.97:0/4019068414 conn(0x7f1fcc06ad40 msgr2=0x7f1fcc06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 shutdown_connections 2024-01-31T05:02:00.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:00.871+0000 7f1fd3200640 1 -- 172.21.15.97:0/4019068414 wait complete. 2024-01-31T05:02:00.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:02:01.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.098+0000 7f0d07200640 1 Processor -- start 2024-01-31T05:02:01.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.099+0000 7f0d07200640 1 -- start start 2024-01-31T05:02:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.100+0000 7f0d07200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d000ce520 0x7f0d000abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.100+0000 7f0d07200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0015abc0 con 0x7f0d0015a730 2024-01-31T05:02:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.100+0000 7f0d07200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0015aea0 con 0x7f0d0015a350 2024-01-31T05:02:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.100+0000 7f0d07200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d000b2c90 con 0x7f0d000ce520 2024-01-31T05:02:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cff400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0d0015a730 0x7f0d000cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38714/0 (socket says 172.21.15.97:38714) 2024-01-31T05:02:01.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cff400640 1 -- 172.21.15.97:0/1997677548 learned_addr learned my addr 172.21.15.97:0/1997677548 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0d05200640 1 --2- 172.21.15.97:0/1997677548 >> v2:172.21.15.188:3300/0 conn(0x7f0d000ce520 0x7f0d000abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669418104 0 0) 0x7f0d0015abc0 con 0x7f0d0015a730 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce0003520 con 0x7f0d0015a730 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314664541 0 0) 0x7f0ce0003520 con 0x7f0d0015a730 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 >> v2:172.21.15.188:3300/0 conn(0x7f0d000ce520 msgr2=0x7f0d000abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.101+0000 7f0cfea00640 1 --2- 172.21.15.97:0/1997677548 >> v2:172.21.15.188:3300/0 conn(0x7f0d000ce520 0x7f0d000abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 >> v1:172.21.15.188:6789/0 conn(0x7f0d0015a350 legacy=0x7f0d0006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0015b180 con 0x7f0d0015a730 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f0ce80019f0 con 0x7f0d0015a730 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 >> v1:172.21.15.97:6789/0 conn(0x7f0d0015a730 legacy=0x7f0d000cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0ce0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce00047c0 0x7f0ce0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce00052a0 0x7f0ce000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b2c90 con 0x7f0ce0003970 2024-01-31T05:02:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce000c170 con 0x7f0ce00052a0 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce000c330 con 0x7f0ce00047c0 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cffe00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0ce0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0d05200640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce00052a0 0x7f0ce000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cff400640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce00047c0 0x7f0ce0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cffe00640 1 -- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce00047c0 msgr2=0x7f0ce0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cffe00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ce00047c0 0x7f0ce0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cffe00640 1 -- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce00052a0 msgr2=0x7f0ce000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cffe00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ce00052a0 0x7f0ce000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0cffe00640 1 -- 172.21.15.97:0/1997677548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0003520 con 0x7f0ce0003970 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.102+0000 7f0d05200640 1 -- 172.21.15.97:0/1997677548 reap_dead start 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.104+0000 7f0cffe00640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0ce0003d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0007fd0 tx=0x7f0ce00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50ce90cd1008032 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.104+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00052d0 con 0x7f0ce0003970 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.104+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf002e050 con 0x7f0ce0003970 2024-01-31T05:02:01.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.104+0000 7f0cfea00640 1 -- 172.21.15.97:0/1997677548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0003640 con 0x7f0ce0003970 2024-01-31T05:02:01.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.104+0000 7f0d07200640 1 -- 172.21.15.97:0/1997677548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 msgr2=0x7f0ce0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.104+0000 7f0d07200640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0ce0003d90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0007fd0 tx=0x7f0ce00057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.105+0000 7f0d07200640 1 -- 172.21.15.97:0/1997677548 shutdown_connections 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.105+0000 7f0d07200640 1 --2- 172.21.15.97:0/1997677548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0ce0003d90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.105+0000 7f0d07200640 1 -- 172.21.15.97:0/1997677548 >> 172.21.15.97:0/1997677548 conn(0x7f0d000b0a10 msgr2=0x7f0d000b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.105+0000 7f0d07200640 1 -- 172.21.15.97:0/1997677548 shutdown_connections 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.105+0000 7f0d07200640 1 -- 172.21.15.97:0/1997677548 wait complete. 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.106+0000 7f0d07200640 1 Processor -- start 2024-01-31T05:02:01.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.107+0000 7f0d07200640 1 -- start start 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.107+0000 7f0d07200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0d0009c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0d07200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d0015c5b0 0x7f0d000c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0d07200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d000c9570 0x7f0d000c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0d07200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0009cad0 con 0x7f0ce0003970 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0d07200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000c9f10 con 0x7f0d000c9570 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0d07200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0015cae0 con 0x7f0d0015c5b0 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0d0009c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0d0009c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40074/0 (socket says 172.21.15.97:40074) 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 -- 172.21.15.97:0/19513545 learned_addr learned my addr 172.21.15.97:0/19513545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 -- 172.21.15.97:0/19513545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d0015c5b0 msgr2=0x7f0d000c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d0015c5b0 0x7f0d000c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 -- 172.21.15.97:0/19513545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d000c9570 msgr2=0x7f0d000c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d000c9570 0x7f0d000c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.108+0000 7f0cffe00640 1 -- 172.21.15.97:0/19513545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0015cdc0 con 0x7f0ce0003970 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.109+0000 7f0cffe00640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0d0009c590 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0034040 tx=0x7f0cf0001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.109+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0006ea0 con 0x7f0ce0003970 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.109+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0007a80 con 0x7f0ce0003970 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.109+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0007490 con 0x7f0ce0003970 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.109+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0015d0a0 con 0x7f0ce0003970 2024-01-31T05:02:01.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.109+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0015d660 con 0x7f0ce0003970 2024-01-31T05:02:01.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.110+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0015b740 con 0x7f0ce0003970 2024-01-31T05:02:01.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.111+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f0cf0003640 con 0x7f0ce0003970 2024-01-31T05:02:01.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.112+0000 7f0cf7e00640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0cd806aed0 0x7f0cd806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.112+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0cf00ab600 con 0x7f0ce0003970 2024-01-31T05:02:01.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.112+0000 7f0cff400640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0cd806aed0 0x7f0cd806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.113+0000 7f0cff400640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0cd806aed0 0x7f0cd806d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80013d0 tx=0x7f0ce80020c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:01.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.116+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f0cf0076170 con 0x7f0ce0003970 2024-01-31T05:02:01.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.310+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0d0015d7f0 con 0x7f0ce0003970 2024-01-31T05:02:01.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.311+0000 7f0cf7e00640 1 -- 172.21.15.97:0/19513545 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0cf0079790 con 0x7f0ce0003970 2024-01-31T05:02:01.310 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0cd806aed0 msgr2=0x7f0cd806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0cd806aed0 0x7f0cd806d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80013d0 tx=0x7f0ce80020c0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 msgr2=0x7f0d0009c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0d0009c590 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0034040 tx=0x7f0cf0001170 comp rx=0 tx=0).stop 2024-01-31T05:02:01.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 shutdown_connections 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d000c9570 0x7f0d000c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d0015c5b0 0x7f0d000c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0cd806aed0 0x7f0cd806d390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 --2- 172.21.15.97:0/19513545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce0003970 0x7f0d0009c590 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 >> 172.21.15.97:0/19513545 conn(0x7f0d000b0a10 msgr2=0x7f0d000b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 shutdown_connections 2024-01-31T05:02:01.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.314+0000 7f0d07200640 1 -- 172.21.15.97:0/19513545 wait complete. 2024-01-31T05:02:01.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:01.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.542+0000 7f75a8e00640 1 Processor -- start 2024-01-31T05:02:01.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.542+0000 7f75a8e00640 1 -- start start 2024-01-31T05:02:01.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a8e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75a40abcf0 0x7f75a40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a8e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a406cd20 con 0x7f75a40cbf20 2024-01-31T05:02:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a8e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a40ceb60 con 0x7f75a415b470 2024-01-31T05:02:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a8e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a415b850 con 0x7f75a40abcf0 2024-01-31T05:02:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a2800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75a40abcf0 0x7f75a40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a2800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75a40abcf0 0x7f75a40722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40086/0 (socket says 172.21.15.97:40086) 2024-01-31T05:02:01.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.544+0000 7f75a2800640 1 -- 172.21.15.97:0/107486421 learned_addr learned my addr 172.21.15.97:0/107486421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.545+0000 7f75a2800640 1 -- 172.21.15.97:0/107486421 >> v1:172.21.15.188:6789/0 conn(0x7f75a415b470 legacy=0x7f75a40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.545+0000 7f75a2800640 1 -- 172.21.15.97:0/107486421 >> v1:172.21.15.97:6789/0 conn(0x7f75a40cbf20 legacy=0x7f75a406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.545+0000 7f75a2800640 1 -- 172.21.15.97:0/107486421 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a406c3f0 con 0x7f75a40abcf0 2024-01-31T05:02:01.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.545+0000 7f75a0a00640 1 -- 172.21.15.97:0/107486421 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710645145 0 0) 0x7f75a406cd20 con 0x7f75a40cbf20 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.546+0000 7f75a2800640 1 --2- 172.21.15.97:0/107486421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40722c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f759c009e50 tx=0x7f759c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680283f05a3fa64e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.546+0000 7f75a0a00640 1 -- 172.21.15.97:0/107486421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c02cbb0 con 0x7f75a40abcf0 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.546+0000 7f75a0a00640 1 -- 172.21.15.97:0/107486421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c02cd10 con 0x7f75a40abcf0 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.546+0000 7f75a0a00640 1 -- 172.21.15.97:0/107486421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c034610 con 0x7f75a40abcf0 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 -- 172.21.15.97:0/107486421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 msgr2=0x7f75a40722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 --2- 172.21.15.97:0/107486421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40722c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f759c009e50 tx=0x7f759c02c010 comp rx=0 tx=0).stop 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 -- 172.21.15.97:0/107486421 shutdown_connections 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 --2- 172.21.15.97:0/107486421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40722c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 -- 172.21.15.97:0/107486421 >> 172.21.15.97:0/107486421 conn(0x7f75a40b0a10 msgr2=0x7f75a40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 -- 172.21.15.97:0/107486421 shutdown_connections 2024-01-31T05:02:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.547+0000 7f75a8e00640 1 -- 172.21.15.97:0/107486421 wait complete. 2024-01-31T05:02:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.549+0000 7f75a8e00640 1 Processor -- start 2024-01-31T05:02:01.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.549+0000 7f75a8e00640 1 -- start start 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a40cbf20 0x7f75a415c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a415b470 0x7f75a41f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a40b0670 con 0x7f75a40abcf0 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a8e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a415cac0 con 0x7f75a40cbf20 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a415cda0 con 0x7f75a415b470 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40100/0 (socket says 172.21.15.97:40100) 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.550+0000 7f75a1e00640 1 -- 172.21.15.97:0/2609713965 learned_addr learned my addr 172.21.15.97:0/2609713965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1400640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a40cbf20 0x7f75a415c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1e00640 1 -- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a415b470 msgr2=0x7f75a41f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a415b470 0x7f75a41f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1e00640 1 -- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a40cbf20 msgr2=0x7f75a415c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a40cbf20 0x7f75a415c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1e00640 1 -- 172.21.15.97:0/2609713965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009b30 con 0x7f75a40abcf0 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1400640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a40cbf20 0x7f75a415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f75a1e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40b0130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f759400c3a0 tx=0x7f759400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.551+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75940052d0 con 0x7f75a40abcf0 2024-01-31T05:02:01.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.552+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7594005450 con 0x7f75a40abcf0 2024-01-31T05:02:01.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.552+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a415d080 con 0x7f75a40abcf0 2024-01-31T05:02:01.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.552+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594003880 con 0x7f75a40abcf0 2024-01-31T05:02:01.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.552+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a415d640 con 0x7f75a40abcf0 2024-01-31T05:02:01.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.553+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a4067830 con 0x7f75a40abcf0 2024-01-31T05:02:01.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.554+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f75940039e0 con 0x7f75a40abcf0 2024-01-31T05:02:01.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.555+0000 7f759a000640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f757c06aed0 0x7f757c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:01.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.555+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7594088f80 con 0x7f75a40abcf0 2024-01-31T05:02:01.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.555+0000 7f75a1400640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f757c06aed0 0x7f757c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:01.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.556+0000 7f75a1400640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f757c06aed0 0x7f757c06d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75a4155760 tx=0x7f758c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.559+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7594053af0 con 0x7f75a40abcf0 2024-01-31T05:02:01.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.767+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f75a415d7d0 con 0x7f75a40abcf0 2024-01-31T05:02:01.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.768+0000 7f759a000640 1 -- 172.21.15.97:0/2609713965 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7594017080 con 0x7f75a40abcf0 2024-01-31T05:02:01.768 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f757c06aed0 msgr2=0x7f757c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f757c06aed0 0x7f757c06d390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f75a4155760 tx=0x7f758c009040 comp rx=0 tx=0).stop 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 msgr2=0x7f75a40b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40b0130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f759400c3a0 tx=0x7f759400c860 comp rx=0 tx=0).stop 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 shutdown_connections 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75a415b470 0x7f75a41f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75a40cbf20 0x7f75a415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f757c06aed0 0x7f757c06d390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 --2- 172.21.15.97:0/2609713965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75a40abcf0 0x7f75a40b0130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 >> 172.21.15.97:0/2609713965 conn(0x7f75a40b0a10 msgr2=0x7f75a40cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 shutdown_connections 2024-01-31T05:02:01.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:01.771+0000 7f75a8e00640 1 -- 172.21.15.97:0/2609713965 wait complete. 2024-01-31T05:02:01.789 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2024-01-31T05:02:01.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:01.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-31T05:02:01.889 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:01.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:02:01.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:01.889 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-31T05:02:01.942 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:02:01.942 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:01.997 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:02:01.998 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:02.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.256+0000 7ff67a000640 1 Processor -- start 2024-01-31T05:02:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.257+0000 7ff67a000640 1 -- start start 2024-01-31T05:02:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.258+0000 7ff67a000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6740b2330 0x7ff6740d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.258+0000 7ff67a000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67406b570 con 0x7ff6740ce000 2024-01-31T05:02:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.258+0000 7ff67a000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740cb6c0 con 0x7ff674160e00 2024-01-31T05:02:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.258+0000 7ff67a000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6741611e0 con 0x7ff6740b2330 2024-01-31T05:02:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff66a800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6740ce000 0x7ff6740cbe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38734/0 (socket says 172.21.15.97:38734) 2024-01-31T05:02:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff66a800640 1 -- 172.21.15.97:0/3986487133 learned_addr learned my addr 172.21.15.97:0/3986487133 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667403634 0 0) 0x7ff67406b570 con 0x7ff6740ce000 2024-01-31T05:02:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff673e00640 1 --2- 172.21.15.97:0/3986487133 >> v2:172.21.15.188:3300/0 conn(0x7ff6740b2330 0x7ff6740d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff654003520 con 0x7ff6740ce000 2024-01-31T05:02:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578306515 0 0) 0x7ff654003520 con 0x7ff6740ce000 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 >> v2:172.21.15.188:3300/0 conn(0x7ff6740b2330 msgr2=0x7ff6740d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 --2- 172.21.15.97:0/3986487133 >> v2:172.21.15.188:3300/0 conn(0x7ff6740b2330 0x7ff6740d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 >> v1:172.21.15.188:6789/0 conn(0x7ff674160e00 legacy=0x7ff6740cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.259+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740cfff0 con 0x7ff6740ce000 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff65c001670 con 0x7ff6740ce000 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 >> v1:172.21.15.97:6789/0 conn(0x7ff6740ce000 legacy=0x7ff6740cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff654003970 0x7ff654003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff654004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff654005180 0x7ff65400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6741611e0 con 0x7ff6540047c0 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65400c020 con 0x7ff654005180 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65400c1e0 con 0x7ff654003970 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff66a800640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff654004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff66a800640 1 -- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff654003970 msgr2=0x7ff654003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff673400640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff654003970 0x7ff654003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff673e00640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff654005180 0x7ff65400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff66a800640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff654003970 0x7ff654003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff66a800640 1 -- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff654005180 msgr2=0x7ff65400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff66a800640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff654005180 0x7ff65400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff66a800640 1 -- 172.21.15.97:0/3986487133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654003520 con 0x7ff6540047c0 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff673e00640 1 -- 172.21.15.97:0/3986487133 reap_dead start 2024-01-31T05:02:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.260+0000 7ff673400640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff654003970 0x7ff654003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.262+0000 7ff66a800640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff654004c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00c240 tx=0x7ff6540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f42aaefe6d9676 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.262+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c005ad0 con 0x7ff6540047c0 2024-01-31T05:02:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.262+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c005c30 con 0x7ff6540047c0 2024-01-31T05:02:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.262+0000 7ff672a00640 1 -- 172.21.15.97:0/3986487133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c02caf0 con 0x7ff6540047c0 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.262+0000 7ff67a000640 1 -- 172.21.15.97:0/3986487133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 msgr2=0x7ff654004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.262+0000 7ff67a000640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff654004c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00c240 tx=0x7ff6540057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.263+0000 7ff67a000640 1 -- 172.21.15.97:0/3986487133 shutdown_connections 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.263+0000 7ff67a000640 1 --2- 172.21.15.97:0/3986487133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff654004c40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.263+0000 7ff67a000640 1 -- 172.21.15.97:0/3986487133 >> 172.21.15.97:0/3986487133 conn(0x7ff6740d1070 msgr2=0x7ff6740d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.263+0000 7ff67a000640 1 -- 172.21.15.97:0/3986487133 shutdown_connections 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.263+0000 7ff67a000640 1 -- 172.21.15.97:0/3986487133 wait complete. 2024-01-31T05:02:02.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.265+0000 7ff67a000640 1 Processor -- start 2024-01-31T05:02:02.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.265+0000 7ff67a000640 1 -- start start 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff67a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff6741f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff67a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6740cabd0 0x7ff6741563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff67a000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff674156910 0x7ff674154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff67a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67414dd80 con 0x7ff6540047c0 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff67a000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff674156e40 con 0x7ff674156910 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff67a000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff674157120 con 0x7ff6740cabd0 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff673400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff6741f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff673400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff6741f28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40112/0 (socket says 172.21.15.97:40112) 2024-01-31T05:02:02.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.266+0000 7ff673400640 1 -- 172.21.15.97:0/574628016 learned_addr learned my addr 172.21.15.97:0/574628016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:02.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff66a800640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6740cabd0 0x7ff6741563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff673400640 1 -- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6740cabd0 msgr2=0x7ff6741563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff673400640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6740cabd0 0x7ff6741563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff673400640 1 -- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff674156910 msgr2=0x7ff674154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff673400640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff674156910 0x7ff674154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff673400640 1 -- 172.21.15.97:0/574628016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674157400 con 0x7ff6540047c0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff66a800640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6740cabd0 0x7ff6741563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff673400640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff6741f28c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff66400d800 tx=0x7ff66400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664003da0 con 0x7ff6540047c0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff664005070 con 0x7ff6540047c0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6741576e0 con 0x7ff6540047c0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.267+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664005460 con 0x7ff6540047c0 2024-01-31T05:02:02.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.268+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff674154aa0 con 0x7ff6540047c0 2024-01-31T05:02:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.269+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6740677c0 con 0x7ff6540047c0 2024-01-31T05:02:02.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.270+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff6640055c0 con 0x7ff6540047c0 2024-01-31T05:02:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.271+0000 7ff66be00640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff64c06af00 0x7ff64c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.271+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff664089c30 con 0x7ff6540047c0 2024-01-31T05:02:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.271+0000 7ff66a800640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff64c06af00 0x7ff64c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.272+0000 7ff66a800640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff64c06af00 0x7ff64c06d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff65c034000 tx=0x7ff65c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:02.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.276+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff664054690 con 0x7ff6540047c0 2024-01-31T05:02:02.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.469+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6740b6ef0 con 0x7ff6540047c0 2024-01-31T05:02:02.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.473+0000 7ff66be00640 1 -- 172.21.15.97:0/574628016 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7ff664057cb0 con 0x7ff6540047c0 2024-01-31T05:02:02.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:02.473 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff64c06af00 msgr2=0x7ff64c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff64c06af00 0x7ff64c06d3c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff65c034000 tx=0x7ff65c009030 comp rx=0 tx=0).stop 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 msgr2=0x7ff6741f28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff6741f28c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff66400d800 tx=0x7ff66400dcc0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 shutdown_connections 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff674156910 0x7ff674154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6740cabd0 0x7ff6741563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff64c06af00 0x7ff64c06d3c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 --2- 172.21.15.97:0/574628016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540047c0 0x7ff6741f28c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.475+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 >> 172.21.15.97:0/574628016 conn(0x7ff6740d1070 msgr2=0x7ff6740cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.476+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 shutdown_connections 2024-01-31T05:02:02.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.476+0000 7ff67a000640 1 -- 172.21.15.97:0/574628016 wait complete. 2024-01-31T05:02:02.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:02:02.496 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:02.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.715+0000 7f0383200640 1 Processor -- start 2024-01-31T05:02:02.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.715+0000 7f0383200640 1 -- start start 2024-01-31T05:02:02.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0383200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f038415b080 0x7f03840b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0383200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 0x7f03840cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0383200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03840b3f60 con 0x7f03840ce4b0 2024-01-31T05:02:02.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0383200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03840b41c0 con 0x7f038415b080 2024-01-31T05:02:02.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0383200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03840ce920 con 0x7f03840ae360 2024-01-31T05:02:02.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0382800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f03840ce4b0 0x7f038415eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38748/0 (socket says 172.21.15.97:38748) 2024-01-31T05:02:02.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0382800640 1 -- 172.21.15.97:0/3869060971 learned_addr learned my addr 172.21.15.97:0/3869060971 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:02.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0381400640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 0x7f03840cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0381e00640 1 --2- 172.21.15.97:0/3869060971 >> v2:172.21.15.188:3300/0 conn(0x7f038415b080 0x7f03840b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845593069 0 0) 0x7f03840b3f60 con 0x7f03840ce4b0 2024-01-31T05:02:02.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.717+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0364003520 con 0x7f03840ce4b0 2024-01-31T05:02:02.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355590820 0 0) 0x7f0364003520 con 0x7f03840ce4b0 2024-01-31T05:02:02.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 msgr2=0x7f03840cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 0x7f03840cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 >> v2:172.21.15.188:3300/0 conn(0x7f038415b080 msgr2=0x7f03840b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 --2- 172.21.15.97:0/3869060971 >> v2:172.21.15.188:3300/0 conn(0x7f038415b080 0x7f03840b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038415c540 con 0x7f03840ce4b0 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f037c0019f0 con 0x7f03840ce4b0 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 >> v1:172.21.15.97:6789/0 conn(0x7f03840ce4b0 legacy=0x7f038415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03640039e0 0x7f0364003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03640047c0 0x7f0364004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 0x7f0364005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840b41c0 con 0x7f03640039e0 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840ce920 con 0x7f03640047c0 2024-01-31T05:02:02.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036400ba00 con 0x7f03840ae360 2024-01-31T05:02:02.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0381e00640 1 --2- 172.21.15.97:0/3869060971 >> v2:172.21.15.188:3300/0 conn(0x7f038415b080 0x7f03840b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:02:02.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.718+0000 7f0381e00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03640039e0 0x7f0364003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.719+0000 7f0382800640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 0x7f0364005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.719+0000 7f0381e00640 1 -- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 msgr2=0x7f0364005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.719+0000 7f0381e00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840ae360 0x7f0364005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.719+0000 7f0381e00640 1 -- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03640047c0 msgr2=0x7f0364004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.719+0000 7f0381e00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03640047c0 0x7f0364004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.719+0000 7f0381e00640 1 -- 172.21.15.97:0/3869060971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364003520 con 0x7f03640039e0 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.720+0000 7f0381e00640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03640039e0 0x7f0364003e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f03740098a0 tx=0x7f03640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70da43bc4ffaad6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.720+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374037600 con 0x7f03640039e0 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.720+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0374037760 con 0x7f03640039e0 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.720+0000 7f0380a00640 1 -- 172.21.15.97:0/3869060971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374037ad0 con 0x7f03640039e0 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0383200640 1 -- 172.21.15.97:0/3869060971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03640039e0 msgr2=0x7f0364003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0383200640 1 --2- 172.21.15.97:0/3869060971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03640039e0 0x7f0364003e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f03740098a0 tx=0x7f03640057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0382800640 1 -- 172.21.15.97:0/3869060971 reap_dead start 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0383200640 1 -- 172.21.15.97:0/3869060971 shutdown_connections 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0383200640 1 -- 172.21.15.97:0/3869060971 >> 172.21.15.97:0/3869060971 conn(0x7f038406ad40 msgr2=0x7f03840b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0383200640 1 -- 172.21.15.97:0/3869060971 shutdown_connections 2024-01-31T05:02:02.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.721+0000 7f0383200640 1 -- 172.21.15.97:0/3869060971 wait complete. 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.723+0000 7f0383200640 1 Processor -- start 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.723+0000 7f0383200640 1 -- start start 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0383200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03640039e0 0x7f0384151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0383200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038409bcb0 0x7f03841477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0383200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 0x7f038415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0383200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03841511a0 con 0x7f038415d7b0 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0383200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038415e170 con 0x7f038409bcb0 2024-01-31T05:02:02.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0383200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0384150d10 con 0x7f03640039e0 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0382800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 0x7f038415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0382800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 0x7f038415dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40124/0 (socket says 172.21.15.97:40124) 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.724+0000 7f0382800640 1 -- 172.21.15.97:0/3210483662 learned_addr learned my addr 172.21.15.97:0/3210483662 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0381e00640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03640039e0 0x7f0384151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0381400640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038409bcb0 0x7f03841477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0382800640 1 -- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03640039e0 msgr2=0x7f0384151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0382800640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03640039e0 0x7f0384151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0382800640 1 -- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038409bcb0 msgr2=0x7f03841477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0382800640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038409bcb0 0x7f03841477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0382800640 1 -- 172.21.15.97:0/3210483662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0384150140 con 0x7f038415d7b0 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0381400640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038409bcb0 0x7f03841477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0382800640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 0x7f038415dc30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f037c000f80 tx=0x7f037c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:02.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c00ec10 con 0x7f038415d7b0 2024-01-31T05:02:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c00ed90 con 0x7f038415d7b0 2024-01-31T05:02:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.725+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03841503e0 con 0x7f038415d7b0 2024-01-31T05:02:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.726+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c013660 con 0x7f038415d7b0 2024-01-31T05:02:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.726+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03841509a0 con 0x7f038415d7b0 2024-01-31T05:02:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.727+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0384150ea0 con 0x7f038415d7b0 2024-01-31T05:02:02.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.728+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f037c0137c0 con 0x7f038415d7b0 2024-01-31T05:02:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.729+0000 7f037a000640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f035c06aed0 0x7f035c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.729+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f037c089d60 con 0x7f038415d7b0 2024-01-31T05:02:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.729+0000 7f0381e00640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f035c06aed0 0x7f035c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:02.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.730+0000 7f0381e00640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f035c06aed0 0x7f035c06d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03740098a0 tx=0x7f0374031000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:02.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.734+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f037c0548d0 con 0x7f038415d7b0 2024-01-31T05:02:02.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.926+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03840676b0 con 0x7f038415d7b0 2024-01-31T05:02:02.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.927+0000 7f037a000640 1 -- 172.21.15.97:0/3210483662 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f037c057ef0 con 0x7f038415d7b0 2024-01-31T05:02:02.926 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:02.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f035c06aed0 msgr2=0x7f035c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f035c06aed0 0x7f035c06d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03740098a0 tx=0x7f0374031000 comp rx=0 tx=0).stop 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 msgr2=0x7f038415dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 0x7f038415dc30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f037c000f80 tx=0x7f037c00e530 comp rx=0 tx=0).stop 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 shutdown_connections 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f035c06aed0 0x7f035c06d390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f038415d7b0 0x7f038415dc30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038409bcb0 0x7f03841477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.929+0000 7f0383200640 1 --2- 172.21.15.97:0/3210483662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03640039e0 0x7f0384151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.930+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 >> 172.21.15.97:0/3210483662 conn(0x7f038406ad40 msgr2=0x7f038415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.930+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 shutdown_connections 2024-01-31T05:02:02.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:02.930+0000 7f0383200640 1 -- 172.21.15.97:0/3210483662 wait complete. 2024-01-31T05:02:02.947 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.153+0000 7f13f7e00640 1 Processor -- start 2024-01-31T05:02:03.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.153+0000 7f13f7e00640 1 -- start start 2024-01-31T05:02:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13f80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f7e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f80b3f60 con 0x7f13f80ce680 2024-01-31T05:02:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f815c540 con 0x7f13f80ae360 2024-01-31T05:02:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80b41c0 con 0x7f13f815b080 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f7400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f13f80ce680 0x7f13f80b21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38752/0 (socket says 172.21.15.97:38752) 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f7400640 1 -- 172.21.15.97:0/898256179 learned_addr learned my addr 172.21.15.97:0/898256179 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.155+0000 7f13f6a00640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13f80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167902189 0 0) 0x7f13f80b3f60 con 0x7f13f80ce680 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d8003520 con 0x7f13f80ce680 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762496056 0 0) 0x7f13d8003520 con 0x7f13f80ce680 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 msgr2=0x7f13f80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13f80b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 >> v1:172.21.15.188:6789/0 conn(0x7f13f80ae360 legacy=0x7f13f80cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f815b780 con 0x7f13f80ce680 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f13f00019f0 con 0x7f13f80ce680 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 >> v1:172.21.15.97:6789/0 conn(0x7f13f80ce680 legacy=0x7f13f80b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8003a10 0x7f13d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8004d00 0x7f13d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80b41c0 con 0x7f13d8003a10 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800ba90 con 0x7f13d8004d00 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.156+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800bc50 con 0x7f13f815b080 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6a00640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8003a10 0x7f13d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6000640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f7400640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8004d00 0x7f13d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6a00640 1 -- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 msgr2=0x7f13d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6a00640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6a00640 1 -- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8004d00 msgr2=0x7f13d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6a00640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8004d00 0x7f13d8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6a00640 1 -- 172.21.15.97:0/898256179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8003520 con 0x7f13d8003a10 2024-01-31T05:02:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f6000640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f815b080 0x7f13d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.157+0000 7f13f7400640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8004d00 0x7f13d8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.158+0000 7f13f6a00640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8003a10 0x7f13d8003e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009a90 tx=0x7f13d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e90fe652818f1da server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.158+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e802c5c0 con 0x7f13d8003a10 2024-01-31T05:02:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.158+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e802c720 con 0x7f13d8003a10 2024-01-31T05:02:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.158+0000 7f13f5600640 1 -- 172.21.15.97:0/898256179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e802cab0 con 0x7f13d8003a10 2024-01-31T05:02:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7e00640 1 -- 172.21.15.97:0/898256179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8003a10 msgr2=0x7f13d8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7e00640 1 --2- 172.21.15.97:0/898256179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13d8003a10 0x7f13d8003e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009a90 tx=0x7f13d80057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7400640 1 -- 172.21.15.97:0/898256179 reap_dead start 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7e00640 1 -- 172.21.15.97:0/898256179 shutdown_connections 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7e00640 1 -- 172.21.15.97:0/898256179 >> 172.21.15.97:0/898256179 conn(0x7f13f806ad40 msgr2=0x7f13f80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7e00640 1 -- 172.21.15.97:0/898256179 shutdown_connections 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.159+0000 7f13f7e00640 1 -- 172.21.15.97:0/898256179 wait complete. 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.161+0000 7f13f7e00640 1 Processor -- start 2024-01-31T05:02:03.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.161+0000 7f13f7e00640 1 -- start start 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8003a10 0x7f13f809a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f809bfa0 0x7f13f81563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 0x7f13f8154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8154640 con 0x7f13f8153c80 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f809caf0 con 0x7f13d8003a10 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f809c810 con 0x7f13f809bfa0 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 0x7f13f8154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 0x7f13f8154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40138/0 (socket says 172.21.15.97:40138) 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.162+0000 7f13f7400640 1 -- 172.21.15.97:0/1603153542 learned_addr learned my addr 172.21.15.97:0/1603153542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f6000640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f809bfa0 0x7f13f81563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7400640 1 -- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f809bfa0 msgr2=0x7f13f81563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f6a00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8003a10 0x7f13f809a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7400640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f809bfa0 0x7f13f81563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7400640 1 -- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8003a10 msgr2=0x7f13f809a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7400640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8003a10 0x7f13f809a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7400640 1 -- 172.21.15.97:0/1603153542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80704b0 con 0x7f13f8153c80 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f6000640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f809bfa0 0x7f13f81563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7400640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 0x7f13f8154100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f13f80b1920 tx=0x7f13f000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000de70 con 0x7f13f8153c80 2024-01-31T05:02:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000bb80 con 0x7f13f8153c80 2024-01-31T05:02:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007110 con 0x7f13f8153c80 2024-01-31T05:02:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f8070790 con 0x7f13f8153c80 2024-01-31T05:02:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.163+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f809c420 con 0x7f13f8153c80 2024-01-31T05:02:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.164+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f809c5e0 con 0x7f13f8153c80 2024-01-31T05:02:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.165+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f13f0007270 con 0x7f13f8153c80 2024-01-31T05:02:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.166+0000 7f13eea00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f13d006aed0 0x7f13d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.166+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f13f00897e0 con 0x7f13f8153c80 2024-01-31T05:02:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.166+0000 7f13f6a00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f13d006aed0 0x7f13d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.167+0000 7f13f6a00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f13d006aed0 0x7f13d006d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13e8036000 tx=0x7f13e8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.170+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f13f00543f0 con 0x7f13f8153c80 2024-01-31T05:02:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.363+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13f8070cb0 con 0x7f13f8153c80 2024-01-31T05:02:03.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.364+0000 7f13eea00640 1 -- 172.21.15.97:0/1603153542 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13f8070cb0 con 0x7f13f8153c80 2024-01-31T05:02:03.364 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f13d006aed0 msgr2=0x7f13d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f13d006aed0 0x7f13d006d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f13e8036000 tx=0x7f13e8030000 comp rx=0 tx=0).stop 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 msgr2=0x7f13f8154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 0x7f13f8154100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f13f80b1920 tx=0x7f13f000b4a0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 shutdown_connections 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f13d006aed0 0x7f13d006d390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13f8153c80 0x7f13f8154100 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13f809bfa0 0x7f13f81563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 --2- 172.21.15.97:0/1603153542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d8003a10 0x7f13f809a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 >> 172.21.15.97:0/1603153542 conn(0x7f13f806ad40 msgr2=0x7f13f806c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 shutdown_connections 2024-01-31T05:02:03.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.367+0000 7f13f7e00640 1 -- 172.21.15.97:0/1603153542 wait complete. 2024-01-31T05:02:03.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:03.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.602+0000 7f4fc3400640 1 Processor -- start 2024-01-31T05:02:03.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.602+0000 7f4fc3400640 1 -- start start 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc3400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fbc15b470 0x7f4fbc0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc0cbf20 0x7f4fbc0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc3400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc06cd20 con 0x7f4fbc0abcf0 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc3400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc0ce970 con 0x7f4fbc15b470 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc3400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15b850 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc0a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fbc15b470 0x7f4fbc0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc0a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4fbc15b470 0x7f4fbc0722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40148/0 (socket says 172.21.15.97:40148) 2024-01-31T05:02:03.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.604+0000 7f4fc0a00640 1 -- 172.21.15.97:0/1462166391 learned_addr learned my addr 172.21.15.97:0/1462166391 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.605+0000 7f4fc1400640 1 --2- 172.21.15.97:0/1462166391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc0cbf20 0x7f4fbc0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.605+0000 7f4fc0a00640 1 -- 172.21.15.97:0/1462166391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc0cbf20 msgr2=0x7f4fbc0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.605+0000 7f4fc0a00640 1 --2- 172.21.15.97:0/1462166391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc0cbf20 0x7f4fbc0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.605+0000 7f4fc0a00640 1 -- 172.21.15.97:0/1462166391 >> v1:172.21.15.97:6789/0 conn(0x7f4fbc0abcf0 legacy=0x7f4fbc0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.605+0000 7f4fbb400640 1 -- 172.21.15.97:0/1462166391 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098466523 0 0) 0x7f4fbc06cd20 con 0x7f4fbc0abcf0 2024-01-31T05:02:03.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.605+0000 7f4fc0a00640 1 -- 172.21.15.97:0/1462166391 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0ceb60 con 0x7f4fbc15b470 2024-01-31T05:02:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.606+0000 7f4fc0a00640 1 --2- 172.21.15.97:0/1462166391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc15b470 0x7f4fbc0722c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009880 tx=0x7f4fac02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4ffedba88c0861 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.606+0000 7f4fbb400640 1 -- 172.21.15.97:0/1462166391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac004230 con 0x7f4fbc15b470 2024-01-31T05:02:03.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.606+0000 7f4fbb400640 1 -- 172.21.15.97:0/1462166391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac004390 con 0x7f4fbc15b470 2024-01-31T05:02:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.606+0000 7f4fbb400640 1 -- 172.21.15.97:0/1462166391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac003810 con 0x7f4fbc15b470 2024-01-31T05:02:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 -- 172.21.15.97:0/1462166391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc15b470 msgr2=0x7f4fbc0722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 --2- 172.21.15.97:0/1462166391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc15b470 0x7f4fbc0722c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009880 tx=0x7f4fac02b960 comp rx=0 tx=0).stop 2024-01-31T05:02:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 -- 172.21.15.97:0/1462166391 shutdown_connections 2024-01-31T05:02:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 --2- 172.21.15.97:0/1462166391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc0cbf20 0x7f4fbc0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 --2- 172.21.15.97:0/1462166391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc15b470 0x7f4fbc0722c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 -- 172.21.15.97:0/1462166391 >> 172.21.15.97:0/1462166391 conn(0x7f4fbc0b0a10 msgr2=0x7f4fbc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:03.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 -- 172.21.15.97:0/1462166391 shutdown_connections 2024-01-31T05:02:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.607+0000 7f4fc3400640 1 -- 172.21.15.97:0/1462166391 wait complete. 2024-01-31T05:02:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.609+0000 7f4fc3400640 1 Processor -- start 2024-01-31T05:02:03.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.609+0000 7f4fc3400640 1 -- start start 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc0abcf0 0x7f4fbc156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 0x7f4fbc0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fc3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc15b470 0x7f4fbc1f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fc3400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1f6880 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fc3400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cb570 con 0x7f4fbc0abcf0 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fc3400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc1f6510 con 0x7f4fbc15b470 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fbbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 0x7f4fbc0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fbbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 0x7f4fbc0b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40156/0 (socket says 172.21.15.97:40156) 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.610+0000 7f4fbbe00640 1 -- 172.21.15.97:0/3660293911 learned_addr learned my addr 172.21.15.97:0/3660293911 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fc1400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc15b470 0x7f4fbc1f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fc0a00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc0abcf0 0x7f4fbc156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fbbe00640 1 -- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc15b470 msgr2=0x7f4fbc1f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fbbe00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc15b470 0x7f4fbc1f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fbbe00640 1 -- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc0abcf0 msgr2=0x7f4fbc156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fbbe00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc0abcf0 0x7f4fbc156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fbbe00640 1 -- 172.21.15.97:0/3660293911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac009460 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fbbe00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 0x7f4fbc0b3a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c460 tx=0x7f4fa400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4005480 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa40035f0 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa40039a0 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.611+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc0c45f0 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.612+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc0c4bb0 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.613+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc06d190 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.614+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4fa4003b00 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.614+0000 7f4fb8c00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4f9406af00 0x7f4f9406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.614+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4fa4089510 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.615+0000 7f4fc0a00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4f9406af00 0x7f4f9406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.615+0000 7f4fc0a00640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4f9406af00 0x7f4f9406d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fac035000 tx=0x7f4fac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.619+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4fbc06d190 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.806+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fbc06d190 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.807+0000 7f4fb8c00640 1 -- 172.21.15.97:0/3660293911 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fbc06d190 con 0x7f4fbc0cbf20 2024-01-31T05:02:03.806 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4f9406af00 msgr2=0x7f4f9406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4f9406af00 0x7f4f9406d3c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fac035000 tx=0x7f4fac008030 comp rx=0 tx=0).stop 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 msgr2=0x7f4fbc0b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 0x7f4fbc0b3a40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400c460 tx=0x7f4fa400c920 comp rx=0 tx=0).stop 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 shutdown_connections 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fbc15b470 0x7f4fbc1f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4f9406af00 0x7f4f9406d3c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cbf20 0x7f4fbc0b3a40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 --2- 172.21.15.97:0/3660293911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc0abcf0 0x7f4fbc156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.809+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 >> 172.21.15.97:0/3660293911 conn(0x7f4fbc0b0a10 msgr2=0x7f4fbc1583e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.810+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 shutdown_connections 2024-01-31T05:02:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:03.810+0000 7f4fc3400640 1 -- 172.21.15.97:0/3660293911 wait complete. 2024-01-31T05:02:03.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:02:04.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.025+0000 7feaa0c00640 1 Processor -- start 2024-01-31T05:02:04.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.025+0000 7feaa0c00640 1 -- start start 2024-01-31T05:02:04.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.026+0000 7feaa0c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea9c0abcf0 0x7fea9c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.026+0000 7feaa0c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c06cd20 con 0x7fea9c15b470 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.026+0000 7feaa0c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c0ceb60 con 0x7fea9c0cbf20 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.026+0000 7feaa0c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c15b850 con 0x7fea9c0abcf0 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea9a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea9c0abcf0 0x7fea9c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea9a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fea9c0abcf0 0x7fea9c0722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40164/0 (socket says 172.21.15.97:40164) 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea9a800640 1 -- 172.21.15.97:0/3835139874 learned_addr learned my addr 172.21.15.97:0/3835139874 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea99e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fea9c15b470 0x7fea9c0cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38772/0 (socket says 172.21.15.97:38772) 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea9a800640 1 -- 172.21.15.97:0/3835139874 >> v1:172.21.15.188:6789/0 conn(0x7fea9c0cbf20 legacy=0x7fea9c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea9a800640 1 -- 172.21.15.97:0/3835139874 >> v1:172.21.15.97:6789/0 conn(0x7fea9c15b470 legacy=0x7fea9c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.027+0000 7fea9a800640 1 -- 172.21.15.97:0/3835139874 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c06c3f0 con 0x7fea9c0abcf0 2024-01-31T05:02:04.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.028+0000 7fea9a800640 1 --2- 172.21.15.97:0/3835139874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0abcf0 0x7fea9c0722c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea94009e50 tx=0x7fea9402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f52f1505f9bf68 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.028+0000 7fea98a00640 1 -- 172.21.15.97:0/3835139874 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9402cb10 con 0x7fea9c0abcf0 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.028+0000 7fea98a00640 1 -- 172.21.15.97:0/3835139874 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9402cc70 con 0x7fea9c0abcf0 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.028+0000 7fea98a00640 1 -- 172.21.15.97:0/3835139874 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940343f0 con 0x7fea9c0abcf0 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 -- 172.21.15.97:0/3835139874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0abcf0 msgr2=0x7fea9c0722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 --2- 172.21.15.97:0/3835139874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0abcf0 0x7fea9c0722c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fea94009e50 tx=0x7fea9402bf70 comp rx=0 tx=0).stop 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 -- 172.21.15.97:0/3835139874 shutdown_connections 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 --2- 172.21.15.97:0/3835139874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0abcf0 0x7fea9c0722c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 -- 172.21.15.97:0/3835139874 >> 172.21.15.97:0/3835139874 conn(0x7fea9c0b0a10 msgr2=0x7fea9c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 -- 172.21.15.97:0/3835139874 shutdown_connections 2024-01-31T05:02:04.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.029+0000 7feaa0c00640 1 -- 172.21.15.97:0/3835139874 wait complete. 2024-01-31T05:02:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.031+0000 7feaa0c00640 1 Processor -- start 2024-01-31T05:02:04.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.031+0000 7feaa0c00640 1 -- start start 2024-01-31T05:02:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7feaa0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9c0abcf0 0x7fea9c0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7feaa0c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 0x7fea9c15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7feaa0c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9c15b470 0x7fea9c0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7feaa0c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c15cd10 con 0x7fea9c0cbf20 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7feaa0c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c15bed0 con 0x7fea9c15b470 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7feaa0c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c15c100 con 0x7fea9c0abcf0 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 0x7fea9c15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 0x7fea9c15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40176/0 (socket says 172.21.15.97:40176) 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 -- 172.21.15.97:0/4079779002 learned_addr learned my addr 172.21.15.97:0/4079779002 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 -- 172.21.15.97:0/4079779002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9c0abcf0 msgr2=0x7fea9c0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9c0abcf0 0x7fea9c0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 -- 172.21.15.97:0/4079779002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9c15b470 msgr2=0x7fea9c0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9c15b470 0x7fea9c0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 -- 172.21.15.97:0/4079779002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94009b30 con 0x7fea9c0cbf20 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.032+0000 7fea99400640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 0x7fea9c15b990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c430 tx=0x7fea8400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.033+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011790 con 0x7fea9c0cbf20 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.033+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea840118f0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.033+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011ce0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.033+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c15c3e0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.033+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c15c9a0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.034+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c0ae0e0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.035+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fea8401c460 con 0x7fea9c0cbf20 2024-01-31T05:02:04.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.035+0000 7fea92000640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea7406aed0 0x7fea7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.036+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fea84089950 con 0x7fea9c0cbf20 2024-01-31T05:02:04.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.036+0000 7fea99e00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea7406aed0 0x7fea7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.036+0000 7fea99e00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea7406aed0 0x7fea7406d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea8c001020 tx=0x7fea8c007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:04.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.046+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fea840544d0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.226+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fea9c15cec0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.227+0000 7fea92000640 1 -- 172.21.15.97:0/4079779002 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fea9c15cec0 con 0x7fea9c0cbf20 2024-01-31T05:02:04.226 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea7406aed0 msgr2=0x7fea7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea7406aed0 0x7fea7406d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea8c001020 tx=0x7fea8c007930 comp rx=0 tx=0).stop 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 msgr2=0x7fea9c15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 0x7fea9c15b990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea8400c430 tx=0x7fea8400c8f0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 shutdown_connections 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9c15b470 0x7fea9c0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fea7406aed0 0x7fea7406d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea9c0cbf20 0x7fea9c15b990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 --2- 172.21.15.97:0/4079779002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9c0abcf0 0x7fea9c0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 >> 172.21.15.97:0/4079779002 conn(0x7fea9c0b0a10 msgr2=0x7fea9c158610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.229+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 shutdown_connections 2024-01-31T05:02:04.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.230+0000 7feaa0c00640 1 -- 172.21.15.97:0/4079779002 wait complete. 2024-01-31T05:02:04.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.451+0000 7f2c6a000640 1 Processor -- start 2024-01-31T05:02:04.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.452+0000 7f2c6a000640 1 -- start start 2024-01-31T05:02:04.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.453+0000 7f2c6a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c640ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.453+0000 7f2c6a000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c64161020 0x7f2c641614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.453+0000 7f2c6a000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6415db40 con 0x7f2c6406d580 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.453+0000 7f2c6a000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6415e7c0 con 0x7f2c64161020 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.453+0000 7f2c6a000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6415de20 con 0x7f2c6415d690 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2c6406d580 0x7f2c6406d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38784/0 (socket says 172.21.15.97:38784) 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63400640 1 -- 172.21.15.97:0/4049695274 learned_addr learned my addr 172.21.15.97:0/4049695274 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63e00640 1 --2- 172.21.15.97:0/4049695274 >> v2:172.21.15.97:3300/0 conn(0x7f2c64161020 0x7f2c641614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c5be00640 1 --2- 172.21.15.97:0/4049695274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c640ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c62a00640 1 -- 172.21.15.97:0/4049695274 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553166372 0 0) 0x7f2c6415db40 con 0x7f2c6406d580 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c62a00640 1 -- 172.21.15.97:0/4049695274 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c44003520 con 0x7f2c6406d580 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63e00640 1 -- 172.21.15.97:0/4049695274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 msgr2=0x7f2c640ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63e00640 1 --2- 172.21.15.97:0/4049695274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c640ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63e00640 1 -- 172.21.15.97:0/4049695274 >> v1:172.21.15.97:6789/0 conn(0x7f2c6406d580 legacy=0x7f2c6406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.454+0000 7f2c63e00640 1 -- 172.21.15.97:0/4049695274 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6415e100 con 0x7f2c64161020 2024-01-31T05:02:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.456+0000 7f2c63e00640 1 --2- 172.21.15.97:0/4049695274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c64161020 0x7f2c641614d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009d90 tx=0x7f2c5c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168a12d0ab8abc30 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.456+0000 7f2c62a00640 1 -- 172.21.15.97:0/4049695274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c02cc50 con 0x7f2c64161020 2024-01-31T05:02:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.456+0000 7f2c62a00640 1 -- 172.21.15.97:0/4049695274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c02cdb0 con 0x7f2c64161020 2024-01-31T05:02:04.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.456+0000 7f2c62a00640 1 -- 172.21.15.97:0/4049695274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c034660 con 0x7f2c64161020 2024-01-31T05:02:04.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 -- 172.21.15.97:0/4049695274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c64161020 msgr2=0x7f2c641614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 --2- 172.21.15.97:0/4049695274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c64161020 0x7f2c641614d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009d90 tx=0x7f2c5c02bea0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 -- 172.21.15.97:0/4049695274 shutdown_connections 2024-01-31T05:02:04.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 --2- 172.21.15.97:0/4049695274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c64161020 0x7f2c641614d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 --2- 172.21.15.97:0/4049695274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c640ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 -- 172.21.15.97:0/4049695274 >> 172.21.15.97:0/4049695274 conn(0x7f2c640b0a10 msgr2=0x7f2c640b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 -- 172.21.15.97:0/4049695274 shutdown_connections 2024-01-31T05:02:04.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.457+0000 7f2c6a000640 1 -- 172.21.15.97:0/4049695274 wait complete. 2024-01-31T05:02:04.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.459+0000 7f2c6a000640 1 Processor -- start 2024-01-31T05:02:04.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.459+0000 7f2c6a000640 1 -- start start 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.460+0000 7f2c6a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 0x7f2c641498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.460+0000 7f2c6a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c641f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c6a000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c64161020 0x7f2c6415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c6a000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c641f4620 con 0x7f2c6406d580 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c6a000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64157a30 con 0x7f2c64161020 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c6a000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64157d10 con 0x7f2c6415d690 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 0x7f2c641498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 0x7f2c641498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40184/0 (socket says 172.21.15.97:40184) 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 -- 172.21.15.97:0/2748890919 learned_addr learned my addr 172.21.15.97:0/2748890919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c5be00640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c641f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 -- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 msgr2=0x7f2c641f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c641f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 -- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c64161020 msgr2=0x7f2c6415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c64161020 0x7f2c6415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.461+0000 7f2c63400640 1 -- 172.21.15.97:0/2748890919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009a70 con 0x7f2c6406d580 2024-01-31T05:02:04.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.462+0000 7f2c63400640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 0x7f2c641498c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c400 tx=0x7f2c5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:04.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.462+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c540074e0 con 0x7f2c6406d580 2024-01-31T05:02:04.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.462+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c540057a0 con 0x7f2c6406d580 2024-01-31T05:02:04.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.462+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54007660 con 0x7f2c6406d580 2024-01-31T05:02:04.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.462+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64157ff0 con 0x7f2c6406d580 2024-01-31T05:02:04.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.462+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c641585b0 con 0x7f2c6406d580 2024-01-31T05:02:04.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.463+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c64067830 con 0x7f2c6406d580 2024-01-31T05:02:04.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.464+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2c54005b50 con 0x7f2c6406d580 2024-01-31T05:02:04.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.465+0000 7f2c5b400640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2c3c06aed0 0x7f2c3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:04.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.465+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2c54088fb0 con 0x7f2c6406d580 2024-01-31T05:02:04.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.465+0000 7f2c5be00640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2c3c06aed0 0x7f2c3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:04.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.466+0000 7f2c5be00640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2c3c06aed0 0x7f2c3c06d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c64157820 tx=0x7f2c4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:04.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.469+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2c540bc190 con 0x7f2c6406d580 2024-01-31T05:02:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.680+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2c640cc6c0 con 0x7f2c6406d580 2024-01-31T05:02:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.681+0000 7f2c5b400640 1 -- 172.21.15.97:0/2748890919 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2c54017020 con 0x7f2c6406d580 2024-01-31T05:02:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2c3c06aed0 msgr2=0x7f2c3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2c3c06aed0 0x7f2c3c06d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2c64157820 tx=0x7f2c4c008040 comp rx=0 tx=0).stop 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 msgr2=0x7f2c641498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 0x7f2c641498c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c400 tx=0x7f2c5400c8c0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 shutdown_connections 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c64161020 0x7f2c6415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c6415d690 0x7f2c641f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2c3c06aed0 0x7f2c3c06d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 --2- 172.21.15.97:0/2748890919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6406d580 0x7f2c641498c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 >> 172.21.15.97:0/2748890919 conn(0x7f2c640b0a10 msgr2=0x7f2c64073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 shutdown_connections 2024-01-31T05:02:04.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:04.684+0000 7f2c6a000640 1 -- 172.21.15.97:0/2748890919 wait complete. 2024-01-31T05:02:04.704 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:02:04.704 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:04.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:05.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.002+0000 7ff729600640 1 Processor -- start 2024-01-31T05:02:05.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.003+0000 7ff729600640 1 -- start start 2024-01-31T05:02:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.004+0000 7ff729600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7240ce000 0x7ff7240b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.004+0000 7ff729600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72406b570 con 0x7ff724160e00 2024-01-31T05:02:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.004+0000 7ff729600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7240cb6c0 con 0x7ff7240d1b90 2024-01-31T05:02:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.004+0000 7ff729600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7240d1fe0 con 0x7ff7240ce000 2024-01-31T05:02:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.004+0000 7ff722a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff724160e00 0x7ff7240cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47100/0 (socket says 172.21.15.97:47100) 2024-01-31T05:02:05.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.004+0000 7ff722a00640 1 -- 172.21.15.97:0/432186273 learned_addr learned my addr 172.21.15.97:0/432186273 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097746819 0 0) 0x7ff72406b570 con 0x7ff724160e00 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff704003520 con 0x7ff724160e00 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff722000640 1 --2- 172.21.15.97:0/432186273 >> v2:172.21.15.188:3300/0 conn(0x7ff7240ce000 0x7ff7240b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2717429299 0 0) 0x7ff704003520 con 0x7ff724160e00 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 >> v2:172.21.15.188:3300/0 conn(0x7ff7240ce000 msgr2=0x7ff7240b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 --2- 172.21.15.97:0/432186273 >> v2:172.21.15.188:3300/0 conn(0x7ff7240ce000 0x7ff7240b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 >> v1:172.21.15.188:6789/0 conn(0x7ff7240d1b90 legacy=0x7ff7240cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7241611e0 con 0x7ff724160e00 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff714001670 con 0x7ff724160e00 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 >> v1:172.21.15.97:6789/0 conn(0x7ff724160e00 legacy=0x7ff7240cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff704003970 0x7ff704003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7040047c0 0x7ff704004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7040052a0 0x7ff70400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240d1fe0 con 0x7ff704003970 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70400c1a0 con 0x7ff7040052a0 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.005+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70400c360 con 0x7ff7040047c0 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722000640 1 --2- 172.21.15.97:0/432186273 >> v2:172.21.15.188:3300/0 conn(0x7ff7240ce000 0x7ff7240b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722a00640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff704003970 0x7ff704003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff723400640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7040052a0 0x7ff70400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722a00640 1 -- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7040047c0 msgr2=0x7ff704004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722a00640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7040047c0 0x7ff704004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722a00640 1 -- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7040052a0 msgr2=0x7ff70400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722a00640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7040052a0 0x7ff70400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff722a00640 1 -- 172.21.15.97:0/432186273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff704003520 con 0x7ff704003970 2024-01-31T05:02:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff723400640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7040052a0 0x7ff70400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:05.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.006+0000 7ff723400640 1 -- 172.21.15.97:0/432186273 reap_dead start 2024-01-31T05:02:05.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.008+0000 7ff722a00640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff704003970 0x7ff704003d70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff714001eb0 tx=0x7ff7040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b87298f5f5838ef server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.008+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714005ad0 con 0x7ff704003970 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.008+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714005c50 con 0x7ff704003970 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.008+0000 7ff721600640 1 -- 172.21.15.97:0/432186273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71402c8b0 con 0x7ff704003970 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 -- 172.21.15.97:0/432186273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff704003970 msgr2=0x7ff704003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff704003970 0x7ff704003d70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff714001eb0 tx=0x7ff7040057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 -- 172.21.15.97:0/432186273 shutdown_connections 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 --2- 172.21.15.97:0/432186273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff704003970 0x7ff704003d70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 -- 172.21.15.97:0/432186273 >> 172.21.15.97:0/432186273 conn(0x7ff7240d1070 msgr2=0x7ff7240d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 -- 172.21.15.97:0/432186273 shutdown_connections 2024-01-31T05:02:05.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.009+0000 7ff729600640 1 -- 172.21.15.97:0/432186273 wait complete. 2024-01-31T05:02:05.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.010+0000 7ff729600640 1 Processor -- start 2024-01-31T05:02:05.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.011+0000 7ff729600640 1 -- start start 2024-01-31T05:02:05.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff729600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff704003970 0x7ff72415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff729600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7240c4ba0 0x7ff7240cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff729600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 0x7ff724146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff729600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240c5310 con 0x7ff724146180 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff729600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241593c0 con 0x7ff7240c4ba0 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff729600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff724153ad0 con 0x7ff704003970 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 0x7ff724146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 0x7ff724146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45554/0 (socket says 172.21.15.97:45554) 2024-01-31T05:02:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 -- 172.21.15.97:0/2797806702 learned_addr learned my addr 172.21.15.97:0/2797806702 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff722a00640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff704003970 0x7ff72415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff722000640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7240c4ba0 0x7ff7240cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 -- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff704003970 msgr2=0x7ff72415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff704003970 0x7ff72415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 -- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7240c4ba0 msgr2=0x7ff7240cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7240c4ba0 0x7ff7240cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.012+0000 7ff723400640 1 -- 172.21.15.97:0/2797806702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72415a5d0 con 0x7ff724146180 2024-01-31T05:02:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.013+0000 7ff723400640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 0x7ff724146630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff724071c40 tx=0x7ff71c00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.013+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c018070 con 0x7ff724146180 2024-01-31T05:02:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.013+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c006b50 con 0x7ff724146180 2024-01-31T05:02:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.013+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7240c5030 con 0x7ff724146180 2024-01-31T05:02:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.013+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c012600 con 0x7ff724146180 2024-01-31T05:02:05.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.013+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff724159890 con 0x7ff724146180 2024-01-31T05:02:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.015+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7240677c0 con 0x7ff724146180 2024-01-31T05:02:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.015+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff71c012760 con 0x7ff724146180 2024-01-31T05:02:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.016+0000 7ff71aa00640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff6fc06af00 0x7ff6fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.016+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff71c089000 con 0x7ff724146180 2024-01-31T05:02:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.016+0000 7ff722a00640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff6fc06af00 0x7ff6fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.017+0000 7ff722a00640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff6fc06af00 0x7ff6fc06d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff714034000 tx=0x7ff714009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.020+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff71c053af0 con 0x7ff724146180 2024-01-31T05:02:05.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.214+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7240b6ef0 con 0x7ff724146180 2024-01-31T05:02:05.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.217+0000 7ff71aa00640 1 -- 172.21.15.97:0/2797806702 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7ff71c057110 con 0x7ff724146180 2024-01-31T05:02:05.217 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:05.218 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff6fc06af00 msgr2=0x7ff6fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff6fc06af00 0x7ff6fc06d3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff714034000 tx=0x7ff714009030 comp rx=0 tx=0).stop 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 msgr2=0x7ff724146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 0x7ff724146630 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff724071c40 tx=0x7ff71c00d900 comp rx=0 tx=0).stop 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 shutdown_connections 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff6fc06af00 0x7ff6fc06d3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff724146180 0x7ff724146630 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7240c4ba0 0x7ff7240cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 --2- 172.21.15.97:0/2797806702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff704003970 0x7ff72415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 >> 172.21.15.97:0/2797806702 conn(0x7ff7240d1070 msgr2=0x7ff72415ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 shutdown_connections 2024-01-31T05:02:05.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.220+0000 7ff729600640 1 -- 172.21.15.97:0/2797806702 wait complete. 2024-01-31T05:02:05.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:02:05.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:05.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.451+0000 7f17db400640 1 Processor -- start 2024-01-31T05:02:05.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.452+0000 7f17db400640 1 -- start start 2024-01-31T05:02:05.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.453+0000 7f17db400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 0x7f17d40cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.453+0000 7f17db400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d40d02f0 con 0x7f17d415d3d0 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.453+0000 7f17db400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d415d7b0 con 0x7f17d40b2a20 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.453+0000 7f17db400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40d0e10 con 0x7f17d40cf990 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.453+0000 7f17da000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f17d415d3d0 0x7f17d40ce770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47110/0 (socket says 172.21.15.97:47110) 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.453+0000 7f17da000640 1 -- 172.21.15.97:0/40469581 learned_addr learned my addr 172.21.15.97:0/40469581 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d9600640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 0x7f17d40cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799511863 0 0) 0x7f17d40d02f0 con 0x7f17d415d3d0 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b4003520 con 0x7f17d415d3d0 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1766177864 0 0) 0x7f17b4003520 con 0x7f17d415d3d0 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 msgr2=0x7f17d40cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d8c00640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 0x7f17d40cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.454+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 >> v1:172.21.15.188:6789/0 conn(0x7f17d40b2a20 legacy=0x7f17d40b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d40d10b0 con 0x7f17d415d3d0 2024-01-31T05:02:05.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f17cc0019f0 con 0x7f17d415d3d0 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 >> v1:172.21.15.97:6789/0 conn(0x7f17d415d3d0 legacy=0x7f17d40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b4003a10 0x7f17b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 0x7f17b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4004d00 0x7f17b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40d0e10 con 0x7f17b4003a10 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b400ba30 con 0x7f17b4004d00 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b400bbf0 con 0x7f17d40cf990 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17da000640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b4003a10 0x7f17b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17d9600640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 0x7f17b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17daa00640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4004d00 0x7f17b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17da000640 1 -- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 msgr2=0x7f17b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17da000640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cf990 0x7f17b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.455+0000 7f17da000640 1 -- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4004d00 msgr2=0x7f17b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.456+0000 7f17da000640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4004d00 0x7f17b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.456+0000 7f17da000640 1 -- 172.21.15.97:0/40469581 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4003520 con 0x7f17b4003a10 2024-01-31T05:02:05.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.456+0000 7f17daa00640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4004d00 0x7f17b4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.457+0000 7f17da000640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b4003a10 0x7f17b4003df0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00a810 tx=0x7f17b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edf51b9d36311b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.457+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc034630 con 0x7f17b4003a10 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.457+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc02f070 con 0x7f17b4003a10 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.457+0000 7f17d8c00640 1 -- 172.21.15.97:0/40469581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0349d0 con 0x7f17b4003a10 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17db400640 1 -- 172.21.15.97:0/40469581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b4003a10 msgr2=0x7f17b4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17db400640 1 --2- 172.21.15.97:0/40469581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17b4003a10 0x7f17b4003df0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00a810 tx=0x7f17b40057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17daa00640 1 -- 172.21.15.97:0/40469581 reap_dead start 2024-01-31T05:02:05.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17db400640 1 -- 172.21.15.97:0/40469581 shutdown_connections 2024-01-31T05:02:05.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17db400640 1 -- 172.21.15.97:0/40469581 >> 172.21.15.97:0/40469581 conn(0x7f17d40d30c0 msgr2=0x7f17d40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:05.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17db400640 1 -- 172.21.15.97:0/40469581 shutdown_connections 2024-01-31T05:02:05.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.458+0000 7f17db400640 1 -- 172.21.15.97:0/40469581 wait complete. 2024-01-31T05:02:05.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.460+0000 7f17db400640 1 Processor -- start 2024-01-31T05:02:05.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.460+0000 7f17db400640 1 -- start start 2024-01-31T05:02:05.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17db400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4003a10 0x7f17d40c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17db400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 0x7f17d415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17db400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cbfc0 0x7f17d40c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17db400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41603b0 con 0x7f17d4156000 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17db400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d415f9e0 con 0x7f17b4003a10 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17db400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d415edd0 con 0x7f17d40cbfc0 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17d9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 0x7f17d415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17d9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 0x7f17d415fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45582/0 (socket says 172.21.15.97:45582) 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.461+0000 7f17d9600640 1 -- 172.21.15.97:0/599990510 learned_addr learned my addr 172.21.15.97:0/599990510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17daa00640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cbfc0 0x7f17d40c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17da000640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4003a10 0x7f17d40c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d9600640 1 -- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cbfc0 msgr2=0x7f17d40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d9600640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cbfc0 0x7f17d40c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d9600640 1 -- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4003a10 msgr2=0x7f17d40c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d9600640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4003a10 0x7f17d40c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d9600640 1 -- 172.21.15.97:0/599990510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d415f0b0 con 0x7f17d4156000 2024-01-31T05:02:05.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d9600640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 0x7f17d415fe70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f17bc011600 tx=0x7f17bc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00f920 con 0x7f17d4156000 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00fa80 con 0x7f17d4156000 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00fdf0 con 0x7f17d4156000 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.462+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d415f390 con 0x7f17d4156000 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.463+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d40cc440 con 0x7f17d4156000 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.464+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d40676b0 con 0x7f17d4156000 2024-01-31T05:02:05.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.465+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f17bc015460 con 0x7f17d4156000 2024-01-31T05:02:05.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.465+0000 7f17d2000640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f17ac06aed0 0x7f17ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.465+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f17bc08a1f0 con 0x7f17d4156000 2024-01-31T05:02:05.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.466+0000 7f17da000640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f17ac06aed0 0x7f17ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.466+0000 7f17da000640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f17ac06aed0 0x7f17ac06d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17cc034000 tx=0x7f17cc02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.470+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f17bc054cd0 con 0x7f17d4156000 2024-01-31T05:02:05.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.662+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17d4072ae0 con 0x7f17d4156000 2024-01-31T05:02:05.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.663+0000 7f17d2000640 1 -- 172.21.15.97:0/599990510 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17bc0582f0 con 0x7f17d4156000 2024-01-31T05:02:05.663 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f17ac06aed0 msgr2=0x7f17ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f17ac06aed0 0x7f17ac06d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17cc034000 tx=0x7f17cc02f000 comp rx=0 tx=0).stop 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 msgr2=0x7f17d415fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 0x7f17d415fe70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f17bc011600 tx=0x7f17bc011ac0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 shutdown_connections 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f17d40cbfc0 0x7f17d40c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f17ac06aed0 0x7f17ac06d390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17d4156000 0x7f17d415fe70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 --2- 172.21.15.97:0/599990510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17b4003a10 0x7f17d40c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 >> 172.21.15.97:0/599990510 conn(0x7f17d40d30c0 msgr2=0x7f17d40afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:05.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 shutdown_connections 2024-01-31T05:02:05.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.667+0000 7f17db400640 1 -- 172.21.15.97:0/599990510 wait complete. 2024-01-31T05:02:05.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:05.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.894+0000 7ff606e00640 1 Processor -- start 2024-01-31T05:02:05.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.894+0000 7ff606e00640 1 -- start start 2024-01-31T05:02:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff606e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff60015b470 0x7ff6000722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff606e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 0x7ff6000ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff606e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60006cd20 con 0x7ff6000abcf0 2024-01-31T05:02:05.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff606e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000ce970 con 0x7ff60015b470 2024-01-31T05:02:05.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff606e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015b850 con 0x7ff6000cbf20 2024-01-31T05:02:05.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff5ff400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6000abcf0 0x7ff6000cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47120/0 (socket says 172.21.15.97:47120) 2024-01-31T05:02:05.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff5ff400640 1 -- 172.21.15.97:0/3764459716 learned_addr learned my addr 172.21.15.97:0/3764459716 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:05.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff604e00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 0x7ff6000ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff5ffe00640 1 --2- 172.21.15.97:0/3764459716 >> v2:172.21.15.188:3300/0 conn(0x7ff60015b470 0x7ff6000722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054919963 0 0) 0x7ff60006cd20 con 0x7ff6000abcf0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.896+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e0003520 con 0x7ff6000abcf0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305696906 0 0) 0x7ff5e0003520 con 0x7ff6000abcf0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 msgr2=0x7ff6000ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 0x7ff6000ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 >> v2:172.21.15.188:3300/0 conn(0x7ff60015b470 msgr2=0x7ff6000722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 --2- 172.21.15.97:0/3764459716 >> v2:172.21.15.188:3300/0 conn(0x7ff60015b470 0x7ff6000722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000ceb60 con 0x7ff6000abcf0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff5e8001670 con 0x7ff6000abcf0 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 >> v1:172.21.15.97:6789/0 conn(0x7ff6000abcf0 legacy=0x7ff6000cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e00039a0 0x7ff5e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 0x7ff5e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0004d00 0x7ff5e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015b850 con 0x7ff5e0004d00 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e000b980 con 0x7ff5e00039a0 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e000bac0 con 0x7ff6000cbf20 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff604e00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0004d00 0x7ff5e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.897+0000 7ff5ff400640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 0x7ff5e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.898+0000 7ff604e00640 1 -- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 msgr2=0x7ff5e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.898+0000 7ff604e00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000cbf20 0x7ff5e00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.898+0000 7ff604e00640 1 -- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e00039a0 msgr2=0x7ff5e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.898+0000 7ff604e00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e00039a0 0x7ff5e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.898+0000 7ff604e00640 1 -- 172.21.15.97:0/3764459716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0003520 con 0x7ff5e0004d00 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.899+0000 7ff604e00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0004d00 0x7ff5e0005180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6000cb140 tx=0x7ff5e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604003aa1462b81d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.899+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f802cb90 con 0x7ff5e0004d00 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.899+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800f040 con 0x7ff5e0004d00 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.899+0000 7ff5fea00640 1 -- 172.21.15.97:0/3764459716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8034440 con 0x7ff5e0004d00 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff606e00640 1 -- 172.21.15.97:0/3764459716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0004d00 msgr2=0x7ff5e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff606e00640 1 --2- 172.21.15.97:0/3764459716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0004d00 0x7ff5e0005180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6000cb140 tx=0x7ff5e00057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff604e00640 1 -- 172.21.15.97:0/3764459716 reap_dead start 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff606e00640 1 -- 172.21.15.97:0/3764459716 shutdown_connections 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff606e00640 1 -- 172.21.15.97:0/3764459716 >> 172.21.15.97:0/3764459716 conn(0x7ff6000b0a10 msgr2=0x7ff6000b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff606e00640 1 -- 172.21.15.97:0/3764459716 shutdown_connections 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.900+0000 7ff606e00640 1 -- 172.21.15.97:0/3764459716 wait complete. 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.901+0000 7ff606e00640 1 Processor -- start 2024-01-31T05:02:05.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.902+0000 7ff606e00640 1 -- start start 2024-01-31T05:02:05.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.902+0000 7ff606e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e0004d00 0x7ff60009a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.902+0000 7ff606e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 0x7ff60006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.902+0000 7ff606e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600152590 0x7ff600152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff606e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600152f40 con 0x7ff6001533c0 2024-01-31T05:02:05.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff606e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015bb50 con 0x7ff5e0004d00 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff606e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015be30 con 0x7ff600152590 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 0x7ff60006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 0x7ff60006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45610/0 (socket says 172.21.15.97:45610) 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 -- 172.21.15.97:0/3239947507 learned_addr learned my addr 172.21.15.97:0/3239947507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 -- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600152590 msgr2=0x7ff600152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600152590 0x7ff600152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ffe00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e0004d00 0x7ff60009a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 -- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e0004d00 msgr2=0x7ff60009a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e0004d00 0x7ff60009a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.903+0000 7ff5ff400640 1 -- 172.21.15.97:0/3239947507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60015c110 con 0x7ff6001533c0 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.904+0000 7ff5ff400640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 0x7ff60006d320 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d830 tx=0x7ff5e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.904+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80074e0 con 0x7ff6001533c0 2024-01-31T05:02:05.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.904+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e8007640 con 0x7ff6001533c0 2024-01-31T05:02:05.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.904+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e80049e0 con 0x7ff6001533c0 2024-01-31T05:02:05.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.904+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60015c3f0 con 0x7ff6001533c0 2024-01-31T05:02:05.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.904+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60009a910 con 0x7ff6001533c0 2024-01-31T05:02:05.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.905+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600067830 con 0x7ff6001533c0 2024-01-31T05:02:05.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.906+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff5e8004b40 con 0x7ff6001533c0 2024-01-31T05:02:05.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.907+0000 7ff5f7e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5d806aed0 0x7ff5d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:05.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.907+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff5e8089620 con 0x7ff6001533c0 2024-01-31T05:02:05.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.907+0000 7ff5ffe00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5d806aed0 0x7ff5d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:05.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.908+0000 7ff5ffe00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5d806aed0 0x7ff5d806d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00047b0 tx=0x7ff5f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:05.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:05.911+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff5e8054230 con 0x7ff6001533c0 2024-01-31T05:02:06.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.108+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff600068010 con 0x7ff6001533c0 2024-01-31T05:02:06.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.109+0000 7ff5f7e00640 1 -- 172.21.15.97:0/3239947507 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5e8057850 con 0x7ff6001533c0 2024-01-31T05:02:06.108 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:06.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5d806aed0 msgr2=0x7ff5d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5d806aed0 0x7ff5d806d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00047b0 tx=0x7ff5f0008040 comp rx=0 tx=0).stop 2024-01-31T05:02:06.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 msgr2=0x7ff60006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 0x7ff60006d320 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800d830 tx=0x7ff5e800dcf0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 shutdown_connections 2024-01-31T05:02:06.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff600152590 0x7ff600152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5d806aed0 0x7ff5d806d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6001533c0 0x7ff60006d320 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 --2- 172.21.15.97:0/3239947507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5e0004d00 0x7ff60009a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.111+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 >> 172.21.15.97:0/3239947507 conn(0x7ff6000b0a10 msgr2=0x7ff6000b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:06.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.112+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 shutdown_connections 2024-01-31T05:02:06.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.112+0000 7ff606e00640 1 -- 172.21.15.97:0/3239947507 wait complete. 2024-01-31T05:02:06.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:06.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.335+0000 7f8a37200640 1 Processor -- start 2024-01-31T05:02:06.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.335+0000 7f8a37200640 1 -- start start 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.336+0000 7f8a37200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a3806d210 0x7f8a3815af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a37200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8a38071860 0x7f8a3806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a37200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3806db30 con 0x7f8a3815da70 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a37200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a380714c0 con 0x7f8a3806d210 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a37200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a38070420 con 0x7f8a38071860 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a3806d210 0x7f8a3815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a3806d210 0x7f8a3815af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45616/0 (socket says 172.21.15.97:45616) 2024-01-31T05:02:06.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35e00640 1 -- 172.21.15.97:0/1706154887 learned_addr learned my addr 172.21.15.97:0/1706154887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a36800640 1 --1- 172.21.15.97:0/1706154887 >> v1:172.21.15.97:6789/0 conn(0x7f8a3815da70 0x7f8a3815dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47126/0 (socket says 172.21.15.97:47126) 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a2d200640 1 --2- 172.21.15.97:0/1706154887 >> v2:172.21.15.188:3300/0 conn(0x7f8a38071860 0x7f8a3806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35e00640 1 -- 172.21.15.97:0/1706154887 >> v2:172.21.15.188:3300/0 conn(0x7f8a38071860 msgr2=0x7f8a3806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35e00640 1 --2- 172.21.15.97:0/1706154887 >> v2:172.21.15.188:3300/0 conn(0x7f8a38071860 0x7f8a3806d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35400640 1 -- 172.21.15.97:0/1706154887 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119977729 0 0) 0x7f8a3806db30 con 0x7f8a3815da70 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.337+0000 7f8a35e00640 1 -- 172.21.15.97:0/1706154887 >> v1:172.21.15.97:6789/0 conn(0x7f8a3815da70 legacy=0x7f8a3815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.338+0000 7f8a35e00640 1 -- 172.21.15.97:0/1706154887 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38070700 con 0x7f8a3806d210 2024-01-31T05:02:06.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.339+0000 7f8a35e00640 1 --2- 172.21.15.97:0/1706154887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3806d210 0x7f8a3815af90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8a380b1380 tx=0x7f8a2802e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dd9db9f9c786e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:06.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.339+0000 7f8a35400640 1 -- 172.21.15.97:0/1706154887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2800b0d0 con 0x7f8a3806d210 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.339+0000 7f8a35400640 1 -- 172.21.15.97:0/1706154887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2800b230 con 0x7f8a3806d210 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.339+0000 7f8a35400640 1 -- 172.21.15.97:0/1706154887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28009600 con 0x7f8a3806d210 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.339+0000 7f8a37200640 1 -- 172.21.15.97:0/1706154887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3806d210 msgr2=0x7f8a3815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.339+0000 7f8a37200640 1 --2- 172.21.15.97:0/1706154887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3806d210 0x7f8a3815af90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8a380b1380 tx=0x7f8a2802e180 comp rx=0 tx=0).stop 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.340+0000 7f8a37200640 1 -- 172.21.15.97:0/1706154887 shutdown_connections 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.340+0000 7f8a37200640 1 --2- 172.21.15.97:0/1706154887 >> v2:172.21.15.188:3300/0 conn(0x7f8a38071860 0x7f8a3806d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.340+0000 7f8a37200640 1 --2- 172.21.15.97:0/1706154887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3806d210 0x7f8a3815af90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.340+0000 7f8a37200640 1 -- 172.21.15.97:0/1706154887 >> 172.21.15.97:0/1706154887 conn(0x7f8a3806ad40 msgr2=0x7f8a380b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.340+0000 7f8a37200640 1 -- 172.21.15.97:0/1706154887 shutdown_connections 2024-01-31T05:02:06.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.340+0000 7f8a37200640 1 -- 172.21.15.97:0/1706154887 wait complete. 2024-01-31T05:02:06.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.341+0000 7f8a37200640 1 Processor -- start 2024-01-31T05:02:06.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 -- start start 2024-01-31T05:02:06.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3806d210 0x7f8a381496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a38071860 0x7f8a380cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 0x7f8a38157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a38149c20 con 0x7f8a3815da70 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380cdb50 con 0x7f8a38071860 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.342+0000 7f8a37200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380cc950 con 0x7f8a3806d210 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 0x7f8a38157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 0x7f8a38157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45618/0 (socket says 172.21.15.97:45618) 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 -- 172.21.15.97:0/3329055535 learned_addr learned my addr 172.21.15.97:0/3329055535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 -- 172.21.15.97:0/3329055535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3806d210 msgr2=0x7f8a381496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3806d210 0x7f8a381496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 -- 172.21.15.97:0/3329055535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a38071860 msgr2=0x7f8a380cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a38071860 0x7f8a380cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 -- 172.21.15.97:0/3329055535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2800bd40 con 0x7f8a3815da70 2024-01-31T05:02:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.343+0000 7f8a36800640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 0x7f8a38157000 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c480 tx=0x7f8a3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:06.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.344+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30007480 con 0x7f8a3815da70 2024-01-31T05:02:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.344+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a30007600 con 0x7f8a3815da70 2024-01-31T05:02:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.344+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a380cd870 con 0x7f8a3815da70 2024-01-31T05:02:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.344+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30004a20 con 0x7f8a3815da70 2024-01-31T05:02:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.344+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a380cbc10 con 0x7f8a3815da70 2024-01-31T05:02:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.345+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a380676b0 con 0x7f8a3815da70 2024-01-31T05:02:06.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.346+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8a30004b80 con 0x7f8a3815da70 2024-01-31T05:02:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.347+0000 7f8a2ea00640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a1006aed0 0x7f8a1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.347+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8a30088e40 con 0x7f8a3815da70 2024-01-31T05:02:06.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.347+0000 7f8a35e00640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a1006aed0 0x7f8a1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.348+0000 7f8a35e00640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a1006aed0 0x7f8a1006d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a28037000 tx=0x7f8a28030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:06.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.351+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8a300539b0 con 0x7f8a3815da70 2024-01-31T05:02:06.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.543+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a380b34d0 con 0x7f8a3815da70 2024-01-31T05:02:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.544+0000 7f8a2ea00640 1 -- 172.21.15.97:0/3329055535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a300180a0 con 0x7f8a3815da70 2024-01-31T05:02:06.544 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a1006aed0 msgr2=0x7f8a1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a1006aed0 0x7f8a1006d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a28037000 tx=0x7f8a28030000 comp rx=0 tx=0).stop 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 msgr2=0x7f8a38157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 0x7f8a38157000 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8a3000c480 tx=0x7f8a3000c940 comp rx=0 tx=0).stop 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 shutdown_connections 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8a1006aed0 0x7f8a1006d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a3815da70 0x7f8a38157000 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a38071860 0x7f8a380cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 --2- 172.21.15.97:0/3329055535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3806d210 0x7f8a381496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 >> 172.21.15.97:0/3329055535 conn(0x7f8a3806ad40 msgr2=0x7f8a3815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 shutdown_connections 2024-01-31T05:02:06.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.547+0000 7f8a37200640 1 -- 172.21.15.97:0/3329055535 wait complete. 2024-01-31T05:02:06.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:02:06.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.769+0000 7f29e9400640 1 Processor -- start 2024-01-31T05:02:06.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.769+0000 7f29e9400640 1 -- start start 2024-01-31T05:02:06.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e9400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29e415a730 0x7f29e40abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e9400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 0x7f29e40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e9400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e415abc0 con 0x7f29e415a350 2024-01-31T05:02:06.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e9400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e40ce7e0 con 0x7f29e415a730 2024-01-31T05:02:06.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e9400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b2c90 con 0x7f29e40cde10 2024-01-31T05:02:06.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e2800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f29e415a350 0x7f29e406da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47138/0 (socket says 172.21.15.97:47138) 2024-01-31T05:02:06.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.771+0000 7f29e2800640 1 -- 172.21.15.97:0/3498273611 learned_addr learned my addr 172.21.15.97:0/3498273611 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:06.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e3200640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 0x7f29e40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1e00640 1 --2- 172.21.15.97:0/3498273611 >> v2:172.21.15.188:3300/0 conn(0x7f29e415a730 0x7f29e40abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098986421 0 0) 0x7f29e415abc0 con 0x7f29e415a350 2024-01-31T05:02:06.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c4005730 con 0x7f29e415a350 2024-01-31T05:02:06.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435807885 0 0) 0x7f29c4005730 con 0x7f29e415a350 2024-01-31T05:02:06.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 msgr2=0x7f29e40ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 0x7f29e40ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 >> v2:172.21.15.188:3300/0 conn(0x7f29e415a730 msgr2=0x7f29e40abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 --2- 172.21.15.97:0/3498273611 >> v2:172.21.15.188:3300/0 conn(0x7f29e415a730 0x7f29e40abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e415aea0 con 0x7f29e415a350 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f29d40017f0 con 0x7f29e415a350 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1e00640 1 --2- 172.21.15.97:0/3498273611 >> v2:172.21.15.188:3300/0 conn(0x7f29e415a730 0x7f29e40abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 >> v1:172.21.15.97:6789/0 conn(0x7f29e415a350 legacy=0x7f29e406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c4005bb0 0x7f29c4005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40069d0 0x7f29c4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 0x7f29c4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40ce7e0 con 0x7f29c4005bb0 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b2c90 con 0x7f29c40069d0 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.772+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c400dc70 con 0x7f29e40cde10 2024-01-31T05:02:06.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e2800640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c4005bb0 0x7f29c4005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e2800640 1 -- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 msgr2=0x7f29c4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e3200640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 0x7f29c4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e1e00640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40069d0 0x7f29c4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e2800640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40cde10 0x7f29c4007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e2800640 1 -- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40069d0 msgr2=0x7f29c4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e2800640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c40069d0 0x7f29c4006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.773+0000 7f29e2800640 1 -- 172.21.15.97:0/3498273611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4005730 con 0x7f29c4005bb0 2024-01-31T05:02:06.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.774+0000 7f29e2800640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c4005bb0 0x7f29c4005fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c160 tx=0x7f29c4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f032564330d386 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4007270 con 0x7f29c4005bb0 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d40073f0 con 0x7f29c4005bb0 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e1400640 1 -- 172.21.15.97:0/3498273611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d402c850 con 0x7f29c4005bb0 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e9400640 1 -- 172.21.15.97:0/3498273611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c4005bb0 msgr2=0x7f29c4005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e9400640 1 --2- 172.21.15.97:0/3498273611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29c4005bb0 0x7f29c4005fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f29d400c160 tx=0x7f29c4007a00 comp rx=0 tx=0).stop 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e3200640 1 -- 172.21.15.97:0/3498273611 reap_dead start 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e9400640 1 -- 172.21.15.97:0/3498273611 shutdown_connections 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e9400640 1 -- 172.21.15.97:0/3498273611 >> 172.21.15.97:0/3498273611 conn(0x7f29e40b0a10 msgr2=0x7f29e40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e9400640 1 -- 172.21.15.97:0/3498273611 shutdown_connections 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.775+0000 7f29e9400640 1 -- 172.21.15.97:0/3498273611 wait complete. 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.777+0000 7f29e9400640 1 Processor -- start 2024-01-31T05:02:06.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.777+0000 7f29e9400640 1 -- start start 2024-01-31T05:02:06.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.778+0000 7f29e9400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4005bb0 0x7f29e414d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.778+0000 7f29e9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 0x7f29e40af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.778+0000 7f29e9400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40afca0 0x7f29e415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.778+0000 7f29e9400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e409a390 con 0x7f29e414e230 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.778+0000 7f29e9400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e414de40 con 0x7f29c4005bb0 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.778+0000 7f29e9400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e40b0220 con 0x7f29e40afca0 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 0x7f29e40af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 0x7f29e40af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45634/0 (socket says 172.21.15.97:45634) 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 -- 172.21.15.97:0/3027917837 learned_addr learned my addr 172.21.15.97:0/3027917837 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e2800640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4005bb0 0x7f29e414d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 -- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40afca0 msgr2=0x7f29e415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40afca0 0x7f29e415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 -- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4005bb0 msgr2=0x7f29e414d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4005bb0 0x7f29e414d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 -- 172.21.15.97:0/3027917837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4099a60 con 0x7f29e414e230 2024-01-31T05:02:06.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e2800640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4005bb0 0x7f29e414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.779+0000 7f29e1e00640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 0x7f29e40af760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f29cc007fa0 tx=0x7f29cc00e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.780+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc015610 con 0x7f29e414e230 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.780+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc00f040 con 0x7f29e414e230 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.780+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc015990 con 0x7f29e414e230 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.780+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e4099d40 con 0x7f29e414e230 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.780+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e4151030 con 0x7f29e414e230 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.781+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e409a540 con 0x7f29e414e230 2024-01-31T05:02:06.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.782+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f29cc015af0 con 0x7f29e414e230 2024-01-31T05:02:06.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.783+0000 7f29daa00640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f29bc06af00 0x7f29bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:06.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.783+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f29cc08a6c0 con 0x7f29e414e230 2024-01-31T05:02:06.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.783+0000 7f29e2800640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f29bc06af00 0x7f29bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:06.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.784+0000 7f29e2800640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f29bc06af00 0x7f29bc06d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29d4034000 tx=0x7f29d4009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:06.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.788+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f29e409a540 con 0x7f29e414e230 2024-01-31T05:02:06.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.983+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f29e4067830 con 0x7f29e414e230 2024-01-31T05:02:06.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.984+0000 7f29daa00640 1 -- 172.21.15.97:0/3027917837 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f29cc055230 con 0x7f29e414e230 2024-01-31T05:02:06.984 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f29bc06af00 msgr2=0x7f29bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f29bc06af00 0x7f29bc06d3c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f29d4034000 tx=0x7f29d4009030 comp rx=0 tx=0).stop 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 msgr2=0x7f29e40af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 0x7f29e40af760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f29cc007fa0 tx=0x7f29cc00e5d0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 shutdown_connections 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e40afca0 0x7f29e415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f29bc06af00 0x7f29bc06d3c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29e414e230 0x7f29e40af760 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 --2- 172.21.15.97:0/3027917837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4005bb0 0x7f29e414d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 >> 172.21.15.97:0/3027917837 conn(0x7f29e40b0a10 msgr2=0x7f29e40acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 shutdown_connections 2024-01-31T05:02:06.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:06.987+0000 7f29e9400640 1 -- 172.21.15.97:0/3027917837 wait complete. 2024-01-31T05:02:07.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:07.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.212+0000 7fcc87c00640 1 Processor -- start 2024-01-31T05:02:07.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.212+0000 7fcc87c00640 1 -- start start 2024-01-31T05:02:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.213+0000 7fcc87c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc800722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.213+0000 7fcc87c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8006cd20 con 0x7fcc800abcf0 2024-01-31T05:02:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.213+0000 7fcc87c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc800ceb60 con 0x7fcc800cbf20 2024-01-31T05:02:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.213+0000 7fcc87c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8015b850 con 0x7fcc8015b470 2024-01-31T05:02:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.214+0000 7fcc7fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcc800abcf0 0x7fcc800cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47146/0 (socket says 172.21.15.97:47146) 2024-01-31T05:02:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.214+0000 7fcc7fe00640 1 -- 172.21.15.97:0/3826551615 learned_addr learned my addr 172.21.15.97:0/3826551615 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.214+0000 7fcc85200640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc800722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.214+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974891619 0 0) 0x7fcc8006cd20 con 0x7fcc800abcf0 2024-01-31T05:02:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.214+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc60003520 con 0x7fcc800abcf0 2024-01-31T05:02:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 106966635 0 0) 0x7fcc60003520 con 0x7fcc800abcf0 2024-01-31T05:02:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 msgr2=0x7fcc800722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc800722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 >> v1:172.21.15.188:6789/0 conn(0x7fcc800cbf20 legacy=0x7fcc8006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8006c3f0 con 0x7fcc800abcf0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fcc68001670 con 0x7fcc800abcf0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 >> v1:172.21.15.97:6789/0 conn(0x7fcc800abcf0 legacy=0x7fcc800cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc600047c0 0x7fcc60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8015b850 con 0x7fcc600039a0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6000ba60 con 0x7fcc600047c0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6000bc20 con 0x7fcc8015b470 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.215+0000 7fcc85200640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85200640 1 -- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 msgr2=0x7fcc60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85c00640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc7fe00640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc600047c0 0x7fcc60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85200640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc60005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85200640 1 -- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc600047c0 msgr2=0x7fcc60004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85200640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc600047c0 0x7fcc60004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85200640 1 -- 172.21.15.97:0/3826551615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc60003520 con 0x7fcc600039a0 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc85c00640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc8015b470 0x7fcc60005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.216+0000 7fcc7fe00640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc600047c0 0x7fcc60004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.217+0000 7fcc85200640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc60003dc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800e140 tx=0x7fcc600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3147a811ea613e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.217+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7802cbf0 con 0x7fcc600039a0 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.217+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7800f040 con 0x7fcc600039a0 2024-01-31T05:02:07.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.217+0000 7fcc7f400640 1 -- 172.21.15.97:0/3826551615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78034440 con 0x7fcc600039a0 2024-01-31T05:02:07.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc87c00640 1 -- 172.21.15.97:0/3826551615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 msgr2=0x7fcc60003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc87c00640 1 --2- 172.21.15.97:0/3826551615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc60003dc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800e140 tx=0x7fcc600057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc85c00640 1 -- 172.21.15.97:0/3826551615 reap_dead start 2024-01-31T05:02:07.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc87c00640 1 -- 172.21.15.97:0/3826551615 shutdown_connections 2024-01-31T05:02:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc87c00640 1 -- 172.21.15.97:0/3826551615 >> 172.21.15.97:0/3826551615 conn(0x7fcc800b0a10 msgr2=0x7fcc800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc87c00640 1 -- 172.21.15.97:0/3826551615 shutdown_connections 2024-01-31T05:02:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.218+0000 7fcc87c00640 1 -- 172.21.15.97:0/3826551615 wait complete. 2024-01-31T05:02:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.220+0000 7fcc87c00640 1 Processor -- start 2024-01-31T05:02:07.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.220+0000 7fcc87c00640 1 -- start start 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc87c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc801f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc87c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc80155a00 0x7fcc80155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc87c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc801544d0 0x7fcc80154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc87c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80153220 con 0x7fcc600039a0 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc87c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801563a0 con 0x7fcc80155a00 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc87c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc80153430 con 0x7fcc801544d0 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc801f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc801f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45656/0 (socket says 172.21.15.97:45656) 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 -- 172.21.15.97:0/4038849537 learned_addr learned my addr 172.21.15.97:0/4038849537 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 -- 172.21.15.97:0/4038849537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc801544d0 msgr2=0x7fcc80154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc801544d0 0x7fcc80154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 -- 172.21.15.97:0/4038849537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc80155a00 msgr2=0x7fcc80155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc80155a00 0x7fcc80155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.221+0000 7fcc85200640 1 -- 172.21.15.97:0/4038849537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80153710 con 0x7fcc600039a0 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.222+0000 7fcc85200640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc801f2390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800e140 tx=0x7fcc78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.222+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78034bd0 con 0x7fcc600039a0 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.222+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78034d30 con 0x7fcc600039a0 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.222+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78034ea0 con 0x7fcc600039a0 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.222+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc801539f0 con 0x7fcc600039a0 2024-01-31T05:02:07.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.222+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc80153fb0 con 0x7fcc600039a0 2024-01-31T05:02:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.223+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc80067830 con 0x7fcc600039a0 2024-01-31T05:02:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.224+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fcc78040450 con 0x7fcc600039a0 2024-01-31T05:02:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.225+0000 7fcc7cc00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc5806aed0 0x7fcc5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.225+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcc780ac250 con 0x7fcc600039a0 2024-01-31T05:02:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.225+0000 7fcc7fe00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc5806aed0 0x7fcc5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.226+0000 7fcc7fe00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc5806aed0 0x7fcc5806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc80155660 tx=0x7fcc68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:07.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.230+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fcc78076d30 con 0x7fcc600039a0 2024-01-31T05:02:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.437+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcc800b1870 con 0x7fcc600039a0 2024-01-31T05:02:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.438+0000 7fcc7cc00640 1 -- 172.21.15.97:0/4038849537 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcc800b1870 con 0x7fcc600039a0 2024-01-31T05:02:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:07.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.440+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc5806aed0 msgr2=0x7fcc5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.440+0000 7fcc87c00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc5806aed0 0x7fcc5806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc80155660 tx=0x7fcc68009040 comp rx=0 tx=0).stop 2024-01-31T05:02:07.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.440+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 msgr2=0x7fcc801f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.440+0000 7fcc87c00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc801f2390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800e140 tx=0x7fcc78009750 comp rx=0 tx=0).stop 2024-01-31T05:02:07.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 shutdown_connections 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc801544d0 0x7fcc80154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc80155a00 0x7fcc80155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcc5806aed0 0x7fcc5806d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 --2- 172.21.15.97:0/4038849537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc600039a0 0x7fcc801f2390 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 >> 172.21.15.97:0/4038849537 conn(0x7fcc800b0a10 msgr2=0x7fcc800cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 shutdown_connections 2024-01-31T05:02:07.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.441+0000 7fcc87c00640 1 -- 172.21.15.97:0/4038849537 wait complete. 2024-01-31T05:02:07.456 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:02:07.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:07.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-01-31T05:02:07.547 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:07.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:02:07.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:07.548 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-31T05:02:07.597 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:02:07.597 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:07.655 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:02:07.655 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:07.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:07.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.923+0000 7fb7d6200640 1 Processor -- start 2024-01-31T05:02:07.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.923+0000 7fb7d6200640 1 -- start start 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7d6200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7d00cb810 0x7fb7d00cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7d6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d00ccd00 0x7fb7d00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7d6200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d006d1a0 con 0x7fb7d015b400 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7d6200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d00cc960 con 0x7fb7d00cb810 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7d6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d00cd6f0 con 0x7fb7d00ccd00 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7cf400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7d00cb810 0x7fb7d00cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7cf400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7d00cb810 0x7fb7d00cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45666/0 (socket says 172.21.15.97:45666) 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7cf400640 1 -- 172.21.15.97:0/644135507 learned_addr learned my addr 172.21.15.97:0/644135507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.925+0000 7fb7cfe00640 1 --2- 172.21.15.97:0/644135507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d00ccd00 0x7fb7d00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.926+0000 7fb7cf400640 1 -- 172.21.15.97:0/644135507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d00ccd00 msgr2=0x7fb7d00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.926+0000 7fb7cf400640 1 --2- 172.21.15.97:0/644135507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d00ccd00 0x7fb7d00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.926+0000 7fb7cf400640 1 -- 172.21.15.97:0/644135507 >> v1:172.21.15.188:6789/0 conn(0x7fb7d015b400 legacy=0x7fb7d006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.926+0000 7fb7cf400640 1 -- 172.21.15.97:0/644135507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d006c600 con 0x7fb7d00cb810 2024-01-31T05:02:07.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.927+0000 7fb7cf400640 1 --2- 172.21.15.97:0/644135507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00cb810 0x7fb7d00cbbf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009870 tx=0x7fb7c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfff82373fb4681 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:07.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.927+0000 7fb7ce000640 1 -- 172.21.15.97:0/644135507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0031640 con 0x7fb7d00cb810 2024-01-31T05:02:07.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7ce000640 1 -- 172.21.15.97:0/644135507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c00317a0 con 0x7fb7d00cb810 2024-01-31T05:02:07.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7ce000640 1 -- 172.21.15.97:0/644135507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0031b30 con 0x7fb7d00cb810 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 -- 172.21.15.97:0/644135507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00cb810 msgr2=0x7fb7d00cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 --2- 172.21.15.97:0/644135507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00cb810 0x7fb7d00cbbf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009870 tx=0x7fb7c002b8f0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 -- 172.21.15.97:0/644135507 shutdown_connections 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 --2- 172.21.15.97:0/644135507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d00ccd00 0x7fb7d00cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 --2- 172.21.15.97:0/644135507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00cb810 0x7fb7d00cbbf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 -- 172.21.15.97:0/644135507 >> 172.21.15.97:0/644135507 conn(0x7fb7d00b0a10 msgr2=0x7fb7d00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 -- 172.21.15.97:0/644135507 shutdown_connections 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.928+0000 7fb7d6200640 1 -- 172.21.15.97:0/644135507 wait complete. 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.930+0000 7fb7d6200640 1 Processor -- start 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.930+0000 7fb7d6200640 1 -- start start 2024-01-31T05:02:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7d6200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d00cb810 0x7fb7d014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7d6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 0x7fb7d00c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7d6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d015b400 0x7fb7d00ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7d6200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d015d270 con 0x7fb7d00ccd00 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7d6200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d015e210 con 0x7fb7d00cb810 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7d6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d014dfd0 con 0x7fb7d015b400 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 0x7fb7d00c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 0x7fb7d00c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45674/0 (socket says 172.21.15.97:45674) 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.931+0000 7fb7cea00640 1 -- 172.21.15.97:0/3849977418 learned_addr learned my addr 172.21.15.97:0/3849977418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7cea00640 1 -- 172.21.15.97:0/3849977418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d015b400 msgr2=0x7fb7d00ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7cea00640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d015b400 0x7fb7d00ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7cea00640 1 -- 172.21.15.97:0/3849977418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d00cb810 msgr2=0x7fb7d014da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7cea00640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d00cb810 0x7fb7d014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7cea00640 1 -- 172.21.15.97:0/3849977418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00094d0 con 0x7fb7d00ccd00 2024-01-31T05:02:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7cea00640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 0x7fb7d00c9ee0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800b3c0 tx=0x7fb7b800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8011830 con 0x7fb7d00ccd00 2024-01-31T05:02:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8011990 con 0x7fb7d00ccd00 2024-01-31T05:02:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8011d80 con 0x7fb7d00ccd00 2024-01-31T05:02:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.932+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d015d450 con 0x7fb7d00ccd00 2024-01-31T05:02:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.933+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d015da10 con 0x7fb7d00ccd00 2024-01-31T05:02:07.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.934+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d00677c0 con 0x7fb7d00ccd00 2024-01-31T05:02:07.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.935+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb7b801b460 con 0x7fb7d00ccd00 2024-01-31T05:02:07.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.935+0000 7fb7c7400640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb7a806af50 0x7fb7a806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:07.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.936+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb7b8089460 con 0x7fb7d00ccd00 2024-01-31T05:02:07.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.936+0000 7fb7cf400640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb7a806af50 0x7fb7a806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:07.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.936+0000 7fb7cf400640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb7a806af50 0x7fb7a806d410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036000 tx=0x7fb7c0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:07.940+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb7b8054050 con 0x7fb7d00ccd00 2024-01-31T05:02:08.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.133+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7d00af650 con 0x7fb7d00ccd00 2024-01-31T05:02:08.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.136+0000 7fb7c7400640 1 -- 172.21.15.97:0/3849977418 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7fb7b8057670 con 0x7fb7d00ccd00 2024-01-31T05:02:08.136 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:08.136 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.138+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb7a806af50 msgr2=0x7fb7a806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.138+0000 7fb7d6200640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb7a806af50 0x7fb7a806d410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0036000 tx=0x7fb7c0030000 comp rx=0 tx=0).stop 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 msgr2=0x7fb7d00c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 0x7fb7d00c9ee0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800b3c0 tx=0x7fb7b800b880 comp rx=0 tx=0).stop 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 shutdown_connections 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d015b400 0x7fb7d00ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb7a806af50 0x7fb7a806d410 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d00ccd00 0x7fb7d00c9ee0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 --2- 172.21.15.97:0/3849977418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d00cb810 0x7fb7d014da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 >> 172.21.15.97:0/3849977418 conn(0x7fb7d00b0a10 msgr2=0x7fb7d00b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 shutdown_connections 2024-01-31T05:02:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.139+0000 7fb7d6200640 1 -- 172.21.15.97:0/3849977418 wait complete. 2024-01-31T05:02:08.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:02:08.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.373+0000 7fab3c600640 1 Processor -- start 2024-01-31T05:02:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.373+0000 7fab3c600640 1 -- start start 2024-01-31T05:02:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab3c600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fab3415b470 0x7fab340722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab3c600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fab340cbf20 0x7fab340ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab3c600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3406cd20 con 0x7fab340abcf0 2024-01-31T05:02:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab3c600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340ce970 con 0x7fab340cbf20 2024-01-31T05:02:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab3c600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3415b850 con 0x7fab3415b470 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab39200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fab340abcf0 0x7fab340cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47152/0 (socket says 172.21.15.97:47152) 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.375+0000 7fab39200640 1 -- 172.21.15.97:0/2173567659 learned_addr learned my addr 172.21.15.97:0/2173567659 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab3a600640 1 --2- 172.21.15.97:0/2173567659 >> v2:172.21.15.97:3300/0 conn(0x7fab340cbf20 0x7fab340ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab39c00640 1 --2- 172.21.15.97:0/2173567659 >> v2:172.21.15.188:3300/0 conn(0x7fab3415b470 0x7fab340722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab33e00640 1 -- 172.21.15.97:0/2173567659 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156308709 0 0) 0x7fab3406cd20 con 0x7fab340abcf0 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab33e00640 1 -- 172.21.15.97:0/2173567659 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab14003520 con 0x7fab340abcf0 2024-01-31T05:02:08.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab3a600640 1 -- 172.21.15.97:0/2173567659 >> v2:172.21.15.188:3300/0 conn(0x7fab3415b470 msgr2=0x7fab340722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab3a600640 1 --2- 172.21.15.97:0/2173567659 >> v2:172.21.15.188:3300/0 conn(0x7fab3415b470 0x7fab340722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab3a600640 1 -- 172.21.15.97:0/2173567659 >> v1:172.21.15.97:6789/0 conn(0x7fab340abcf0 legacy=0x7fab340cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab33e00640 1 -- 172.21.15.97:0/2173567659 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2179698920 0 0) 0x7fab14003520 con 0x7fab340abcf0 2024-01-31T05:02:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.376+0000 7fab3a600640 1 -- 172.21.15.97:0/2173567659 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340ceb60 con 0x7fab340cbf20 2024-01-31T05:02:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.377+0000 7fab3a600640 1 --2- 172.21.15.97:0/2173567659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab340cbf20 0x7fab340ce430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fab34067370 tx=0x7fab2c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495ccab2b08d40f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab33e00640 1 -- 172.21.15.97:0/2173567659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c001670 con 0x7fab340cbf20 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab33e00640 1 -- 172.21.15.97:0/2173567659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c02c8b0 con 0x7fab340cbf20 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab33e00640 1 -- 172.21.15.97:0/2173567659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c02cc20 con 0x7fab340cbf20 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 -- 172.21.15.97:0/2173567659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab340cbf20 msgr2=0x7fab340ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 --2- 172.21.15.97:0/2173567659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab340cbf20 0x7fab340ce430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fab34067370 tx=0x7fab2c02bd80 comp rx=0 tx=0).stop 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 -- 172.21.15.97:0/2173567659 shutdown_connections 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 --2- 172.21.15.97:0/2173567659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab340cbf20 0x7fab340ce430 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 --2- 172.21.15.97:0/2173567659 >> v2:172.21.15.188:3300/0 conn(0x7fab3415b470 0x7fab340722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 -- 172.21.15.97:0/2173567659 >> 172.21.15.97:0/2173567659 conn(0x7fab340b0a10 msgr2=0x7fab340b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 -- 172.21.15.97:0/2173567659 shutdown_connections 2024-01-31T05:02:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.378+0000 7fab3c600640 1 -- 172.21.15.97:0/2173567659 wait complete. 2024-01-31T05:02:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.380+0000 7fab3c600640 1 Processor -- start 2024-01-31T05:02:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.380+0000 7fab3c600640 1 -- start start 2024-01-31T05:02:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3c600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab340abcf0 0x7fab34159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3c600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab340cbf20 0x7fab340b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3c600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 0x7fab340b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3c600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340b3c60 con 0x7fab3415b470 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3c600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab340c8130 con 0x7fab340cbf20 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3c600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab340c8410 con 0x7fab340abcf0 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.381+0000 7fab3a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 0x7fab340b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 0x7fab340b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45694/0 (socket says 172.21.15.97:45694) 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 -- 172.21.15.97:0/4251460500 learned_addr learned my addr 172.21.15.97:0/4251460500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab39c00640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab340abcf0 0x7fab34159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab39200640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab340cbf20 0x7fab340b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 -- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab340abcf0 msgr2=0x7fab34159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab340abcf0 0x7fab34159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 -- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab340cbf20 msgr2=0x7fab340b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab340cbf20 0x7fab340b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 -- 172.21.15.97:0/4251460500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009a70 con 0x7fab3415b470 2024-01-31T05:02:08.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.382+0000 7fab3a600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 0x7fab340b3430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab2c037040 tx=0x7fab2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.383+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c02cea0 con 0x7fab3415b470 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.383+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c034e70 con 0x7fab3415b470 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.383+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c040670 con 0x7fab3415b470 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.383+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab340c86f0 con 0x7fab3415b470 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.383+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab340b3970 con 0x7fab3415b470 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.384+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab340c8c10 con 0x7fab3415b470 2024-01-31T05:02:08.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.385+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fab2c0407d0 con 0x7fab3415b470 2024-01-31T05:02:08.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.386+0000 7fab31600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fab0c06aed0 0x7fab0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.386+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fab2c0ac230 con 0x7fab3415b470 2024-01-31T05:02:08.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.386+0000 7fab39c00640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fab0c06aed0 0x7fab0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.386+0000 7fab39c00640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fab0c06aed0 0x7fab0c06d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab24009a20 tx=0x7fab24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:08.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.390+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fab2c076e30 con 0x7fab3415b470 2024-01-31T05:02:08.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.584+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab34067830 con 0x7fab3415b470 2024-01-31T05:02:08.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.585+0000 7fab31600640 1 -- 172.21.15.97:0/4251460500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab2c009450 con 0x7fab3415b470 2024-01-31T05:02:08.585 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fab0c06aed0 msgr2=0x7fab0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fab0c06aed0 0x7fab0c06d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab24009a20 tx=0x7fab24008040 comp rx=0 tx=0).stop 2024-01-31T05:02:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 msgr2=0x7fab340b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 0x7fab340b3430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab2c037040 tx=0x7fab2c004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:08.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 shutdown_connections 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fab0c06aed0 0x7fab0c06d390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab3415b470 0x7fab340b3430 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab340cbf20 0x7fab340b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 --2- 172.21.15.97:0/4251460500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab340abcf0 0x7fab34159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 >> 172.21.15.97:0/4251460500 conn(0x7fab340b0a10 msgr2=0x7fab340b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 shutdown_connections 2024-01-31T05:02:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.588+0000 7fab3c600640 1 -- 172.21.15.97:0/4251460500 wait complete. 2024-01-31T05:02:08.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.818+0000 7ff61bc00640 1 Processor -- start 2024-01-31T05:02:08.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.818+0000 7ff61bc00640 1 -- start start 2024-01-31T05:02:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff61bc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff61415b470 0x7ff6140722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff61bc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140abcf0 0x7ff6140ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff61bc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61406cd20 con 0x7ff6140cb810 2024-01-31T05:02:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff61bc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6140ce970 con 0x7ff61415b470 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff61bc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415b850 con 0x7ff6140abcf0 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff619200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff61415b470 0x7ff6140722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff619200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff61415b470 0x7ff6140722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45696/0 (socket says 172.21.15.97:45696) 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.820+0000 7ff619200640 1 -- 172.21.15.97:0/3710835433 learned_addr learned my addr 172.21.15.97:0/3710835433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.821+0000 7ff619200640 1 -- 172.21.15.97:0/3710835433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140abcf0 msgr2=0x7ff6140ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.821+0000 7ff619200640 1 --2- 172.21.15.97:0/3710835433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140abcf0 0x7ff6140ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.821+0000 7ff619200640 1 -- 172.21.15.97:0/3710835433 >> v1:172.21.15.188:6789/0 conn(0x7ff6140cb810 legacy=0x7ff6140ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:08.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.821+0000 7ff619200640 1 -- 172.21.15.97:0/3710835433 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6140ceb60 con 0x7ff61415b470 2024-01-31T05:02:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.822+0000 7ff619200640 1 --2- 172.21.15.97:0/3710835433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff61415b470 0x7ff6140722c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff604009880 tx=0x7ff60402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789fcda7246fece5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:08.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.822+0000 7ff613e00640 1 -- 172.21.15.97:0/3710835433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604004230 con 0x7ff61415b470 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.822+0000 7ff613e00640 1 -- 172.21.15.97:0/3710835433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604004390 con 0x7ff61415b470 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.822+0000 7ff613e00640 1 -- 172.21.15.97:0/3710835433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604031830 con 0x7ff61415b470 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 -- 172.21.15.97:0/3710835433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff61415b470 msgr2=0x7ff6140722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 --2- 172.21.15.97:0/3710835433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff61415b470 0x7ff6140722c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff604009880 tx=0x7ff60402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 -- 172.21.15.97:0/3710835433 shutdown_connections 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 --2- 172.21.15.97:0/3710835433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140abcf0 0x7ff6140ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 --2- 172.21.15.97:0/3710835433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff61415b470 0x7ff6140722c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 -- 172.21.15.97:0/3710835433 >> 172.21.15.97:0/3710835433 conn(0x7ff6140b0a10 msgr2=0x7ff6140b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 -- 172.21.15.97:0/3710835433 shutdown_connections 2024-01-31T05:02:08.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.823+0000 7ff61bc00640 1 -- 172.21.15.97:0/3710835433 wait complete. 2024-01-31T05:02:08.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.825+0000 7ff61bc00640 1 Processor -- start 2024-01-31T05:02:08.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.825+0000 7ff61bc00640 1 -- start start 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff61bc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 0x7ff614144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff61bc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140cb810 0x7ff6141514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff61bc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff61415b470 0x7ff6141538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff61bc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415c7d0 con 0x7ff6140abcf0 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff61bc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415cab0 con 0x7ff61415b470 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff61bc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61415cd90 con 0x7ff6140cb810 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff619200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 0x7ff614144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff619200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 0x7ff614144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45710/0 (socket says 172.21.15.97:45710) 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff619200640 1 -- 172.21.15.97:0/1927252273 learned_addr learned my addr 172.21.15.97:0/1927252273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.826+0000 7ff60bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140cb810 0x7ff6141514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff619200640 1 -- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140cb810 msgr2=0x7ff6141514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff619200640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140cb810 0x7ff6141514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff619200640 1 -- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff61415b470 msgr2=0x7ff6141538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff619200640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff61415b470 0x7ff6141538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff619200640 1 -- 172.21.15.97:0/1927252273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604009460 con 0x7ff6140abcf0 2024-01-31T05:02:08.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff60bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140cb810 0x7ff6141514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:08.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.827+0000 7ff619200640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 0x7ff614144870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff6040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:08.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.828+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604031a40 con 0x7ff6140abcf0 2024-01-31T05:02:08.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.828+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61415d070 con 0x7ff6140abcf0 2024-01-31T05:02:08.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.828+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604005e90 con 0x7ff6140abcf0 2024-01-31T05:02:08.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.828+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604031de0 con 0x7ff6140abcf0 2024-01-31T05:02:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.828+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff614142060 con 0x7ff6140abcf0 2024-01-31T05:02:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.829+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff614067830 con 0x7ff6140abcf0 2024-01-31T05:02:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.830+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff604003610 con 0x7ff6140abcf0 2024-01-31T05:02:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.831+0000 7ff611600640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5ec06aed0 0x7ff5ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.831+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff6040abff0 con 0x7ff6140abcf0 2024-01-31T05:02:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.831+0000 7ff60bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5ec06aed0 0x7ff5ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:08.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.832+0000 7ff60bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5ec06aed0 0x7ff5ec06d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006fd0 tx=0x7ff5fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:08.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:08.835+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff6040df190 con 0x7ff6140abcf0 2024-01-31T05:02:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.027+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff614158d00 con 0x7ff6140abcf0 2024-01-31T05:02:09.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.028+0000 7ff611600640 1 -- 172.21.15.97:0/1927252273 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff604076b10 con 0x7ff6140abcf0 2024-01-31T05:02:09.027 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.030+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5ec06aed0 msgr2=0x7ff5ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.030+0000 7ff61bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5ec06aed0 0x7ff5ec06d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006fd0 tx=0x7ff5fc008040 comp rx=0 tx=0).stop 2024-01-31T05:02:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.030+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 msgr2=0x7ff614144870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.030+0000 7ff61bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 0x7ff614144870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff604000c00 tx=0x7ff6040046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 shutdown_connections 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff61415b470 0x7ff6141538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6140cb810 0x7ff6141514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff5ec06aed0 0x7ff5ec06d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 --2- 172.21.15.97:0/1927252273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6140abcf0 0x7ff614144870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 >> 172.21.15.97:0/1927252273 conn(0x7ff6140b0a10 msgr2=0x7ff614150950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 shutdown_connections 2024-01-31T05:02:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.031+0000 7ff61bc00640 1 -- 172.21.15.97:0/1927252273 wait complete. 2024-01-31T05:02:09.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:09.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.258+0000 7f6b79400640 1 Processor -- start 2024-01-31T05:02:09.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.258+0000 7f6b79400640 1 -- start start 2024-01-31T05:02:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.260+0000 7f6b79400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b740ce1e0 0x7f6b740b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.260+0000 7f6b79400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b740cb7d0 0x7f6b740cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.260+0000 7f6b79400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b74161fc0 con 0x7f6b74161b10 2024-01-31T05:02:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.260+0000 7f6b79400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b74162c40 con 0x7f6b740cb7d0 2024-01-31T05:02:09.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.260+0000 7f6b79400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741622a0 con 0x7f6b740ce1e0 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.260+0000 7f6b73200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b740cb7d0 0x7f6b740cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b73200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b740cb7d0 0x7f6b740cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45724/0 (socket says 172.21.15.97:45724) 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b73200640 1 -- 172.21.15.97:0/3182069928 learned_addr learned my addr 172.21.15.97:0/3182069928 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b72800640 1 --2- 172.21.15.97:0/3182069928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b740ce1e0 0x7f6b740b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b73200640 1 -- 172.21.15.97:0/3182069928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b740ce1e0 msgr2=0x7f6b740b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b73200640 1 --2- 172.21.15.97:0/3182069928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b740ce1e0 0x7f6b740b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b73200640 1 -- 172.21.15.97:0/3182069928 >> v1:172.21.15.97:6789/0 conn(0x7f6b74161b10 legacy=0x7f6b740ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.261+0000 7f6b73200640 1 -- 172.21.15.97:0/3182069928 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74162580 con 0x7f6b740cb7d0 2024-01-31T05:02:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.262+0000 7f6b73200640 1 --2- 172.21.15.97:0/3182069928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740cb7d0 0x7f6b740cbc80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009870 tx=0x7f6b6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8111feb4848c4d9d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.262+0000 7f6b71400640 1 -- 172.21.15.97:0/3182069928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c031640 con 0x7f6b740cb7d0 2024-01-31T05:02:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.262+0000 7f6b71400640 1 -- 172.21.15.97:0/3182069928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c0317a0 con 0x7f6b740cb7d0 2024-01-31T05:02:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.262+0000 7f6b71400640 1 -- 172.21.15.97:0/3182069928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c031b30 con 0x7f6b740cb7d0 2024-01-31T05:02:09.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 -- 172.21.15.97:0/3182069928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740cb7d0 msgr2=0x7f6b740cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 --2- 172.21.15.97:0/3182069928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740cb7d0 0x7f6b740cbc80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009870 tx=0x7f6b6c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 -- 172.21.15.97:0/3182069928 shutdown_connections 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 --2- 172.21.15.97:0/3182069928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740cb7d0 0x7f6b740cbc80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 --2- 172.21.15.97:0/3182069928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b740ce1e0 0x7f6b740b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 -- 172.21.15.97:0/3182069928 >> 172.21.15.97:0/3182069928 conn(0x7f6b740b0a10 msgr2=0x7f6b740b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 -- 172.21.15.97:0/3182069928 shutdown_connections 2024-01-31T05:02:09.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.263+0000 7f6b79400640 1 -- 172.21.15.97:0/3182069928 wait complete. 2024-01-31T05:02:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.265+0000 7f6b79400640 1 Processor -- start 2024-01-31T05:02:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.265+0000 7f6b79400640 1 -- start start 2024-01-31T05:02:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.266+0000 7f6b79400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b740cb7d0 0x7f6b7414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b79400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 0x7f6b7414ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b79400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b74161b10 0x7f6b74152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b79400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7409ce90 con 0x7f6b740ce1e0 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b79400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74150470 con 0x7f6b740cb7d0 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b79400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7409cbb0 con 0x7f6b74161b10 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 0x7f6b7414ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 0x7f6b7414ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45730/0 (socket says 172.21.15.97:45730) 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 -- 172.21.15.97:0/3612831023 learned_addr learned my addr 172.21.15.97:0/3612831023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 -- 172.21.15.97:0/3612831023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b74161b10 msgr2=0x7f6b74152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b74161b10 0x7f6b74152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 -- 172.21.15.97:0/3612831023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b740cb7d0 msgr2=0x7f6b7414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b740cb7d0 0x7f6b7414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.267+0000 7f6b71e00640 1 -- 172.21.15.97:0/3612831023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0094d0 con 0x7f6b740ce1e0 2024-01-31T05:02:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.268+0000 7f6b71e00640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 0x7f6b7414ff30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c7d0 tx=0x7f6b5c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.268+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c006d50 con 0x7f6b740ce1e0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.268+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c003970 con 0x7f6b740ce1e0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.268+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b741508c0 con 0x7f6b740ce1e0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.268+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c003d20 con 0x7f6b740ce1e0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.268+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b74150e80 con 0x7f6b740ce1e0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.269+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b74067830 con 0x7f6b740ce1e0 2024-01-31T05:02:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.270+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6b5c00a850 con 0x7f6b740ce1e0 2024-01-31T05:02:09.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.271+0000 7f6b6aa00640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b4c06aed0 0x7f6b4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.271+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6b5c0895a0 con 0x7f6b740ce1e0 2024-01-31T05:02:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.271+0000 7f6b72800640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b4c06aed0 0x7f6b4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.272+0000 7f6b72800640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b4c06aed0 0x7f6b4c06d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b64006e20 tx=0x7f6b64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.275+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6b5c054110 con 0x7f6b740ce1e0 2024-01-31T05:02:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.466+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b740cc6c0 con 0x7f6b740ce1e0 2024-01-31T05:02:09.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.466+0000 7f6b6aa00640 1 -- 172.21.15.97:0/3612831023 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b5c057730 con 0x7f6b740ce1e0 2024-01-31T05:02:09.466 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b4c06aed0 msgr2=0x7f6b4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b4c06aed0 0x7f6b4c06d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b64006e20 tx=0x7f6b64008040 comp rx=0 tx=0).stop 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 msgr2=0x7f6b7414ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 0x7f6b7414ff30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00c7d0 tx=0x7f6b5c00cc90 comp rx=0 tx=0).stop 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 shutdown_connections 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b74161b10 0x7f6b74152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b4c06aed0 0x7f6b4c06d390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b740ce1e0 0x7f6b7414ff30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 --2- 172.21.15.97:0/3612831023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b740cb7d0 0x7f6b7414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 >> 172.21.15.97:0/3612831023 conn(0x7f6b740b0a10 msgr2=0x7f6b740ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:09.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 shutdown_connections 2024-01-31T05:02:09.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.469+0000 7f6b79400640 1 -- 172.21.15.97:0/3612831023 wait complete. 2024-01-31T05:02:09.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:02:09.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.697+0000 7faac6800640 1 Processor -- start 2024-01-31T05:02:09.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.697+0000 7faac6800640 1 -- start start 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.698+0000 7faac6800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faac015a350 0x7faac00abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.698+0000 7faac6800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac00cde10 0x7faac00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.698+0000 7faac6800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac015abc0 con 0x7faac015a730 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.698+0000 7faac6800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac00ce7e0 con 0x7faac015a350 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.698+0000 7faac6800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b2c90 con 0x7faac00cde10 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faac015a350 0x7faac00abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faac015a350 0x7faac00abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45740/0 (socket says 172.21.15.97:45740) 2024-01-31T05:02:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 -- 172.21.15.97:0/2326459567 learned_addr learned my addr 172.21.15.97:0/2326459567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 -- 172.21.15.97:0/2326459567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac00cde10 msgr2=0x7faac00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 --2- 172.21.15.97:0/2326459567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac00cde10 0x7faac00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 -- 172.21.15.97:0/2326459567 >> v1:172.21.15.188:6789/0 conn(0x7faac015a730 legacy=0x7faac006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:09.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.699+0000 7faabf400640 1 -- 172.21.15.97:0/2326459567 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac015aea0 con 0x7faac015a350 2024-01-31T05:02:09.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.700+0000 7faabf400640 1 --2- 172.21.15.97:0/2326459567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac015a350 0x7faac00abcf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faab0009890 tx=0x7faab002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc0dd0c40e12083 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.700+0000 7faabe000640 1 -- 172.21.15.97:0/2326459567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab002f8a0 con 0x7faac015a350 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faabe000640 1 -- 172.21.15.97:0/2326459567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab002fa00 con 0x7faac015a350 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faabe000640 1 -- 172.21.15.97:0/2326459567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab002fd90 con 0x7faac015a350 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 -- 172.21.15.97:0/2326459567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac015a350 msgr2=0x7faac00abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 --2- 172.21.15.97:0/2326459567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac015a350 0x7faac00abcf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faab0009890 tx=0x7faab002dae0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 -- 172.21.15.97:0/2326459567 shutdown_connections 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 --2- 172.21.15.97:0/2326459567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac00cde10 0x7faac00ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 --2- 172.21.15.97:0/2326459567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac015a350 0x7faac00abcf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 -- 172.21.15.97:0/2326459567 >> 172.21.15.97:0/2326459567 conn(0x7faac00b0a10 msgr2=0x7faac00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 -- 172.21.15.97:0/2326459567 shutdown_connections 2024-01-31T05:02:09.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.701+0000 7faac6800640 1 -- 172.21.15.97:0/2326459567 wait complete. 2024-01-31T05:02:09.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.703+0000 7faac6800640 1 Processor -- start 2024-01-31T05:02:09.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.703+0000 7faac6800640 1 -- start start 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faac6800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac00cde10 0x7faac014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faac6800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac015a350 0x7faac009c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faac6800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 0x7faac00c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faac6800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01f32d0 con 0x7faac009c750 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faac6800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01f35b0 con 0x7faac00cde10 2024-01-31T05:02:09.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faac6800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac01f3890 con 0x7faac015a350 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faabfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 0x7faac00c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faabfe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 0x7faac00c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45746/0 (socket says 172.21.15.97:45746) 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faabfe00640 1 -- 172.21.15.97:0/3022615959 learned_addr learned my addr 172.21.15.97:0/3022615959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.704+0000 7faabea00640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac015a350 0x7faac009c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabf400640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac00cde10 0x7faac014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabfe00640 1 -- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac015a350 msgr2=0x7faac009c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabfe00640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac015a350 0x7faac009c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabfe00640 1 -- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac00cde10 msgr2=0x7faac014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabfe00640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac00cde10 0x7faac014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabfe00640 1 -- 172.21.15.97:0/3022615959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0009470 con 0x7faac009c750 2024-01-31T05:02:09.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faabfe00640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 0x7faac00c8090 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faab800b970 tx=0x7faab800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800cb40 con 0x7faac009c750 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800ccc0 con 0x7faac009c750 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac01f3b70 con 0x7faac009c750 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8014680 con 0x7faac009c750 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.705+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01f2f80 con 0x7faac009c750 2024-01-31T05:02:09.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.706+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac00ae710 con 0x7faac009c750 2024-01-31T05:02:09.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.707+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7faab801d3b0 con 0x7faac009c750 2024-01-31T05:02:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.708+0000 7faab7400640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa9806af00 0x7faa9806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.708+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7faab80898f0 con 0x7faac009c750 2024-01-31T05:02:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.709+0000 7faabf400640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa9806af00 0x7faa9806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:09.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.709+0000 7faabf400640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa9806af00 0x7faa9806d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faab0038000 tx=0x7faab003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.713+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7faac00ae710 con 0x7faac009c750 2024-01-31T05:02:09.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.901+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faac00ae710 con 0x7faac009c750 2024-01-31T05:02:09.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.901+0000 7faab7400640 1 -- 172.21.15.97:0/3022615959 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7faab8019020 con 0x7faac009c750 2024-01-31T05:02:09.901 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa9806af00 msgr2=0x7faa9806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa9806af00 0x7faa9806d3c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faab0038000 tx=0x7faab003b040 comp rx=0 tx=0).stop 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 msgr2=0x7faac00c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 0x7faac00c8090 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faab800b970 tx=0x7faab800be30 comp rx=0 tx=0).stop 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 shutdown_connections 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa9806af00 0x7faa9806d3c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac009c750 0x7faac00c8090 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faac015a350 0x7faac009c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 --2- 172.21.15.97:0/3022615959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faac00cde10 0x7faac014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:09.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 >> 172.21.15.97:0/3022615959 conn(0x7faac00b0a10 msgr2=0x7faac00ac3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:09.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 shutdown_connections 2024-01-31T05:02:09.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:09.904+0000 7faac6800640 1 -- 172.21.15.97:0/3022615959 wait complete. 2024-01-31T05:02:09.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.125+0000 7f2a5ea00640 1 Processor -- start 2024-01-31T05:02:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.126+0000 7f2a5ea00640 1 -- start start 2024-01-31T05:02:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.127+0000 7f2a5ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a5815d590 0x7f2a5815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.127+0000 7f2a5ea00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a58161ea0 con 0x7f2a580cfae0 2024-01-31T05:02:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.127+0000 7f2a5ea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a580cded0 con 0x7f2a580cff20 2024-01-31T05:02:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.127+0000 7f2a5ea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5815d970 con 0x7f2a5815d590 2024-01-31T05:02:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.127+0000 7f2a57400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2a580cfae0 0x7f2a58161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47160/0 (socket says 172.21.15.97:47160) 2024-01-31T05:02:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.127+0000 7f2a57400640 1 -- 172.21.15.97:0/638378336 learned_addr learned my addr 172.21.15.97:0/638378336 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:10.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210916037 0 0) 0x7f2a58161ea0 con 0x7f2a580cfae0 2024-01-31T05:02:10.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a38003520 con 0x7f2a580cfae0 2024-01-31T05:02:10.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a57e00640 1 --2- 172.21.15.97:0/638378336 >> v2:172.21.15.188:3300/0 conn(0x7f2a5815d590 0x7f2a5815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127759612 0 0) 0x7f2a38003520 con 0x7f2a580cfae0 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 >> v2:172.21.15.188:3300/0 conn(0x7f2a5815d590 msgr2=0x7f2a5815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 --2- 172.21.15.97:0/638378336 >> v2:172.21.15.188:3300/0 conn(0x7f2a5815d590 0x7f2a5815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 >> v1:172.21.15.188:6789/0 conn(0x7f2a580cff20 legacy=0x7f2a58161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580cf210 con 0x7f2a580cfae0 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f2a40001670 con 0x7f2a580cfae0 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 >> v1:172.21.15.97:6789/0 conn(0x7f2a580cfae0 legacy=0x7f2a58161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a38003970 0x7f2a38003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a380047c0 0x7f2a38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a38005180 0x7f2a3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5815d970 con 0x7f2a380047c0 2024-01-31T05:02:10.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3800c090 con 0x7f2a38003970 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.128+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3800c250 con 0x7f2a38005180 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57400640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a380047c0 0x7f2a38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57e00640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a38003970 0x7f2a38003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57400640 1 -- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a38005180 msgr2=0x7f2a3800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57400640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a38005180 0x7f2a3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57400640 1 -- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a38003970 msgr2=0x7f2a38003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57400640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a38003970 0x7f2a38003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a57400640 1 -- 172.21.15.97:0/638378336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a38003520 con 0x7f2a380047c0 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.129+0000 7f2a5ca00640 1 -- 172.21.15.97:0/638378336 reap_dead start 2024-01-31T05:02:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.130+0000 7f2a57400640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a380047c0 0x7f2a38004c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c210 tx=0x7f2a380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ce7721d6a8a106 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.130+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40005ad0 con 0x7f2a380047c0 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.130+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a40005c50 con 0x7f2a380047c0 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.130+0000 7f2a56a00640 1 -- 172.21.15.97:0/638378336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4002c8d0 con 0x7f2a380047c0 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 -- 172.21.15.97:0/638378336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a380047c0 msgr2=0x7f2a38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a380047c0 0x7f2a38004c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c210 tx=0x7f2a380057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 -- 172.21.15.97:0/638378336 shutdown_connections 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/638378336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a380047c0 0x7f2a38004c40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 -- 172.21.15.97:0/638378336 >> 172.21.15.97:0/638378336 conn(0x7f2a5809c560 msgr2=0x7f2a580d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 -- 172.21.15.97:0/638378336 shutdown_connections 2024-01-31T05:02:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.131+0000 7f2a5ea00640 1 -- 172.21.15.97:0/638378336 wait complete. 2024-01-31T05:02:10.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.133+0000 7f2a5ea00640 1 Processor -- start 2024-01-31T05:02:10.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.133+0000 7f2a5ea00640 1 -- start start 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a380047c0 0x7f2a5815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a58159b60 0x7f2a580b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 0x7f2a580c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580ca070 con 0x7f2a580c96b0 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580b76d0 con 0x7f2a58159b60 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580b6e40 con 0x7f2a380047c0 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 0x7f2a580c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 0x7f2a580c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45762/0 (socket says 172.21.15.97:45762) 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 -- 172.21.15.97:0/4174602211 learned_addr learned my addr 172.21.15.97:0/4174602211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a57e00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a380047c0 0x7f2a5815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 -- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a380047c0 msgr2=0x7f2a5815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a380047c0 0x7f2a5815a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 -- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a58159b60 msgr2=0x7f2a580b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.134+0000 7f2a5ca00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a58159b60 0x7f2a580b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a5ca00640 1 -- 172.21.15.97:0/4174602211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580b7110 con 0x7f2a580c96b0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a5ca00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 0x7f2a580c9b30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2a580d2f90 tx=0x7f2a5000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50007070 con 0x7f2a580c96b0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a500071f0 con 0x7f2a580c96b0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500075a0 con 0x7f2a580c96b0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a580b73f0 con 0x7f2a580c96b0 2024-01-31T05:02:10.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.135+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a580c71c0 con 0x7f2a580c96b0 2024-01-31T05:02:10.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.136+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58067830 con 0x7f2a580c96b0 2024-01-31T05:02:10.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.137+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2a50007700 con 0x7f2a580c96b0 2024-01-31T05:02:10.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.138+0000 7f2a4fe00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3006aed0 0x7f2a3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.138+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2a50088e40 con 0x7f2a580c96b0 2024-01-31T05:02:10.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.138+0000 7f2a57e00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3006aed0 0x7f2a3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.139+0000 7f2a57e00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3006aed0 0x7f2a3006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a58050350 tx=0x7f2a48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:10.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.142+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2a500539b0 con 0x7f2a580c96b0 2024-01-31T05:02:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.348+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a581603d0 con 0x7f2a580c96b0 2024-01-31T05:02:10.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.349+0000 7f2a4fe00640 1 -- 172.21.15.97:0/4174602211 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2a50017020 con 0x7f2a580c96b0 2024-01-31T05:02:10.349 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3006aed0 msgr2=0x7f2a3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3006aed0 0x7f2a3006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2a58050350 tx=0x7f2a48008040 comp rx=0 tx=0).stop 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 msgr2=0x7f2a580c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 0x7f2a580c9b30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2a580d2f90 tx=0x7f2a5000c980 comp rx=0 tx=0).stop 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 shutdown_connections 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3006aed0 0x7f2a3006d390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a580c96b0 0x7f2a580c9b30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a58159b60 0x7f2a580b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 --2- 172.21.15.97:0/4174602211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a380047c0 0x7f2a5815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 >> 172.21.15.97:0/4174602211 conn(0x7f2a5809c560 msgr2=0x7f2a58160f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 shutdown_connections 2024-01-31T05:02:10.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.352+0000 7f2a5ea00640 1 -- 172.21.15.97:0/4174602211 wait complete. 2024-01-31T05:02:10.370 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:02:10.370 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:10.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:10.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.638+0000 7f4085200640 1 Processor -- start 2024-01-31T05:02:10.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.638+0000 7f4085200640 1 -- start start 2024-01-31T05:02:10.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f4085200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4080159a50 0x7f40800caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f4085200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40800b1d80 0x7f40800cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f4085200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40800cddb0 con 0x7f40800cbb90 2024-01-31T05:02:10.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f4085200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40800cbfc0 con 0x7f40800b1d80 2024-01-31T05:02:10.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f4085200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40800cb440 con 0x7f4080159a50 2024-01-31T05:02:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f407d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40800b1d80 0x7f40800cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f407d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40800b1d80 0x7f40800cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45774/0 (socket says 172.21.15.97:45774) 2024-01-31T05:02:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f407d400640 1 -- 172.21.15.97:0/4042763508 learned_addr learned my addr 172.21.15.97:0/4042763508 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:10.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.640+0000 7f407de00640 1 --2- 172.21.15.97:0/4042763508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4080159a50 0x7f40800caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.641+0000 7f407d400640 1 -- 172.21.15.97:0/4042763508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4080159a50 msgr2=0x7f40800caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.641+0000 7f407d400640 1 --2- 172.21.15.97:0/4042763508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4080159a50 0x7f40800caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.641+0000 7f407d400640 1 -- 172.21.15.97:0/4042763508 >> v1:172.21.15.188:6789/0 conn(0x7f40800cbb90 legacy=0x7f408015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.641+0000 7f407d400640 1 -- 172.21.15.97:0/4042763508 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800ce090 con 0x7f40800b1d80 2024-01-31T05:02:10.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.642+0000 7f407d400640 1 --2- 172.21.15.97:0/4042763508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800cb650 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4068009870 tx=0x7f406802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6e6d3c23d2ffd6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:10.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.642+0000 7f407ca00640 1 -- 172.21.15.97:0/4042763508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068030840 con 0x7f40800b1d80 2024-01-31T05:02:10.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.643+0000 7f407ca00640 1 -- 172.21.15.97:0/4042763508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40680309a0 con 0x7f40800b1d80 2024-01-31T05:02:10.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.643+0000 7f407ca00640 1 -- 172.21.15.97:0/4042763508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068030d30 con 0x7f40800b1d80 2024-01-31T05:02:10.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 -- 172.21.15.97:0/4042763508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 msgr2=0x7f40800cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 --2- 172.21.15.97:0/4042763508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800cb650 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4068009870 tx=0x7f406802b980 comp rx=0 tx=0).stop 2024-01-31T05:02:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 -- 172.21.15.97:0/4042763508 shutdown_connections 2024-01-31T05:02:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 --2- 172.21.15.97:0/4042763508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800cb650 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 --2- 172.21.15.97:0/4042763508 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4080159a50 0x7f40800caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 -- 172.21.15.97:0/4042763508 >> 172.21.15.97:0/4042763508 conn(0x7f40800b0a10 msgr2=0x7f40800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 -- 172.21.15.97:0/4042763508 shutdown_connections 2024-01-31T05:02:10.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.644+0000 7f4085200640 1 -- 172.21.15.97:0/4042763508 wait complete. 2024-01-31T05:02:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.646+0000 7f4085200640 1 Processor -- start 2024-01-31T05:02:10.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.646+0000 7f4085200640 1 -- start start 2024-01-31T05:02:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f4085200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f4085200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40800cbb90 0x7f408009b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f4085200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4080159a50 0x7f408009bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f4085200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800abcf0 con 0x7f40800b1d80 2024-01-31T05:02:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f4085200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4080070e40 con 0x7f4080159a50 2024-01-31T05:02:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f4085200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408015c860 con 0x7f40800cbb90 2024-01-31T05:02:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f407de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f407de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45776/0 (socket says 172.21.15.97:45776) 2024-01-31T05:02:10.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.648+0000 7f407de00640 1 -- 172.21.15.97:0/3621909323 learned_addr learned my addr 172.21.15.97:0/3621909323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f407de00640 1 -- 172.21.15.97:0/3621909323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40800cbb90 msgr2=0x7f408009b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f407de00640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40800cbb90 0x7f408009b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f407de00640 1 -- 172.21.15.97:0/3621909323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4080159a50 msgr2=0x7f408009bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f407de00640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4080159a50 0x7f408009bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f407de00640 1 -- 172.21.15.97:0/3621909323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680094d0 con 0x7f40800b1d80 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f407de00640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800ceeb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f407000e870 tx=0x7f407000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40700094f0 con 0x7f40800b1d80 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000c940 con 0x7f40800b1d80 2024-01-31T05:02:10.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000ccf0 con 0x7f40800b1d80 2024-01-31T05:02:10.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408015cb40 con 0x7f40800b1d80 2024-01-31T05:02:10.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.649+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40800ac1e0 con 0x7f40800b1d80 2024-01-31T05:02:10.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.651+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40800677c0 con 0x7f40800b1d80 2024-01-31T05:02:10.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.652+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4070013470 con 0x7f40800b1d80 2024-01-31T05:02:10.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.652+0000 7f4076000640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f405806aed0 0x7f405806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:10.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.653+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4070089af0 con 0x7f40800b1d80 2024-01-31T05:02:10.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.653+0000 7f407d400640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f405806aed0 0x7f405806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:10.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.653+0000 7f407d400640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f405806aed0 0x7f405806d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4068036000 tx=0x7f406803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:10.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.657+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4070054660 con 0x7f40800b1d80 2024-01-31T05:02:10.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.850+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f408015db20 con 0x7f40800b1d80 2024-01-31T05:02:10.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.852+0000 7f4076000640 1 -- 172.21.15.97:0/3621909323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f4070021090 con 0x7f40800b1d80 2024-01-31T05:02:10.852 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:10.852 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f405806aed0 msgr2=0x7f405806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f405806aed0 0x7f405806d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4068036000 tx=0x7f406803b040 comp rx=0 tx=0).stop 2024-01-31T05:02:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 msgr2=0x7f40800ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:10.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800ceeb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f407000e870 tx=0x7f407000ed30 comp rx=0 tx=0).stop 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 shutdown_connections 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4080159a50 0x7f408009bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40800cbb90 0x7f408009b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f405806aed0 0x7f405806d390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 --2- 172.21.15.97:0/3621909323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40800b1d80 0x7f40800ceeb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.854+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 >> 172.21.15.97:0/3621909323 conn(0x7f40800b0a10 msgr2=0x7f40800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.855+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 shutdown_connections 2024-01-31T05:02:10.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:10.855+0000 7f4085200640 1 -- 172.21.15.97:0/3621909323 wait complete. 2024-01-31T05:02:10.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:02:10.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:11.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.076+0000 7fa5c7600640 1 Processor -- start 2024-01-31T05:02:11.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.076+0000 7fa5c7600640 1 -- start start 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5c7600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5c015b470 0x7fa5c00722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5c7600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5c00abcf0 0x7fa5c00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5c7600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c006cd20 con 0x7fa5c00cb810 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5c7600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c00ce970 con 0x7fa5c00abcf0 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5c7600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c015b850 con 0x7fa5c015b470 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5c00abcf0 0x7fa5c00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa5c00abcf0 0x7fa5c00ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45790/0 (socket says 172.21.15.97:45790) 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5c5600640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5c00cb810 0x7fa5c00ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47170/0 (socket says 172.21.15.97:47170) 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 -- 172.21.15.97:0/142159354 learned_addr learned my addr 172.21.15.97:0/142159354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 -- 172.21.15.97:0/142159354 >> v2:172.21.15.188:3300/0 conn(0x7fa5c015b470 msgr2=0x7fa5c00722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 --2- 172.21.15.97:0/142159354 >> v2:172.21.15.188:3300/0 conn(0x7fa5c015b470 0x7fa5c00722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 -- 172.21.15.97:0/142159354 >> v1:172.21.15.97:6789/0 conn(0x7fa5c00cb810 legacy=0x7fa5c00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.078+0000 7fa5bfe00640 1 -- 172.21.15.97:0/142159354 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00ceb60 con 0x7fa5c00abcf0 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.079+0000 7fa5bfe00640 1 --2- 172.21.15.97:0/142159354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c00ce430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0067370 tx=0x7fa5a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206f82f9f0422d30 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5bf400640 1 -- 172.21.15.97:0/142159354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8030860 con 0x7fa5c00abcf0 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5bf400640 1 -- 172.21.15.97:0/142159354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a80309c0 con 0x7fa5c00abcf0 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5bf400640 1 -- 172.21.15.97:0/142159354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8030d50 con 0x7fa5c00abcf0 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5c7600640 1 -- 172.21.15.97:0/142159354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 msgr2=0x7fa5c00ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5c7600640 1 --2- 172.21.15.97:0/142159354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c00ce430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0067370 tx=0x7fa5a802b9a0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5c7600640 1 -- 172.21.15.97:0/142159354 shutdown_connections 2024-01-31T05:02:11.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5c7600640 1 --2- 172.21.15.97:0/142159354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c00ce430 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5c7600640 1 --2- 172.21.15.97:0/142159354 >> v2:172.21.15.188:3300/0 conn(0x7fa5c015b470 0x7fa5c00722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.080+0000 7fa5c7600640 1 -- 172.21.15.97:0/142159354 >> 172.21.15.97:0/142159354 conn(0x7fa5c00b0a10 msgr2=0x7fa5c00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.081+0000 7fa5c7600640 1 -- 172.21.15.97:0/142159354 shutdown_connections 2024-01-31T05:02:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.081+0000 7fa5c7600640 1 -- 172.21.15.97:0/142159354 wait complete. 2024-01-31T05:02:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.082+0000 7fa5c7600640 1 Processor -- start 2024-01-31T05:02:11.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.082+0000 7fa5c7600640 1 -- start start 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c7600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c01f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c7600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c00cb810 0x7fa5c00999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c7600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c015b470 0x7fa5c00c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c7600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01f3e20 con 0x7fa5c00abcf0 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c7600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01f4100 con 0x7fa5c015b470 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c7600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00b1780 con 0x7fa5c00cb810 2024-01-31T05:02:11.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c01f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c01f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45794/0 (socket says 172.21.15.97:45794) 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.083+0000 7fa5c4c00640 1 -- 172.21.15.97:0/2657176298 learned_addr learned my addr 172.21.15.97:0/2657176298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c4c00640 1 -- 172.21.15.97:0/2657176298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c00cb810 msgr2=0x7fa5c00999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c4c00640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c00cb810 0x7fa5c00999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c4c00640 1 -- 172.21.15.97:0/2657176298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c015b470 msgr2=0x7fa5c00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c4c00640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c015b470 0x7fa5c00c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c4c00640 1 -- 172.21.15.97:0/2657176298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80094d0 con 0x7fa5c00abcf0 2024-01-31T05:02:11.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c4c00640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c01f34d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000b3f0 tx=0x7fa5b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0005970 con 0x7fa5c00abcf0 2024-01-31T05:02:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0005ad0 con 0x7fa5c00abcf0 2024-01-31T05:02:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0005e80 con 0x7fa5c00abcf0 2024-01-31T05:02:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c00b1a60 con 0x7fa5c00abcf0 2024-01-31T05:02:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.084+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c01f3a10 con 0x7fa5c00abcf0 2024-01-31T05:02:11.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.085+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c0067830 con 0x7fa5c00abcf0 2024-01-31T05:02:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.086+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa5b0011630 con 0x7fa5c00abcf0 2024-01-31T05:02:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.087+0000 7fa5bcc00640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa59806aed0 0x7fa59806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.087+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa5b0089710 con 0x7fa5c00abcf0 2024-01-31T05:02:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.087+0000 7fa5bfe00640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa59806aed0 0x7fa59806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.088+0000 7fa5bfe00640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa59806aed0 0x7fa59806d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8036000 tx=0x7fa5a803b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.092+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa5b00541f0 con 0x7fa5c00abcf0 2024-01-31T05:02:11.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.283+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5c009aeb0 con 0x7fa5c00abcf0 2024-01-31T05:02:11.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.284+0000 7fa5bcc00640 1 -- 172.21.15.97:0/2657176298 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5b0015020 con 0x7fa5c00abcf0 2024-01-31T05:02:11.283 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:11.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.286+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa59806aed0 msgr2=0x7fa59806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa59806aed0 0x7fa59806d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8036000 tx=0x7fa5a803b040 comp rx=0 tx=0).stop 2024-01-31T05:02:11.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 msgr2=0x7fa5c01f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c01f34d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000b3f0 tx=0x7fa5b000b8b0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 shutdown_connections 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c015b470 0x7fa5c00c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c00cb810 0x7fa5c00999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa59806aed0 0x7fa59806d390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 --2- 172.21.15.97:0/2657176298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c00abcf0 0x7fa5c01f34d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 >> 172.21.15.97:0/2657176298 conn(0x7fa5c00b0a10 msgr2=0x7fa5c00ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 shutdown_connections 2024-01-31T05:02:11.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.287+0000 7fa5c7600640 1 -- 172.21.15.97:0/2657176298 wait complete. 2024-01-31T05:02:11.303 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.513+0000 7fba1c000640 1 Processor -- start 2024-01-31T05:02:11.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.514+0000 7fba1c000640 1 -- start start 2024-01-31T05:02:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.515+0000 7fba1c000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba140b3e90 0x7fba140b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.515+0000 7fba1c000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba140d1b20 con 0x7fba140d3bb0 2024-01-31T05:02:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.515+0000 7fba1c000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba140d1e00 con 0x7fba140cfca0 2024-01-31T05:02:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.515+0000 7fba1c000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba140d20e0 con 0x7fba140b3e90 2024-01-31T05:02:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.515+0000 7fba18c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fba140d3bb0 0x7fba140ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47178/0 (socket says 172.21.15.97:47178) 2024-01-31T05:02:11.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.515+0000 7fba18c00640 1 -- 172.21.15.97:0/3279372454 learned_addr learned my addr 172.21.15.97:0/3279372454 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695582393 0 0) 0x7fba140d1b20 con 0x7fba140d3bb0 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f4003520 con 0x7fba140d3bb0 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba1a000640 1 --2- 172.21.15.97:0/3279372454 >> v2:172.21.15.188:3300/0 conn(0x7fba140b3e90 0x7fba140b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996393729 0 0) 0x7fb9f4003520 con 0x7fba140d3bb0 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 >> v2:172.21.15.188:3300/0 conn(0x7fba140b3e90 msgr2=0x7fba140b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 --2- 172.21.15.97:0/3279372454 >> v2:172.21.15.188:3300/0 conn(0x7fba140b3e90 0x7fba140b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 >> v1:172.21.15.188:6789/0 conn(0x7fba140cfca0 legacy=0x7fba140b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140d4120 con 0x7fba140d3bb0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fb9fc001670 con 0x7fba140d3bb0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 >> v1:172.21.15.97:6789/0 conn(0x7fba140d3bb0 legacy=0x7fba140ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f40039c0 0x7fb9f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.516+0000 7fba13e00640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f40051d0 0x7fb9f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba140d20e0 con 0x7fb9f40051d0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f400c120 con 0x7fb9f40039c0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f400c2e0 con 0x7fb9f40047c0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba1a000640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f40051d0 0x7fb9f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba18c00640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba1a000640 1 -- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40047c0 msgr2=0x7fb9f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba1a000640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40047c0 0x7fb9f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba19600640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f40039c0 0x7fb9f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba1a000640 1 -- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f40039c0 msgr2=0x7fb9f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba1a000640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9f40039c0 0x7fb9f4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.517+0000 7fba1a000640 1 -- 172.21.15.97:0/3279372454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4003520 con 0x7fb9f40051d0 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.518+0000 7fba1a000640 1 -- 172.21.15.97:0/3279372454 reap_dead start 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.518+0000 7fba1a000640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f40051d0 0x7fb9f400b980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba0c007360 tx=0x7fb9f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88045dcc8d4519ad server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c02ccf0 con 0x7fb9f40051d0 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c02ce90 con 0x7fb9f40051d0 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba13e00640 1 -- 172.21.15.97:0/3279372454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c003bd0 con 0x7fb9f40051d0 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 -- 172.21.15.97:0/3279372454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f40051d0 msgr2=0x7fb9f400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f40051d0 0x7fb9f400b980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fba0c007360 tx=0x7fb9f40057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 -- 172.21.15.97:0/3279372454 shutdown_connections 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 --2- 172.21.15.97:0/3279372454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9f40051d0 0x7fb9f400b980 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 -- 172.21.15.97:0/3279372454 >> 172.21.15.97:0/3279372454 conn(0x7fba1409c570 msgr2=0x7fba140d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:11.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 -- 172.21.15.97:0/3279372454 shutdown_connections 2024-01-31T05:02:11.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.519+0000 7fba1c000640 1 -- 172.21.15.97:0/3279372454 wait complete. 2024-01-31T05:02:11.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.521+0000 7fba1c000640 1 Processor -- start 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.521+0000 7fba1c000640 1 -- start start 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40051d0 0x7fba141515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1c000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba140c8dd0 0x7fba140c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1c000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 0x7fba140b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1c000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba140c9790 con 0x7fba140b0380 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1c000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1415a510 con 0x7fba140c8dd0 2024-01-31T05:02:11.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1415a7f0 con 0x7fb9f40051d0 2024-01-31T05:02:11.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.522+0000 7fba1a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 0x7fba140b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 0x7fba140b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45818/0 (socket says 172.21.15.97:45818) 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 -- 172.21.15.97:0/934951714 learned_addr learned my addr 172.21.15.97:0/934951714 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba19600640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40051d0 0x7fba141515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba18c00640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba140c8dd0 0x7fba140c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 -- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40051d0 msgr2=0x7fba141515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40051d0 0x7fba141515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 -- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba140c8dd0 msgr2=0x7fba140c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba140c8dd0 0x7fba140c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 -- 172.21.15.97:0/934951714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1415aad0 con 0x7fba140b0380 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.523+0000 7fba1a000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 0x7fba140b0800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fba0c007480 tx=0x7fba0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.524+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c004700 con 0x7fba140b0380 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.524+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c004860 con 0x7fba140b0380 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.524+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c03f6b0 con 0x7fba140b0380 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.524+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1415adb0 con 0x7fba140b0380 2024-01-31T05:02:11.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.524+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba14151b20 con 0x7fba140b0380 2024-01-31T05:02:11.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.525+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba140b00a0 con 0x7fba140b0380 2024-01-31T05:02:11.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.526+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fba0c034440 con 0x7fba140b0380 2024-01-31T05:02:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.527+0000 7fba11600640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9ec06af00 0x7fb9ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.527+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fba0c03b070 con 0x7fba140b0380 2024-01-31T05:02:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.527+0000 7fba19600640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9ec06af00 0x7fb9ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.528+0000 7fba19600640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9ec06af00 0x7fb9ec06d3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba04001670 tx=0x7fba04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.531+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fba140b00a0 con 0x7fba140b0380 2024-01-31T05:02:11.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.722+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba140b00a0 con 0x7fba140b0380 2024-01-31T05:02:11.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.723+0000 7fba11600640 1 -- 172.21.15.97:0/934951714 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba140b00a0 con 0x7fba140b0380 2024-01-31T05:02:11.722 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.725+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9ec06af00 msgr2=0x7fb9ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.725+0000 7fba1c000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9ec06af00 0x7fb9ec06d3c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fba04001670 tx=0x7fba04009040 comp rx=0 tx=0).stop 2024-01-31T05:02:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.725+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 msgr2=0x7fba140b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.725+0000 7fba1c000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 0x7fba140b0800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fba0c007480 tx=0x7fba0c004ab0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 shutdown_connections 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9ec06af00 0x7fb9ec06d3c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba140b0380 0x7fba140b0800 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba140c8dd0 0x7fba140c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 --2- 172.21.15.97:0/934951714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9f40051d0 0x7fba141515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 >> 172.21.15.97:0/934951714 conn(0x7fba1409c570 msgr2=0x7fba1406c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 shutdown_connections 2024-01-31T05:02:11.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.726+0000 7fba1c000640 1 -- 172.21.15.97:0/934951714 wait complete. 2024-01-31T05:02:11.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:11.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.957+0000 7efd8f400640 1 Processor -- start 2024-01-31T05:02:11.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.957+0000 7efd8f400640 1 -- start start 2024-01-31T05:02:11.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 0x7efd880b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8f400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd880b3f60 con 0x7efd8815b080 2024-01-31T05:02:11.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8f400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8815c540 con 0x7efd880ae360 2024-01-31T05:02:11.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880b41c0 con 0x7efd880ce680 2024-01-31T05:02:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8e000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efd8815b080 0x7efd880cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47180/0 (socket says 172.21.15.97:47180) 2024-01-31T05:02:11.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8e000640 1 -- 172.21.15.97:0/2237560151 learned_addr learned my addr 172.21.15.97:0/2237560151 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:11.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073743634 0 0) 0x7efd880b3f60 con 0x7efd8815b080 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd68003520 con 0x7efd8815b080 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.959+0000 7efd8d600640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 0x7efd880b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1000317026 0 0) 0x7efd68003520 con 0x7efd8815b080 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 msgr2=0x7efd880b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 0x7efd880b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 >> v1:172.21.15.188:6789/0 conn(0x7efd880ae360 legacy=0x7efd880b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8815b780 con 0x7efd8815b080 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7efd78001670 con 0x7efd8815b080 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 >> v1:172.21.15.97:6789/0 conn(0x7efd8815b080 legacy=0x7efd880cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd68003a10 0x7efd68003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd680047c0 0x7efd68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 0x7efd6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd880b41c0 con 0x7efd680047c0 2024-01-31T05:02:11.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd680054e0 con 0x7efd68003a10 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6800bec0 con 0x7efd880ce680 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.960+0000 7efd8d600640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd680047c0 0x7efd68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.961+0000 7efd8d600640 1 -- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 msgr2=0x7efd6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.961+0000 7efd8d600640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd880ce680 0x7efd6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.961+0000 7efd8e000640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd68003a10 0x7efd68003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.961+0000 7efd8d600640 1 -- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd68003a10 msgr2=0x7efd68003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.961+0000 7efd8d600640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd68003a10 0x7efd68003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.961+0000 7efd8d600640 1 -- 172.21.15.97:0/2237560151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68003520 con 0x7efd680047c0 2024-01-31T05:02:11.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.962+0000 7efd8d600640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd680047c0 0x7efd68004c40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efd7000ba30 tx=0x7efd680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72882d76cad85fb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.962+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70032660 con 0x7efd680047c0 2024-01-31T05:02:11.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.962+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd700327c0 con 0x7efd680047c0 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.962+0000 7efd8cc00640 1 -- 172.21.15.97:0/2237560151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70032b50 con 0x7efd680047c0 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8f400640 1 -- 172.21.15.97:0/2237560151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd680047c0 msgr2=0x7efd68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8f400640 1 --2- 172.21.15.97:0/2237560151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd680047c0 0x7efd68004c40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efd7000ba30 tx=0x7efd680057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8ea00640 1 -- 172.21.15.97:0/2237560151 reap_dead start 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8f400640 1 -- 172.21.15.97:0/2237560151 shutdown_connections 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8f400640 1 -- 172.21.15.97:0/2237560151 >> 172.21.15.97:0/2237560151 conn(0x7efd8806ad40 msgr2=0x7efd880b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8f400640 1 -- 172.21.15.97:0/2237560151 shutdown_connections 2024-01-31T05:02:11.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.963+0000 7efd8f400640 1 -- 172.21.15.97:0/2237560151 wait complete. 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.965+0000 7efd8f400640 1 Processor -- start 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.965+0000 7efd8f400640 1 -- start start 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd680047c0 0x7efd880999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd8814fb50 0x7efd88142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 0x7efd881513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88099fa0 con 0x7efd8809b040 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8f400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8809a280 con 0x7efd8814fb50 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8809a560 con 0x7efd680047c0 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 0x7efd881513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 0x7efd881513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45834/0 (socket says 172.21.15.97:45834) 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 -- 172.21.15.97:0/2859130959 learned_addr learned my addr 172.21.15.97:0/2859130959 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8e000640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd680047c0 0x7efd880999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8d600640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd8814fb50 0x7efd88142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 -- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd680047c0 msgr2=0x7efd880999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd680047c0 0x7efd880999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 -- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd8814fb50 msgr2=0x7efd88142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd8814fb50 0x7efd88142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:11.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.966+0000 7efd8ea00640 1 -- 172.21.15.97:0/2859130959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8809a840 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.967+0000 7efd8ea00640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 0x7efd881513b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efd88072650 tx=0x7efd8000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.967+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80005d40 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.967+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8000ced0 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.967+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8809ab20 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.967+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8000a850 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.967+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88142aa0 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.968+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd880676b0 con 0x7efd8809b040 2024-01-31T05:02:11.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.969+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7efd8000a9b0 con 0x7efd8809b040 2024-01-31T05:02:11.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.970+0000 7efd86000640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd6006aed0 0x7efd6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:11.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.970+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7efd80088ea0 con 0x7efd8809b040 2024-01-31T05:02:11.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.970+0000 7efd8e000640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd6006aed0 0x7efd6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:11.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.971+0000 7efd8e000640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd6006aed0 0x7efd6006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efd78000ff0 tx=0x7efd78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:11.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:11.975+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7efd80053980 con 0x7efd8809b040 2024-01-31T05:02:12.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.170+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd880b0ab0 con 0x7efd8809b040 2024-01-31T05:02:12.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.171+0000 7efd86000640 1 -- 172.21.15.97:0/2859130959 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd80017020 con 0x7efd8809b040 2024-01-31T05:02:12.171 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd6006aed0 msgr2=0x7efd6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd6006aed0 0x7efd6006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efd78000ff0 tx=0x7efd78009040 comp rx=0 tx=0).stop 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 msgr2=0x7efd881513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 0x7efd881513b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efd88072650 tx=0x7efd8000cb90 comp rx=0 tx=0).stop 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 shutdown_connections 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd6006aed0 0x7efd6006d390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd8809b040 0x7efd881513b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd8814fb50 0x7efd88142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 --2- 172.21.15.97:0/2859130959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd680047c0 0x7efd880999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 >> 172.21.15.97:0/2859130959 conn(0x7efd8806ad40 msgr2=0x7efd88153250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 shutdown_connections 2024-01-31T05:02:12.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.174+0000 7efd8f400640 1 -- 172.21.15.97:0/2859130959 wait complete. 2024-01-31T05:02:12.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:02:12.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.393+0000 7ff6a1800640 1 Processor -- start 2024-01-31T05:02:12.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.393+0000 7ff6a1800640 1 -- start start 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.394+0000 7ff6a1800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff69c0ce1e0 0x7ff69c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 0x7ff69c0cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff6a1800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c161fc0 con 0x7ff69c161b10 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff6a1800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c162c40 con 0x7ff69c0ce1e0 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff6a1800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1622a0 con 0x7ff69c0cb7d0 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff69a000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff69c161b10 0x7ff69c0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47192/0 (socket says 172.21.15.97:47192) 2024-01-31T05:02:12.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff69a000640 1 -- 172.21.15.97:0/1759783466 learned_addr learned my addr 172.21.15.97:0/1759783466 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699785310 0 0) 0x7ff69c161fc0 con 0x7ff69c161b10 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff67c003520 con 0x7ff69c161b10 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff69b400640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 0x7ff69c0cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.395+0000 7ff69aa00640 1 --2- 172.21.15.97:0/1759783466 >> v2:172.21.15.188:3300/0 conn(0x7ff69c0ce1e0 0x7ff69c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897222850 0 0) 0x7ff67c003520 con 0x7ff69c161b10 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 msgr2=0x7ff69c0cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 0x7ff69c0cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 >> v2:172.21.15.188:3300/0 conn(0x7ff69c0ce1e0 msgr2=0x7ff69c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 --2- 172.21.15.97:0/1759783466 >> v2:172.21.15.188:3300/0 conn(0x7ff69c0ce1e0 0x7ff69c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c162580 con 0x7ff69c161b10 2024-01-31T05:02:12.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff684001670 con 0x7ff69c161b10 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff69aa00640 1 --2- 172.21.15.97:0/1759783466 >> v2:172.21.15.188:3300/0 conn(0x7ff69c0ce1e0 0x7ff69c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 >> v1:172.21.15.97:6789/0 conn(0x7ff69c161b10 legacy=0x7ff69c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c003970 0x7ff67c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 0x7ff67c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67c004d00 0x7ff67c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c162c40 con 0x7ff67c004d00 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1622a0 con 0x7ff67c003970 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c00bc10 con 0x7ff69c0cb7d0 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff69b400640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67c004d00 0x7ff67c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff69aa00640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c003970 0x7ff67c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.396+0000 7ff69a000640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 0x7ff67c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.397+0000 7ff69b400640 1 -- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 msgr2=0x7ff67c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.397+0000 7ff69b400640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c0cb7d0 0x7ff67c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.397+0000 7ff69b400640 1 -- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c003970 msgr2=0x7ff67c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.397+0000 7ff69b400640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c003970 0x7ff67c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.397+0000 7ff69b400640 1 -- 172.21.15.97:0/1759783466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c003520 con 0x7ff67c004d00 2024-01-31T05:02:12.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff69b400640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67c004d00 0x7ff67c005180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff694007360 tx=0x7ff67c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8463295d02d6df39 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69402ccd0 con 0x7ff67c004d00 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69402ce30 con 0x7ff67c004d00 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff699600640 1 -- 172.21.15.97:0/1759783466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694034660 con 0x7ff67c004d00 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff6a1800640 1 -- 172.21.15.97:0/1759783466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67c004d00 msgr2=0x7ff67c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff6a1800640 1 --2- 172.21.15.97:0/1759783466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff67c004d00 0x7ff67c005180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff694007360 tx=0x7ff67c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.398+0000 7ff69b400640 1 -- 172.21.15.97:0/1759783466 reap_dead start 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.399+0000 7ff6a1800640 1 -- 172.21.15.97:0/1759783466 shutdown_connections 2024-01-31T05:02:12.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.399+0000 7ff6a1800640 1 -- 172.21.15.97:0/1759783466 >> 172.21.15.97:0/1759783466 conn(0x7ff69c0b0a10 msgr2=0x7ff69c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.399+0000 7ff6a1800640 1 -- 172.21.15.97:0/1759783466 shutdown_connections 2024-01-31T05:02:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.399+0000 7ff6a1800640 1 -- 172.21.15.97:0/1759783466 wait complete. 2024-01-31T05:02:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.400+0000 7ff6a1800640 1 Processor -- start 2024-01-31T05:02:12.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.401+0000 7ff6a1800640 1 -- start start 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.401+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c004d00 0x7ff69c164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.401+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c099b50 0x7ff69c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff6a1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 0x7ff69c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff6a1800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1648b0 con 0x7ff69c1632c0 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff6a1800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c163ca0 con 0x7ff67c004d00 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff6a1800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c163f80 con 0x7ff69c099b50 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 0x7ff69c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 0x7ff69c0ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45848/0 (socket says 172.21.15.97:45848) 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 -- 172.21.15.97:0/3191896149 learned_addr learned my addr 172.21.15.97:0/3191896149 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69a000640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c099b50 0x7ff69c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69aa00640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c004d00 0x7ff69c164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 -- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c099b50 msgr2=0x7ff69c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c099b50 0x7ff69c151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 -- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c004d00 msgr2=0x7ff69c164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c004d00 0x7ff69c164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69b400640 1 -- 172.21.15.97:0/3191896149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c164260 con 0x7ff69c1632c0 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.402+0000 7ff69aa00640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c004d00 0x7ff69c164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.403+0000 7ff69b400640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 0x7ff69c0ae830 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff694037040 tx=0x7ff694004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.403+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69403e600 con 0x7ff69c1632c0 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.403+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69403e780 con 0x7ff69c1632c0 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.403+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c164540 con 0x7ff69c1632c0 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.403+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69403eb30 con 0x7ff69c1632c0 2024-01-31T05:02:12.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.403+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c163750 con 0x7ff69c1632c0 2024-01-31T05:02:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.404+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c165280 con 0x7ff69c1632c0 2024-01-31T05:02:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.405+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff69403ec90 con 0x7ff69c1632c0 2024-01-31T05:02:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.406+0000 7ff692a00640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff67406aed0 0x7ff67406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.406+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff6940abee0 con 0x7ff69c1632c0 2024-01-31T05:02:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.407+0000 7ff69aa00640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff67406aed0 0x7ff67406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.407+0000 7ff69aa00640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff67406aed0 0x7ff67406d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:12.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.411+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff694076930 con 0x7ff69c1632c0 2024-01-31T05:02:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.602+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff69c1646d0 con 0x7ff69c1632c0 2024-01-31T05:02:12.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.603+0000 7ff692a00640 1 -- 172.21.15.97:0/3191896149 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff694039070 con 0x7ff69c1632c0 2024-01-31T05:02:12.603 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff67406aed0 msgr2=0x7ff67406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff67406aed0 0x7ff67406d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c008040 comp rx=0 tx=0).stop 2024-01-31T05:02:12.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 msgr2=0x7ff69c0ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 0x7ff69c0ae830 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff694037040 tx=0x7ff694004ab0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 shutdown_connections 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff67406aed0 0x7ff67406d390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff69c1632c0 0x7ff69c0ae830 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff69c099b50 0x7ff69c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 --2- 172.21.15.97:0/3191896149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff67c004d00 0x7ff69c164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.605+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 >> 172.21.15.97:0/3191896149 conn(0x7ff69c0b0a10 msgr2=0x7ff69c0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.606+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 shutdown_connections 2024-01-31T05:02:12.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.606+0000 7ff6a1800640 1 -- 172.21.15.97:0/3191896149 wait complete. 2024-01-31T05:02:12.624 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:12.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:02:12.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:02:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.821+0000 7fdfa5e00640 1 Processor -- start 2024-01-31T05:02:12.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.821+0000 7fdfa5e00640 1 -- start start 2024-01-31T05:02:12.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.823+0000 7fdfa5e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdfa015d630 0x7fdfa006d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.823+0000 7fdfa5e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa015eb90 con 0x7fdfa015da10 2024-01-31T05:02:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.823+0000 7fdfa5e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa015df80 con 0x7fdfa006ae20 2024-01-31T05:02:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.823+0000 7fdfa5e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa015e260 con 0x7fdfa015d630 2024-01-31T05:02:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.823+0000 7fdf95e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdfa015da10 0x7fdfa006a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47204/0 (socket says 172.21.15.97:47204) 2024-01-31T05:02:12.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.823+0000 7fdf95e00640 1 -- 172.21.15.97:0/3026778976 learned_addr learned my addr 172.21.15.97:0/3026778976 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:12.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/3026778976 >> v2:172.21.15.188:3300/0 conn(0x7fdfa015d630 0x7fdfa006d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002898084 0 0) 0x7fdfa015eb90 con 0x7fdfa015da10 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf80003520 con 0x7fdfa015da10 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2063827189 0 0) 0x7fdf80003520 con 0x7fdfa015da10 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 >> v2:172.21.15.188:3300/0 conn(0x7fdfa015d630 msgr2=0x7fdfa006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 --2- 172.21.15.97:0/3026778976 >> v2:172.21.15.188:3300/0 conn(0x7fdfa015d630 0x7fdfa006d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 >> v1:172.21.15.188:6789/0 conn(0x7fdfa006ae20 legacy=0x7fdfa00717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa015e540 con 0x7fdfa015da10 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fdf88001670 con 0x7fdfa015da10 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 >> v1:172.21.15.97:6789/0 conn(0x7fdfa015da10 legacy=0x7fdfa006a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf80003970 0x7fdf80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf800047c0 0x7fdf80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf80005180 0x7fdf8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa015e260 con 0x7fdf80003970 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8000c020 con 0x7fdf800047c0 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.824+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8000c1e0 con 0x7fdf80005180 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf80003970 0x7fdf80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9ea00640 1 -- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf80005180 msgr2=0x7fdf8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf80005180 0x7fdf8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf95e00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf800047c0 0x7fdf80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9ea00640 1 -- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf800047c0 msgr2=0x7fdf80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf800047c0 0x7fdf80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9ea00640 1 -- 172.21.15.97:0/3026778976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf80003520 con 0x7fdf80003970 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf9f400640 1 -- 172.21.15.97:0/3026778976 reap_dead start 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.825+0000 7fdf95e00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf800047c0 0x7fdf80004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.826+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf80003970 0x7fdf80003d90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009fa0 tx=0x7fdf800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ee203a3b53fc0e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:12.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.826+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9002ccb0 con 0x7fdf80003970 2024-01-31T05:02:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.826+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9000f040 con 0x7fdf80003970 2024-01-31T05:02:12.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.826+0000 7fdf9e000640 1 -- 172.21.15.97:0/3026778976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90034660 con 0x7fdf80003970 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 -- 172.21.15.97:0/3026778976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf80003970 msgr2=0x7fdf80003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf80003970 0x7fdf80003d90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009fa0 tx=0x7fdf800057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 -- 172.21.15.97:0/3026778976 shutdown_connections 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/3026778976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf80003970 0x7fdf80003d90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 -- 172.21.15.97:0/3026778976 >> 172.21.15.97:0/3026778976 conn(0x7fdfa00b1a90 msgr2=0x7fdfa00b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 -- 172.21.15.97:0/3026778976 shutdown_connections 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.827+0000 7fdfa5e00640 1 -- 172.21.15.97:0/3026778976 wait complete. 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.828+0000 7fdfa5e00640 1 Processor -- start 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.829+0000 7fdfa5e00640 1 -- start start 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.829+0000 7fdfa5e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80003970 0x7fdfa014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.829+0000 7fdfa5e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 0x7fdfa0152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.829+0000 7fdfa5e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfa015c6a0 0x7fdfa00ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdfa5e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa014af90 con 0x7fdfa0153ab0 2024-01-31T05:02:12.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdfa5e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0152740 con 0x7fdf80003970 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdfa5e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0152a10 con 0x7fdfa015c6a0 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 0x7fdfa0152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 0x7fdfa0152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45862/0 (socket says 172.21.15.97:45862) 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 -- 172.21.15.97:0/1755738668 learned_addr learned my addr 172.21.15.97:0/1755738668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80003970 0x7fdfa014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 -- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfa015c6a0 msgr2=0x7fdfa00ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfa015c6a0 0x7fdfa00ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 -- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80003970 msgr2=0x7fdfa014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80003970 0x7fdfa014aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 -- 172.21.15.97:0/1755738668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0152cf0 con 0x7fdfa0153ab0 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80003970 0x7fdfa014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf95e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 0x7fdfa0152200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800d7e0 tx=0x7fdf8800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:12.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.830+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf880074e0 con 0x7fdfa0153ab0 2024-01-31T05:02:12.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.831+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88007640 con 0x7fdfa0153ab0 2024-01-31T05:02:12.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.831+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88004a00 con 0x7fdfa0153ab0 2024-01-31T05:02:12.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.831+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa0152fd0 con 0x7fdfa0153ab0 2024-01-31T05:02:12.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.831+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa0153590 con 0x7fdfa0153ab0 2024-01-31T05:02:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.832+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa0072c30 con 0x7fdfa0153ab0 2024-01-31T05:02:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.833+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fdf88004b60 con 0x7fdfa0153ab0 2024-01-31T05:02:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.834+0000 7fdf97e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdf7806aed0 0x7fdf7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.834+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdf88089230 con 0x7fdfa0153ab0 2024-01-31T05:02:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.834+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdf7806aed0 0x7fdf7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:12.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.835+0000 7fdf9ea00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdf7806aed0 0x7fdf7806d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf90036000 tx=0x7fdf9002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:12.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:12.838+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fdf88053e40 con 0x7fdfa0153ab0 2024-01-31T05:02:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.046+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdfa0067830 con 0x7fdfa0153ab0 2024-01-31T05:02:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.047+0000 7fdf97e00640 1 -- 172.21.15.97:0/1755738668 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdf88057460 con 0x7fdfa0153ab0 2024-01-31T05:02:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdf7806aed0 msgr2=0x7fdf7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdf7806aed0 0x7fdf7806d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf90036000 tx=0x7fdf9002e000 comp rx=0 tx=0).stop 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 msgr2=0x7fdfa0152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 0x7fdfa0152200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800d7e0 tx=0x7fdf8800dca0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 shutdown_connections 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfa015c6a0 0x7fdfa00ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdf7806aed0 0x7fdf7806d390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfa0153ab0 0x7fdfa0152200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 --2- 172.21.15.97:0/1755738668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80003970 0x7fdfa014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 >> 172.21.15.97:0/1755738668 conn(0x7fdfa00b1a90 msgr2=0x7fdfa00ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 shutdown_connections 2024-01-31T05:02:13.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.050+0000 7fdfa5e00640 1 -- 172.21.15.97:0/1755738668 wait complete. 2024-01-31T05:02:13.067 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi188.front.sepia.ceph.com... 2024-01-31T05:02:13.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:13.067 DEBUG:teuthology.orchestra.run.smithi188:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-01-31T05:02:13.166 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:13.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:02:13.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:13.166 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-31T05:02:13.214 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-31T05:02:13.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:13.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.425+0000 7f49a2a00640 1 Processor -- start 2024-01-31T05:02:13.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.425+0000 7f49a2a00640 1 -- start start 2024-01-31T05:02:13.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c15b400 0x7f499c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a2a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f499c06db70 0x7f499c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a2a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c06d1a0 con 0x7f499c0cb810 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a2a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c0cc960 con 0x7f499c06db70 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a2a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c06c600 con 0x7f499c15b400 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a0a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f499c06db70 0x7f499c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a0a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f499c06db70 0x7f499c0ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45864/0 (socket says 172.21.15.97:45864) 2024-01-31T05:02:13.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.427+0000 7f49a0a00640 1 -- 172.21.15.97:0/3819425850 learned_addr learned my addr 172.21.15.97:0/3819425850 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:13.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.428+0000 7f49a0a00640 1 -- 172.21.15.97:0/3819425850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c15b400 msgr2=0x7f499c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:13.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.428+0000 7f49a0a00640 1 --2- 172.21.15.97:0/3819425850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c15b400 0x7f499c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.428+0000 7f49a0a00640 1 -- 172.21.15.97:0/3819425850 >> v1:172.21.15.97:6789/0 conn(0x7f499c0cb810 legacy=0x7f499c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.428+0000 7f49a0a00640 1 -- 172.21.15.97:0/3819425850 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c06c870 con 0x7f499c06db70 2024-01-31T05:02:13.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.429+0000 7f49a0a00640 1 --2- 172.21.15.97:0/3819425850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c06db70 0x7f499c0ccd00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4994009e50 tx=0x7f499402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8d6e0ed43536fb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:13.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.429+0000 7f499b400640 1 -- 172.21.15.97:0/3819425850 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499402cb00 con 0x7f499c06db70 2024-01-31T05:02:13.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.429+0000 7f499b400640 1 -- 172.21.15.97:0/3819425850 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499402cc60 con 0x7f499c06db70 2024-01-31T05:02:13.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.429+0000 7f499b400640 1 -- 172.21.15.97:0/3819425850 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940343f0 con 0x7f499c06db70 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 -- 172.21.15.97:0/3819425850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c06db70 msgr2=0x7f499c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 --2- 172.21.15.97:0/3819425850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c06db70 0x7f499c0ccd00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4994009e50 tx=0x7f499402bf60 comp rx=0 tx=0).stop 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 -- 172.21.15.97:0/3819425850 shutdown_connections 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 --2- 172.21.15.97:0/3819425850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c06db70 0x7f499c0ccd00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 --2- 172.21.15.97:0/3819425850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c15b400 0x7f499c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 -- 172.21.15.97:0/3819425850 >> 172.21.15.97:0/3819425850 conn(0x7f499c0b0a10 msgr2=0x7f499c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 -- 172.21.15.97:0/3819425850 shutdown_connections 2024-01-31T05:02:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.430+0000 7f49a2a00640 1 -- 172.21.15.97:0/3819425850 wait complete. 2024-01-31T05:02:13.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.432+0000 7f49a2a00640 1 Processor -- start 2024-01-31T05:02:13.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.432+0000 7f49a2a00640 1 -- start start 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c06db70 0x7f499c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f499c0cb810 0x7f499c1519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 0x7f499c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a2a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c151f10 con 0x7f499c15b400 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a2a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c15bd50 con 0x7f499c0cb810 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a2a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c15bff0 con 0x7f499c06db70 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 0x7f499c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 0x7f499c15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45878/0 (socket says 172.21.15.97:45878) 2024-01-31T05:02:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 -- 172.21.15.97:0/1931883808 learned_addr learned my addr 172.21.15.97:0/1931883808 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 -- 172.21.15.97:0/1931883808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c06db70 msgr2=0x7f499c154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c06db70 0x7f499c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 -- 172.21.15.97:0/1931883808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f499c0cb810 msgr2=0x7f499c1519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f499c0cb810 0x7f499c1519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.433+0000 7f49a0a00640 1 -- 172.21.15.97:0/1931883808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009b30 con 0x7f499c15b400 2024-01-31T05:02:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.434+0000 7f49a0a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 0x7f499c15b810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4994036040 tx=0x7f4994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.434+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499403e450 con 0x7f499c15b400 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.434+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499403e5b0 con 0x7f499c15b400 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.434+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c15c2d0 con 0x7f499c15b400 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.434+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499403e960 con 0x7f499c15b400 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.434+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c15c890 con 0x7f499c15b400 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.435+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c0cb680 con 0x7f499c15b400 2024-01-31T05:02:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.436+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f499403eac0 con 0x7f499c15b400 2024-01-31T05:02:13.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.437+0000 7f4998c00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f497406aed0 0x7f497406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.437+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f49940ac860 con 0x7f499c15b400 2024-01-31T05:02:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.437+0000 7f499be00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f497406aed0 0x7f497406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.438+0000 7f499be00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f497406aed0 0x7f497406d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f498c000f60 tx=0x7f498c0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:13.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.441+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f49940773d0 con 0x7f499c15b400 2024-01-31T05:02:13.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.676+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f499c0b3a80 con 0x7f499c15b400 2024-01-31T05:02:13.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.677+0000 7f4998c00640 1 -- 172.21.15.97:0/1931883808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f499407a9f0 con 0x7f499c15b400 2024-01-31T05:02:13.677 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:13.677 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:02:13.677 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:02:13.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.679+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f497406aed0 msgr2=0x7f497406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.679+0000 7f49a2a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f497406aed0 0x7f497406d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f498c000f60 tx=0x7f498c0079d0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.679+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 msgr2=0x7f499c15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.679+0000 7f49a2a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 0x7f499c15b810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4994036040 tx=0x7f4994004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 shutdown_connections 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f497406aed0 0x7f497406d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f499c15b400 0x7f499c15b810 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f499c0cb810 0x7f499c1519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 --2- 172.21.15.97:0/1931883808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f499c06db70 0x7f499c154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 >> 172.21.15.97:0/1931883808 conn(0x7f499c0b0a10 msgr2=0x7f499c0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 shutdown_connections 2024-01-31T05:02:13.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.680+0000 7f49a2a00640 1 -- 172.21.15.97:0/1931883808 wait complete. 2024-01-31T05:02:13.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:13.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.901+0000 7f0109000640 1 Processor -- start 2024-01-31T05:02:13.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.901+0000 7f0109000640 1 -- start start 2024-01-31T05:02:13.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.902+0000 7f0109000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01040ae3b0 0x7f010406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.902+0000 7f0109000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0104070400 con 0x7f010415ec70 2024-01-31T05:02:13.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.902+0000 7f0109000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01040ca580 con 0x7f010406d0d0 2024-01-31T05:02:13.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.902+0000 7f0109000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01040ad8c0 con 0x7f01040ae3b0 2024-01-31T05:02:13.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0103400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f010415ec70 0x7f0104071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47210/0 (socket says 172.21.15.97:47210) 2024-01-31T05:02:13.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0103400640 1 -- 172.21.15.97:0/206873001 learned_addr learned my addr 172.21.15.97:0/206873001 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:13.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222109112 0 0) 0x7f0104070400 con 0x7f010415ec70 2024-01-31T05:02:13.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e4003520 con 0x7f010415ec70 2024-01-31T05:02:13.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102a00640 1 --2- 172.21.15.97:0/206873001 >> v2:172.21.15.188:3300/0 conn(0x7f01040ae3b0 0x7f010406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2538700898 0 0) 0x7f00e4003520 con 0x7f010415ec70 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 >> v2:172.21.15.188:3300/0 conn(0x7f01040ae3b0 msgr2=0x7f010406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 --2- 172.21.15.97:0/206873001 >> v2:172.21.15.188:3300/0 conn(0x7f01040ae3b0 0x7f010406cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 >> v1:172.21.15.188:6789/0 conn(0x7f010406d0d0 legacy=0x7f010415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.903+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104070650 con 0x7f010415ec70 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f00f4001670 con 0x7f010415ec70 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 >> v1:172.21.15.97:6789/0 conn(0x7f010415ec70 legacy=0x7f0104071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e4003970 0x7f00e4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e40047c0 0x7f00e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e4005380 0x7f00e400b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01040ad8c0 con 0x7f00e40047c0 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e400c2c0 con 0x7f00e4005380 2024-01-31T05:02:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e400c480 con 0x7f00e4003970 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102a00640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e40047c0 0x7f00e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0103400640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e4003970 0x7f00e4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102a00640 1 -- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e4003970 msgr2=0x7f00e4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102a00640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e4003970 0x7f00e4003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102a00640 1 -- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e4005380 msgr2=0x7f00e400b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0103e00640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e4005380 0x7f00e400b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102a00640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e4005380 0x7f00e400b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0102a00640 1 -- 172.21.15.97:0/206873001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4003520 con 0x7f00e40047c0 2024-01-31T05:02:13.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.904+0000 7f0103e00640 1 -- 172.21.15.97:0/206873001 reap_dead start 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0102a00640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e40047c0 0x7f00e4004c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00a9a0 tx=0x7f00e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e50706a6d2269 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec030400 con 0x7f00e40047c0 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec030560 con 0x7f00e40047c0 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0102000640 1 -- 172.21.15.97:0/206873001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec030950 con 0x7f00e40047c0 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0109000640 1 -- 172.21.15.97:0/206873001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e40047c0 msgr2=0x7f00e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0109000640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e40047c0 0x7f00e4004c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00a9a0 tx=0x7f00e40057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0109000640 1 -- 172.21.15.97:0/206873001 shutdown_connections 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0109000640 1 --2- 172.21.15.97:0/206873001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00e40047c0 0x7f00e4004c40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.906+0000 7f0109000640 1 -- 172.21.15.97:0/206873001 >> 172.21.15.97:0/206873001 conn(0x7f01040cec40 msgr2=0x7f01040cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.907+0000 7f0109000640 1 -- 172.21.15.97:0/206873001 shutdown_connections 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.907+0000 7f0109000640 1 -- 172.21.15.97:0/206873001 wait complete. 2024-01-31T05:02:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.908+0000 7f0109000640 1 Processor -- start 2024-01-31T05:02:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.908+0000 7f0109000640 1 -- start start 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.909+0000 7f0109000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e40047c0 0x7f010415da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.909+0000 7f0109000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01041f3c10 0x7f010415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.909+0000 7f0109000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 0x7f01040b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.909+0000 7f0109000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01041f3780 con 0x7f01040b25e0 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.909+0000 7f0109000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010415e4b0 con 0x7f01041f3c10 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.909+0000 7f0109000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01041f2ec0 con 0x7f00e40047c0 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 0x7f01040b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 0x7f01040b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45910/0 (socket says 172.21.15.97:45910) 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 -- 172.21.15.97:0/3856948974 learned_addr learned my addr 172.21.15.97:0/3856948974 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103400640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e40047c0 0x7f010415da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 -- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e40047c0 msgr2=0x7f010415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e40047c0 0x7f010415da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0102a00640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01041f3c10 0x7f010415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 -- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01041f3c10 msgr2=0x7f010415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01041f3c10 0x7f010415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 -- 172.21.15.97:0/3856948974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01041f3130 con 0x7f01040b25e0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.910+0000 7f0103e00640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 0x7f01040b2a90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01040cfa10 tx=0x7f00fc00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.911+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc012800 con 0x7f01040b25e0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.911+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00fc012960 con 0x7f01040b25e0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.911+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc012d10 con 0x7f01040b25e0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.911+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01041f3410 con 0x7f01040b25e0 2024-01-31T05:02:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.911+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01040b2fd0 con 0x7f01040b25e0 2024-01-31T05:02:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.912+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01041f3930 con 0x7f01040b25e0 2024-01-31T05:02:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.913+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f00fc01e460 con 0x7f01040b25e0 2024-01-31T05:02:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.914+0000 7f00fb400640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00dc06af00 0x7f00dc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.914+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f00fc089750 con 0x7f01040b25e0 2024-01-31T05:02:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.914+0000 7f0103400640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00dc06af00 0x7f00dc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.915+0000 7f0103400640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00dc06af00 0x7f00dc06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00f4001eb0 tx=0x7f00f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:13.918+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f01041f3930 con 0x7f01040b25e0 2024-01-31T05:02:14.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.113+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01041f3930 con 0x7f01040b25e0 2024-01-31T05:02:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.115+0000 7f00fb400640 1 -- 172.21.15.97:0/3856948974 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f01041f3930 con 0x7f01040b25e0 2024-01-31T05:02:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:14.115 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00dc06af00 msgr2=0x7f00dc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00dc06af00 0x7f00dc06d3c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00f4001eb0 tx=0x7f00f4009040 comp rx=0 tx=0).stop 2024-01-31T05:02:14.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 msgr2=0x7f01040b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 0x7f01040b2a90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01040cfa10 tx=0x7f00fc00d960 comp rx=0 tx=0).stop 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 shutdown_connections 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00dc06af00 0x7f00dc06d3c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01040b25e0 0x7f01040b2a90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.117+0000 7f0109000640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01041f3c10 0x7f010415df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.118+0000 7f0109000640 1 --2- 172.21.15.97:0/3856948974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e40047c0 0x7f010415da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.118+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 >> 172.21.15.97:0/3856948974 conn(0x7f01040cec40 msgr2=0x7f0104071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.118+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 shutdown_connections 2024-01-31T05:02:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.118+0000 7f0109000640 1 -- 172.21.15.97:0/3856948974 wait complete. 2024-01-31T05:02:14.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:00:44.317943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:00:45.040037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-01-31T05:00:48.648377+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-01-31T05:02:14.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:14.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.353+0000 7f5093e00640 1 Processor -- start 2024-01-31T05:02:14.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.353+0000 7f5093e00640 1 -- start start 2024-01-31T05:02:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.354+0000 7f5093e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 0x7f508c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.354+0000 7f5093e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c15db40 con 0x7f508c06d580 2024-01-31T05:02:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.354+0000 7f5093e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c15de20 con 0x7f508c161020 2024-01-31T05:02:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.354+0000 7f5093e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15e7c0 con 0x7f508c15d690 2024-01-31T05:02:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.355+0000 7f5091400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f508c06d580 0x7f508c06d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47212/0 (socket says 172.21.15.97:47212) 2024-01-31T05:02:14.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.355+0000 7f5091400640 1 -- 172.21.15.97:0/3052588257 learned_addr learned my addr 172.21.15.97:0/3052588257 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:14.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.355+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909476319 0 0) 0x7f508c15db40 con 0x7f508c06d580 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.355+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f506c003520 con 0x7f508c06d580 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.355+0000 7f5090a00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 0x7f508c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.355+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3663047413 0 0) 0x7f506c003520 con 0x7f508c06d580 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 msgr2=0x7f508c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 0x7f508c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 >> v1:172.21.15.188:6789/0 conn(0x7f508c161020 legacy=0x7f508c1614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c15e100 con 0x7f508c06d580 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f507c0017f0 con 0x7f508c06d580 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 >> v1:172.21.15.97:6789/0 conn(0x7f508c06d580 legacy=0x7f508c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0039a0 0x7f506c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 0x7f506c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c004d00 0x7f506c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15e7c0 con 0x7f506c004d00 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c00bab0 con 0x7f506c0039a0 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c00bc70 con 0x7f508c15d690 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f5091e00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c004d00 0x7f506c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f5090a00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 0x7f506c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.356+0000 7f5091400640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0039a0 0x7f506c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.357+0000 7f5091e00640 1 -- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 msgr2=0x7f506c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.357+0000 7f5091e00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c15d690 0x7f506c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.357+0000 7f5091e00640 1 -- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0039a0 msgr2=0x7f506c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.357+0000 7f5091e00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0039a0 0x7f506c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.357+0000 7f5091e00640 1 -- 172.21.15.97:0/3052588257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c003520 con 0x7f506c004d00 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.358+0000 7f5091e00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c004d00 0x7f506c005180 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5084003db0 tx=0x7f506c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9c16426e85ea87 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.358+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084032600 con 0x7f506c004d00 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.358+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508402e050 con 0x7f506c004d00 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.358+0000 7f508be00640 1 -- 172.21.15.97:0/3052588257 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840329a0 con 0x7f506c004d00 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5093e00640 1 -- 172.21.15.97:0/3052588257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c004d00 msgr2=0x7f506c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5093e00640 1 --2- 172.21.15.97:0/3052588257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c004d00 0x7f506c005180 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5084003db0 tx=0x7f506c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5091e00640 1 -- 172.21.15.97:0/3052588257 reap_dead start 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5093e00640 1 -- 172.21.15.97:0/3052588257 shutdown_connections 2024-01-31T05:02:14.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5093e00640 1 -- 172.21.15.97:0/3052588257 >> 172.21.15.97:0/3052588257 conn(0x7f508c0b0a10 msgr2=0x7f508c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5093e00640 1 -- 172.21.15.97:0/3052588257 shutdown_connections 2024-01-31T05:02:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.359+0000 7f5093e00640 1 -- 172.21.15.97:0/3052588257 wait complete. 2024-01-31T05:02:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.360+0000 7f5093e00640 1 Processor -- start 2024-01-31T05:02:14.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.361+0000 7f5093e00640 1 -- start start 2024-01-31T05:02:14.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.361+0000 7f5093e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c004d00 0x7f508c070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.361+0000 7f5093e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 0x7f508c145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5093e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c0cf920 0x7f508c145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5093e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c09bb50 con 0x7f508c0cf4a0 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5093e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c145b70 con 0x7f506c004d00 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5093e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c145e50 con 0x7f508c0cf920 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 0x7f508c145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 0x7f508c145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45930/0 (socket says 172.21.15.97:45930) 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 -- 172.21.15.97:0/4135254566 learned_addr learned my addr 172.21.15.97:0/4135254566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 -- 172.21.15.97:0/4135254566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c0cf920 msgr2=0x7f508c145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c0cf920 0x7f508c145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 -- 172.21.15.97:0/4135254566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c004d00 msgr2=0x7f508c070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c004d00 0x7f508c070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 -- 172.21.15.97:0/4135254566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c09b770 con 0x7f508c0cf4a0 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.362+0000 7f5090a00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 0x7f508c145000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5074009fd0 tx=0x7f5074007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.363+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074009bd0 con 0x7f508c0cf4a0 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.363+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074015420 con 0x7f508c0cf4a0 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.363+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50740157d0 con 0x7f508c0cf4a0 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.363+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c099c60 con 0x7f508c0cf4a0 2024-01-31T05:02:14.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.363+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c09b390 con 0x7f508c0cf4a0 2024-01-31T05:02:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.364+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c09bdc0 con 0x7f508c0cf4a0 2024-01-31T05:02:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.365+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5074015930 con 0x7f508c0cf4a0 2024-01-31T05:02:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.366+0000 7f5089600640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f506406aed0 0x7f506406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.366+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f507408a480 con 0x7f508c0cf4a0 2024-01-31T05:02:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.366+0000 7f5091400640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f506406aed0 0x7f506406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.367+0000 7f5091400640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f506406aed0 0x7f506406d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f507c007960 tx=0x7f507c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:14.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.370+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5074054ff0 con 0x7f508c0cf4a0 2024-01-31T05:02:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.566+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f508c09b5b0 con 0x7f508c0cf4a0 2024-01-31T05:02:14.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.567+0000 7f5089600640 1 -- 172.21.15.97:0/4135254566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5074058610 con 0x7f508c0cf4a0 2024-01-31T05:02:14.566 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:14.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f506406aed0 msgr2=0x7f506406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f506406aed0 0x7f506406d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f507c007960 tx=0x7f507c009040 comp rx=0 tx=0).stop 2024-01-31T05:02:14.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 msgr2=0x7f508c145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 0x7f508c145000 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5074009fd0 tx=0x7f5074007700 comp rx=0 tx=0).stop 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 shutdown_connections 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f508c0cf920 0x7f508c145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f506406aed0 0x7f506406d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f508c0cf4a0 0x7f508c145000 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 --2- 172.21.15.97:0/4135254566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c004d00 0x7f508c070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.569+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 >> 172.21.15.97:0/4135254566 conn(0x7f508c0b0a10 msgr2=0x7f508c0af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.570+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 shutdown_connections 2024-01-31T05:02:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.570+0000 7f5093e00640 1 -- 172.21.15.97:0/4135254566 wait complete. 2024-01-31T05:02:14.590 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.793+0000 7f76e6c00640 1 Processor -- start 2024-01-31T05:02:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.793+0000 7f76e6c00640 1 -- start start 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e6c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76e01576b0 0x7f76e0070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e6c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76e0071440 0x7f76e006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e6c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e015e7b0 con 0x7f76e0157a90 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e6c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00b4130 con 0x7f76e01576b0 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e6c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e015e4d0 con 0x7f76e0071440 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76e01576b0 0x7f76e0070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f76e01576b0 0x7f76e0070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45942/0 (socket says 172.21.15.97:45942) 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 -- 172.21.15.97:0/2586806455 learned_addr learned my addr 172.21.15.97:0/2586806455 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:14.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 -- 172.21.15.97:0/2586806455 >> v2:172.21.15.188:3300/0 conn(0x7f76e0071440 msgr2=0x7f76e006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:14.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 --2- 172.21.15.97:0/2586806455 >> v2:172.21.15.188:3300/0 conn(0x7f76e0071440 0x7f76e006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 -- 172.21.15.97:0/2586806455 >> v1:172.21.15.188:6789/0 conn(0x7f76e0157a90 legacy=0x7f76e00b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:14.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.795+0000 7f76e5800640 1 -- 172.21.15.97:0/2586806455 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0070420 con 0x7f76e01576b0 2024-01-31T05:02:14.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.796+0000 7f76e5800640 1 --2- 172.21.15.97:0/2586806455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e01576b0 0x7f76e0070f00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f76e006b770 tx=0x7f76c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d40cacb9cc0368 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:14.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.797+0000 7f76dfe00640 1 -- 172.21.15.97:0/2586806455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8031870 con 0x7f76e01576b0 2024-01-31T05:02:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.797+0000 7f76dfe00640 1 -- 172.21.15.97:0/2586806455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c80319d0 con 0x7f76e01576b0 2024-01-31T05:02:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.797+0000 7f76dfe00640 1 -- 172.21.15.97:0/2586806455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8031d60 con 0x7f76e01576b0 2024-01-31T05:02:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.797+0000 7f76e6c00640 1 -- 172.21.15.97:0/2586806455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e01576b0 msgr2=0x7f76e0070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.797+0000 7f76e6c00640 1 --2- 172.21.15.97:0/2586806455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e01576b0 0x7f76e0070f00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f76e006b770 tx=0x7f76c802b910 comp rx=0 tx=0).stop 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.798+0000 7f76e6c00640 1 -- 172.21.15.97:0/2586806455 shutdown_connections 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.798+0000 7f76e6c00640 1 --2- 172.21.15.97:0/2586806455 >> v2:172.21.15.188:3300/0 conn(0x7f76e0071440 0x7f76e006c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.798+0000 7f76e6c00640 1 --2- 172.21.15.97:0/2586806455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e01576b0 0x7f76e0070f00 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.798+0000 7f76e6c00640 1 -- 172.21.15.97:0/2586806455 >> 172.21.15.97:0/2586806455 conn(0x7f76e006ad40 msgr2=0x7f76e00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.798+0000 7f76e6c00640 1 -- 172.21.15.97:0/2586806455 shutdown_connections 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.798+0000 7f76e6c00640 1 -- 172.21.15.97:0/2586806455 wait complete. 2024-01-31T05:02:14.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.799+0000 7f76e6c00640 1 Processor -- start 2024-01-31T05:02:14.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.799+0000 7f76e6c00640 1 -- start start 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.800+0000 7f76e6c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e0071440 0x7f76e01e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.800+0000 7f76e6c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e01576b0 0x7f76e0146050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.800+0000 7f76e6c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 0x7f76e00ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.800+0000 7f76e6c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0146590 con 0x7f76e0157a90 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.800+0000 7f76e6c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01e58c0 con 0x7f76e01576b0 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.800+0000 7f76e6c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01e5ba0 con 0x7f76e0071440 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 0x7f76e00ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 0x7f76e00ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45948/0 (socket says 172.21.15.97:45948) 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 -- 172.21.15.97:0/3870927635 learned_addr learned my addr 172.21.15.97:0/3870927635 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e5800640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e0071440 0x7f76e01e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e4e00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e01576b0 0x7f76e0146050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 -- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e0071440 msgr2=0x7f76e01e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e0071440 0x7f76e01e74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 -- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e01576b0 msgr2=0x7f76e0146050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e01576b0 0x7f76e0146050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e6200640 1 -- 172.21.15.97:0/3870927635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c80094d0 con 0x7f76e0157a90 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.801+0000 7f76e4e00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e01576b0 0x7f76e0146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:14.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.802+0000 7f76e6200640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 0x7f76e00ac4e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f76d800b930 tx=0x7f76d800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.802+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d800cb20 con 0x7f76e0157a90 2024-01-31T05:02:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.802+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d800ccc0 con 0x7f76e0157a90 2024-01-31T05:02:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.802+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8014680 con 0x7f76e0157a90 2024-01-31T05:02:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.802+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e01e5e80 con 0x7f76e0157a90 2024-01-31T05:02:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.802+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e00afb70 con 0x7f76e0157a90 2024-01-31T05:02:14.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.803+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e01e63a0 con 0x7f76e0157a90 2024-01-31T05:02:14.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.804+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f76d80147e0 con 0x7f76e0157a90 2024-01-31T05:02:14.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.805+0000 7f76dd600640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76b806af00 0x7f76b806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:14.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.805+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f76d80897a0 con 0x7f76e0157a90 2024-01-31T05:02:14.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.805+0000 7f76e5800640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76b806af00 0x7f76b806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:14.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.806+0000 7f76e5800640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76b806af00 0x7f76b806d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76c8036000 tx=0x7f76c803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:14.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:14.809+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f76d8054330 con 0x7f76e0157a90 2024-01-31T05:02:15.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.004+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76e00676b0 con 0x7f76e0157a90 2024-01-31T05:02:15.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.005+0000 7f76dd600640 1 -- 172.21.15.97:0/3870927635 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76d8057950 con 0x7f76e0157a90 2024-01-31T05:02:15.005 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76b806af00 msgr2=0x7f76b806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76b806af00 0x7f76b806d3c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76c8036000 tx=0x7f76c803c040 comp rx=0 tx=0).stop 2024-01-31T05:02:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 msgr2=0x7f76e00ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 0x7f76e00ac4e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f76d800b930 tx=0x7f76d800bdf0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 shutdown_connections 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76b806af00 0x7f76b806d3c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76e0157a90 0x7f76e00ac4e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e01576b0 0x7f76e0146050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 --2- 172.21.15.97:0/3870927635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e0071440 0x7f76e01e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 >> 172.21.15.97:0/3870927635 conn(0x7f76e006ad40 msgr2=0x7f76e015fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 shutdown_connections 2024-01-31T05:02:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.008+0000 7f76e6c00640 1 -- 172.21.15.97:0/3870927635 wait complete. 2024-01-31T05:02:15.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:15.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.238+0000 7f699fe00640 1 Processor -- start 2024-01-31T05:02:15.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.238+0000 7f699fe00640 1 -- start start 2024-01-31T05:02:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69a01576b0 0x7f69a0070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69a0071440 0x7f69a006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0157a90 0x7f69a006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699fe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a00b4130 con 0x7f69a0071440 2024-01-31T05:02:15.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69a015e7b0 con 0x7f69a01576b0 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a015e4d0 con 0x7f69a0157a90 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699e000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69a0071440 0x7f69a006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699e000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69a0071440 0x7f69a006c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45132/0 (socket says 172.21.15.97:45132) 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.240+0000 7f699e000640 1 -- 172.21.15.97:0/2090867279 learned_addr learned my addr 172.21.15.97:0/2090867279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699ea00640 1 --2- 172.21.15.97:0/2090867279 >> v2:172.21.15.188:3300/0 conn(0x7f69a01576b0 0x7f69a0070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699f400640 1 --2- 172.21.15.97:0/2090867279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0157a90 0x7f69a006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699e000640 1 -- 172.21.15.97:0/2090867279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0157a90 msgr2=0x7f69a006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699e000640 1 --2- 172.21.15.97:0/2090867279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0157a90 0x7f69a006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699e000640 1 -- 172.21.15.97:0/2090867279 >> v2:172.21.15.188:3300/0 conn(0x7f69a01576b0 msgr2=0x7f69a0070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699e000640 1 --2- 172.21.15.97:0/2090867279 >> v2:172.21.15.188:3300/0 conn(0x7f69a01576b0 0x7f69a0070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.241+0000 7f699e000640 1 -- 172.21.15.97:0/2090867279 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a0070420 con 0x7f69a0071440 2024-01-31T05:02:15.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.242+0000 7f699e000640 1 --2- 172.21.15.97:0/2090867279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0071440 0x7f69a006c100 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f69a006dfd0 tx=0x7f698802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c716c9d4396444 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:15.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699d600640 1 -- 172.21.15.97:0/2090867279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988031870 con 0x7f69a0071440 2024-01-31T05:02:15.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699d600640 1 -- 172.21.15.97:0/2090867279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69880319d0 con 0x7f69a0071440 2024-01-31T05:02:15.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699d600640 1 -- 172.21.15.97:0/2090867279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988031d60 con 0x7f69a0071440 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 -- 172.21.15.97:0/2090867279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0071440 msgr2=0x7f69a006c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 --2- 172.21.15.97:0/2090867279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0071440 0x7f69a006c100 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f69a006dfd0 tx=0x7f698802b910 comp rx=0 tx=0).stop 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 -- 172.21.15.97:0/2090867279 shutdown_connections 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 --2- 172.21.15.97:0/2090867279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0157a90 0x7f69a006c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 --2- 172.21.15.97:0/2090867279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0071440 0x7f69a006c100 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 --2- 172.21.15.97:0/2090867279 >> v2:172.21.15.188:3300/0 conn(0x7f69a01576b0 0x7f69a0070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 -- 172.21.15.97:0/2090867279 >> 172.21.15.97:0/2090867279 conn(0x7f69a006ad40 msgr2=0x7f69a00b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 -- 172.21.15.97:0/2090867279 shutdown_connections 2024-01-31T05:02:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.243+0000 7f699fe00640 1 -- 172.21.15.97:0/2090867279 wait complete. 2024-01-31T05:02:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.245+0000 7f699fe00640 1 Processor -- start 2024-01-31T05:02:15.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.245+0000 7f699fe00640 1 -- start start 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0071440 0x7f69a015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a01576b0 0x7f69a0145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 0x7f69a0145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a0143470 con 0x7f69a0157a90 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a0156ae0 con 0x7f69a01576b0 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69a01436c0 con 0x7f69a0071440 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 0x7f69a0145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 0x7f69a0145c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45134/0 (socket says 172.21.15.97:45134) 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.246+0000 7f699f400640 1 -- 172.21.15.97:0/1512234125 learned_addr learned my addr 172.21.15.97:0/1512234125 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699ea00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0071440 0x7f69a015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699e000640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a01576b0 0x7f69a0145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699f400640 1 -- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0071440 msgr2=0x7f69a015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699f400640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0071440 0x7f69a015cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699f400640 1 -- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a01576b0 msgr2=0x7f69a0145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699f400640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a01576b0 0x7f69a0145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699f400640 1 -- 172.21.15.97:0/1512234125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880094d0 con 0x7f69a0157a90 2024-01-31T05:02:15.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699ea00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0071440 0x7f69a015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699f400640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 0x7f69a0145c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6998007e60 tx=0x7f699800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998014420 con 0x7f69a0157a90 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69980145a0 con 0x7f69a0157a90 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6998014950 con 0x7f69a0157a90 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.247+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a0143960 con 0x7f69a0157a90 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.248+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a0143f20 con 0x7f69a0157a90 2024-01-31T05:02:15.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.249+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a00676b0 con 0x7f69a0157a90 2024-01-31T05:02:15.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.250+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6998014ab0 con 0x7f69a0157a90 2024-01-31T05:02:15.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.251+0000 7f6996a00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697806aed0 0x7f697806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.251+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f699808a360 con 0x7f69a0157a90 2024-01-31T05:02:15.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.251+0000 7f699ea00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697806aed0 0x7f697806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.252+0000 7f699ea00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697806aed0 0x7f697806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:15.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.256+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6998054e40 con 0x7f69a0157a90 2024-01-31T05:02:15.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.449+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69a00b1550 con 0x7f69a0157a90 2024-01-31T05:02:15.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.450+0000 7f6996a00640 1 -- 172.21.15.97:0/1512234125 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6998058460 con 0x7f69a0157a90 2024-01-31T05:02:15.449 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.452+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697806aed0 msgr2=0x7f697806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.452+0000 7f699fe00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697806aed0 0x7f697806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6990006fd0 tx=0x7f6990008040 comp rx=0 tx=0).stop 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.452+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 msgr2=0x7f69a0145c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.452+0000 7f699fe00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 0x7f69a0145c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6998007e60 tx=0x7f699800e4b0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 shutdown_connections 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697806aed0 0x7f697806d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69a0157a90 0x7f69a0145c90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a01576b0 0x7f69a0145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 --2- 172.21.15.97:0/1512234125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a0071440 0x7f69a015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 >> 172.21.15.97:0/1512234125 conn(0x7f69a006ad40 msgr2=0x7f69a006c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 shutdown_connections 2024-01-31T05:02:15.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.453+0000 7f699fe00640 1 -- 172.21.15.97:0/1512234125 wait complete. 2024-01-31T05:02:15.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-01-31T05:02:15.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.687+0000 7f2fd7600640 1 Processor -- start 2024-01-31T05:02:15.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.687+0000 7f2fd7600640 1 -- start start 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fd7600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2fd015b470 0x7f2fd00722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fd7600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fd00abcf0 0x7f2fd00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fd7600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd006cd20 con 0x7f2fd00cb810 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fd7600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00ce970 con 0x7f2fd00abcf0 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fd7600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd015b850 con 0x7f2fd015b470 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fcfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fd00abcf0 0x7f2fd00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fcfe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fd00abcf0 0x7f2fd00ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45142/0 (socket says 172.21.15.97:45142) 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fcfe00640 1 -- 172.21.15.97:0/3298856953 learned_addr learned my addr 172.21.15.97:0/3298856953 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.689+0000 7f2fd4c00640 1 --2- 172.21.15.97:0/3298856953 >> v2:172.21.15.188:3300/0 conn(0x7f2fd015b470 0x7f2fd00722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.690+0000 7f2fcfe00640 1 -- 172.21.15.97:0/3298856953 >> v2:172.21.15.188:3300/0 conn(0x7f2fd015b470 msgr2=0x7f2fd00722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.690+0000 7f2fcfe00640 1 --2- 172.21.15.97:0/3298856953 >> v2:172.21.15.188:3300/0 conn(0x7f2fd015b470 0x7f2fd00722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.690+0000 7f2fcfe00640 1 -- 172.21.15.97:0/3298856953 >> v1:172.21.15.97:6789/0 conn(0x7f2fd00cb810 legacy=0x7f2fd00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.690+0000 7f2fcfe00640 1 -- 172.21.15.97:0/3298856953 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00ceb60 con 0x7f2fd00abcf0 2024-01-31T05:02:15.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.690+0000 7f2fd4c00640 1 --2- 172.21.15.97:0/3298856953 >> v2:172.21.15.188:3300/0 conn(0x7f2fd015b470 0x7f2fd00722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:15.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.691+0000 7f2fcfe00640 1 --2- 172.21.15.97:0/3298856953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd00ce430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0067370 tx=0x7f2fb802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f71154294cc9a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.691+0000 7f2fcf400640 1 -- 172.21.15.97:0/3298856953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb802cb10 con 0x7f2fd00abcf0 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.691+0000 7f2fcf400640 1 -- 172.21.15.97:0/3298856953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb802cc70 con 0x7f2fd00abcf0 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.691+0000 7f2fcf400640 1 -- 172.21.15.97:0/3298856953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80343f0 con 0x7f2fd00abcf0 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 -- 172.21.15.97:0/3298856953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 msgr2=0x7f2fd00ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3298856953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd00ce430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0067370 tx=0x7f2fb802bf70 comp rx=0 tx=0).stop 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 -- 172.21.15.97:0/3298856953 shutdown_connections 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3298856953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd00ce430 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3298856953 >> v2:172.21.15.188:3300/0 conn(0x7f2fd015b470 0x7f2fd00722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 -- 172.21.15.97:0/3298856953 >> 172.21.15.97:0/3298856953 conn(0x7f2fd00b0a10 msgr2=0x7f2fd00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 -- 172.21.15.97:0/3298856953 shutdown_connections 2024-01-31T05:02:15.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.692+0000 7f2fd7600640 1 -- 172.21.15.97:0/3298856953 wait complete. 2024-01-31T05:02:15.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.694+0000 7f2fd7600640 1 Processor -- start 2024-01-31T05:02:15.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.694+0000 7f2fd7600640 1 -- start start 2024-01-31T05:02:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.695+0000 7f2fd7600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd01456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.695+0000 7f2fd7600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd00cb810 0x7f2fd00b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.695+0000 7f2fd7600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd015b470 0x7f2fd00b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd7600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00cafc0 con 0x7f2fd00abcf0 2024-01-31T05:02:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd7600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00cace0 con 0x7f2fd015b470 2024-01-31T05:02:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd7600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd014b3b0 con 0x7f2fd00cb810 2024-01-31T05:02:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd01456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd01456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45156/0 (socket says 172.21.15.97:45156) 2024-01-31T05:02:15.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 -- 172.21.15.97:0/3773569937 learned_addr learned my addr 172.21.15.97:0/3773569937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fcfe00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd00cb810 0x7f2fd00b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 -- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd00cb810 msgr2=0x7f2fd00b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd00cb810 0x7f2fd00b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 -- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd015b470 msgr2=0x7f2fd00b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd5600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd015b470 0x7f2fd00b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd015b470 0x7f2fd00b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.696+0000 7f2fd4c00640 1 -- 172.21.15.97:0/3773569937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8009b30 con 0x7f2fd00abcf0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.697+0000 7f2fd4c00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd01456c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c7f0 tx=0x7f2fc800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.697+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8004220 con 0x7f2fd00abcf0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.697+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8004380 con 0x7f2fd00abcf0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.697+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8005bb0 con 0x7f2fd00abcf0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.697+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd014b690 con 0x7f2fd00abcf0 2024-01-31T05:02:15.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.697+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd00b3580 con 0x7f2fd00abcf0 2024-01-31T05:02:15.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.698+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd014b210 con 0x7f2fd00abcf0 2024-01-31T05:02:15.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.699+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2fc8006030 con 0x7f2fd00abcf0 2024-01-31T05:02:15.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.700+0000 7f2fccc00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fa806af00 0x7f2fa806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:15.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.700+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2fc8089f60 con 0x7f2fd00abcf0 2024-01-31T05:02:15.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.700+0000 7f2fcfe00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fa806af00 0x7f2fa806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:15.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.701+0000 7f2fcfe00640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fa806af00 0x7f2fa806d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8035000 tx=0x7f2fb8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:15.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.704+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2fd014b210 con 0x7f2fd00abcf0 2024-01-31T05:02:15.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.900+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2fd00b2820 con 0x7f2fd00abcf0 2024-01-31T05:02:15.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.901+0000 7f2fccc00640 1 -- 172.21.15.97:0/3773569937 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2fd00b2820 con 0x7f2fd00abcf0 2024-01-31T05:02:15.901 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fa806af00 msgr2=0x7f2fa806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fa806af00 0x7f2fa806d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8035000 tx=0x7f2fb8030000 comp rx=0 tx=0).stop 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 msgr2=0x7f2fd01456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd01456c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800c7f0 tx=0x7f2fc800ccb0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 shutdown_connections 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd015b470 0x7f2fd00b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd00cb810 0x7f2fd00b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2fa806af00 0x7f2fa806d3c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 --2- 172.21.15.97:0/3773569937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd00abcf0 0x7f2fd01456c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 >> 172.21.15.97:0/3773569937 conn(0x7f2fd00b0a10 msgr2=0x7f2fd00b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:15.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 shutdown_connections 2024-01-31T05:02:15.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:15.904+0000 7f2fd7600640 1 -- 172.21.15.97:0/3773569937 wait complete. 2024-01-31T05:02:15.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:16.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.130+0000 7f713ee00640 1 Processor -- start 2024-01-31T05:02:16.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.130+0000 7f713ee00640 1 -- start start 2024-01-31T05:02:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f713ee00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f713815d520 0x7f71380cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f713ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71380ce430 0x7f7138160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f713ee00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380abcf0 con 0x7f7138161500 2024-01-31T05:02:16.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f713ee00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7138161990 con 0x7f713815d520 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f713ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713815d900 con 0x7f71380ce430 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f7137e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f713815d520 0x7f71380cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.132+0000 7f713ce00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7138161500 0x7f71380b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41936/0 (socket says 172.21.15.97:41936) 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f713ce00640 1 -- 172.21.15.97:0/498779852 learned_addr learned my addr 172.21.15.97:0/498779852 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f712fe00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71380ce430 0x7f7138160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f7137e00640 1 -- 172.21.15.97:0/498779852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71380ce430 msgr2=0x7f7138160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f7137e00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71380ce430 0x7f7138160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f7137e00640 1 -- 172.21.15.97:0/498779852 >> v1:172.21.15.97:6789/0 conn(0x7f7138161500 legacy=0x7f71380b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f7137e00640 1 -- 172.21.15.97:0/498779852 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713806ca40 con 0x7f713815d520 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.133+0000 7f712fe00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71380ce430 0x7f7138160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.134+0000 7f7137e00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f713815d520 0x7f71380cd710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f71280097a0 tx=0x7f712802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106249ed883e399c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.134+0000 7f7137400640 1 -- 172.21.15.97:0/498779852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712802d040 con 0x7f713815d520 2024-01-31T05:02:16.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.134+0000 7f7137400640 1 -- 172.21.15.97:0/498779852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128031810 con 0x7f713815d520 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.134+0000 7f7137400640 1 -- 172.21.15.97:0/498779852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128031ba0 con 0x7f713815d520 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 -- 172.21.15.97:0/498779852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f713815d520 msgr2=0x7f71380cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f713815d520 0x7f71380cd710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f71280097a0 tx=0x7f712802b8b0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 -- 172.21.15.97:0/498779852 shutdown_connections 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71380ce430 0x7f7138160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 --2- 172.21.15.97:0/498779852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f713815d520 0x7f71380cd710 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 -- 172.21.15.97:0/498779852 >> 172.21.15.97:0/498779852 conn(0x7f71380b0a10 msgr2=0x7f71380b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 -- 172.21.15.97:0/498779852 shutdown_connections 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.135+0000 7f713ee00640 1 -- 172.21.15.97:0/498779852 wait complete. 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.137+0000 7f713ee00640 1 Processor -- start 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.137+0000 7f713ee00640 1 -- start start 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71380ce430 0x7f7138144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f713815d520 0x7f71381f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 0x7f7138147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713809bde0 con 0x7f7138161500 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138147560 con 0x7f71380ce430 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71381f34d0 con 0x7f713815d520 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 0x7f7138147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ce00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 0x7f7138147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45168/0 (socket says 172.21.15.97:45168) 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.138+0000 7f713ce00640 1 -- 172.21.15.97:0/790594610 learned_addr learned my addr 172.21.15.97:0/790594610 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f712fe00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f713815d520 0x7f71381f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ce00640 1 -- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f713815d520 msgr2=0x7f71381f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ce00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f713815d520 0x7f71381f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ce00640 1 -- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71380ce430 msgr2=0x7f7138144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ce00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71380ce430 0x7f7138144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ce00640 1 -- 172.21.15.97:0/790594610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128009400 con 0x7f7138161500 2024-01-31T05:02:16.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f712fe00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f713815d520 0x7f71381f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ce00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 0x7f7138147020 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f713000ca80 tx=0x7f713000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713000ac30 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713000ad90 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71381f37b0 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130012600 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.139+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381f3d70 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.140+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713809bb40 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.141+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7130007070 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.142+0000 7f7134c00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f711006aed0 0x7f711006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.142+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f71300897c0 con 0x7f7138161500 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.143+0000 7f7137e00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f711006aed0 0x7f711006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.143+0000 7f7137e00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f711006aed0 0x7f711006d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7128036000 tx=0x7f7128030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.147+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f713809bb40 con 0x7f7138161500 2024-01-31T05:02:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.381+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f713809bb40 con 0x7f7138161500 2024-01-31T05:02:16.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.382+0000 7f7134c00640 1 -- 172.21.15.97:0/790594610 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f713809bb40 con 0x7f7138161500 2024-01-31T05:02:16.382 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:16.382 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:02:16.382 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f711006aed0 msgr2=0x7f711006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f711006aed0 0x7f711006d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7128036000 tx=0x7f7128030000 comp rx=0 tx=0).stop 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 msgr2=0x7f7138147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 0x7f7138147020 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f713000ca80 tx=0x7f713000cf40 comp rx=0 tx=0).stop 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 shutdown_connections 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f711006aed0 0x7f711006d390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7138161500 0x7f7138147020 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f713815d520 0x7f71381f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 --2- 172.21.15.97:0/790594610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71380ce430 0x7f7138144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.384+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 >> 172.21.15.97:0/790594610 conn(0x7f71380b0a10 msgr2=0x7f71380afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.385+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 shutdown_connections 2024-01-31T05:02:16.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.385+0000 7f713ee00640 1 -- 172.21.15.97:0/790594610 wait complete. 2024-01-31T05:02:16.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:16.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.609+0000 7fa353200640 1 Processor -- start 2024-01-31T05:02:16.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.610+0000 7fa353200640 1 -- start start 2024-01-31T05:02:16.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa353200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415ec70 0x7fa354071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa353200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3540723a0 0x7fa35415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa353200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa354070400 con 0x7fa3540ae3b0 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa353200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3540ad8c0 con 0x7fa3540723a0 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa353200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540ca580 con 0x7fa35415ec70 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa351400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa3540ae3b0 0x7fa35406cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41950/0 (socket says 172.21.15.97:41950) 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa351400640 1 -- 172.21.15.97:0/2049491001 learned_addr learned my addr 172.21.15.97:0/2049491001 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:16.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.611+0000 7fa352800640 1 --2- 172.21.15.97:0/2049491001 >> v2:172.21.15.97:3300/0 conn(0x7fa3540723a0 0x7fa35415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa351e00640 1 --2- 172.21.15.97:0/2049491001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415ec70 0x7fa354071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa350a00640 1 -- 172.21.15.97:0/2049491001 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973586826 0 0) 0x7fa354070400 con 0x7fa3540ae3b0 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa350a00640 1 -- 172.21.15.97:0/2049491001 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa334003520 con 0x7fa3540ae3b0 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa352800640 1 -- 172.21.15.97:0/2049491001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415ec70 msgr2=0x7fa354071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa352800640 1 --2- 172.21.15.97:0/2049491001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415ec70 0x7fa354071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa352800640 1 -- 172.21.15.97:0/2049491001 >> v1:172.21.15.97:6789/0 conn(0x7fa3540ae3b0 legacy=0x7fa35406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.612+0000 7fa352800640 1 -- 172.21.15.97:0/2049491001 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354070650 con 0x7fa3540723a0 2024-01-31T05:02:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.613+0000 7fa352800640 1 --2- 172.21.15.97:0/2049491001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa35415f050 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009d90 tx=0x7fa34c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bdc42b843ea7ae server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa350a00640 1 -- 172.21.15.97:0/2049491001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c02cb30 con 0x7fa3540723a0 2024-01-31T05:02:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa350a00640 1 -- 172.21.15.97:0/2049491001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c02cc90 con 0x7fa3540723a0 2024-01-31T05:02:16.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa350a00640 1 -- 172.21.15.97:0/2049491001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c034440 con 0x7fa3540723a0 2024-01-31T05:02:16.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 -- 172.21.15.97:0/2049491001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 msgr2=0x7fa35415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 --2- 172.21.15.97:0/2049491001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa35415f050 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009d90 tx=0x7fa34c02bd80 comp rx=0 tx=0).stop 2024-01-31T05:02:16.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 -- 172.21.15.97:0/2049491001 shutdown_connections 2024-01-31T05:02:16.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 --2- 172.21.15.97:0/2049491001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa35415f050 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 --2- 172.21.15.97:0/2049491001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415ec70 0x7fa354071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 -- 172.21.15.97:0/2049491001 >> 172.21.15.97:0/2049491001 conn(0x7fa3540cec40 msgr2=0x7fa3540cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:16.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.614+0000 7fa353200640 1 -- 172.21.15.97:0/2049491001 shutdown_connections 2024-01-31T05:02:16.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.615+0000 7fa353200640 1 -- 172.21.15.97:0/2049491001 wait complete. 2024-01-31T05:02:16.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.616+0000 7fa353200640 1 Processor -- start 2024-01-31T05:02:16.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.616+0000 7fa353200640 1 -- start start 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa353200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa3540abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa353200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3540ae3b0 0x7fa3541f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa353200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35415ec70 0x7fa3540b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa353200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35409ba20 con 0x7fa3540723a0 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa353200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35409b740 con 0x7fa35415ec70 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa353200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b1b30 con 0x7fa3540ae3b0 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa351e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa3540abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa351e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa3540abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45182/0 (socket says 172.21.15.97:45182) 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.617+0000 7fa351e00640 1 -- 172.21.15.97:0/1364114951 learned_addr learned my addr 172.21.15.97:0/1364114951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351400640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3540ae3b0 0x7fa3541f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351e00640 1 -- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3540ae3b0 msgr2=0x7fa3541f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351e00640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3540ae3b0 0x7fa3541f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351e00640 1 -- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35415ec70 msgr2=0x7fa3540b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa352800640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35415ec70 0x7fa3540b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351e00640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35415ec70 0x7fa3540b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351e00640 1 -- 172.21.15.97:0/1364114951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c009a70 con 0x7fa3540723a0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa351e00640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa3540abf60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa34400d800 tx=0x7fa34400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400b6c0 con 0x7fa3540723a0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34400b820 con 0x7fa3540723a0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400bbb0 con 0x7fa3540723a0 2024-01-31T05:02:16.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.618+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3540b1e10 con 0x7fa3540723a0 2024-01-31T05:02:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.619+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3540ac4a0 con 0x7fa3540723a0 2024-01-31T05:02:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.620+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3540b1990 con 0x7fa3540723a0 2024-01-31T05:02:16.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.620+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa34400bd10 con 0x7fa3540723a0 2024-01-31T05:02:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.621+0000 7fa34a000640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa32c06aed0 0x7fa32c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.621+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa344089eb0 con 0x7fa3540723a0 2024-01-31T05:02:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.622+0000 7fa351400640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa32c06aed0 0x7fa32c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:16.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.622+0000 7fa351400640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa32c06aed0 0x7fa32c06d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa3540721b0 tx=0x7fa33c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:16.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.626+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa3540b1990 con 0x7fa3540723a0 2024-01-31T05:02:16.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.820+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3540b1990 con 0x7fa3540723a0 2024-01-31T05:02:16.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.823+0000 7fa34a000640 1 -- 172.21.15.97:0/1364114951 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7fa3540b1990 con 0x7fa3540723a0 2024-01-31T05:02:16.823 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:16.824 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa32c06aed0 msgr2=0x7fa32c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa32c06aed0 0x7fa32c06d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa3540721b0 tx=0x7fa33c009040 comp rx=0 tx=0).stop 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 msgr2=0x7fa3540abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa3540abf60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa34400d800 tx=0x7fa34400dcc0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 shutdown_connections 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35415ec70 0x7fa3540b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3540ae3b0 0x7fa3541f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa32c06aed0 0x7fa32c06d390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 --2- 172.21.15.97:0/1364114951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3540723a0 0x7fa3540abf60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:16.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 >> 172.21.15.97:0/1364114951 conn(0x7fa3540cec40 msgr2=0x7fa3540af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:16.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.826+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 shutdown_connections 2024-01-31T05:02:16.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:16.827+0000 7fa353200640 1 -- 172.21.15.97:0/1364114951 wait complete. 2024-01-31T05:02:16.846 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-31T05:02:16.846 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.052+0000 7f254b400640 1 Processor -- start 2024-01-31T05:02:17.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.053+0000 7f254b400640 1 -- start start 2024-01-31T05:02:17.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.054+0000 7f254b400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f254406c940 0x7f254409a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.054+0000 7f254b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254406ba30 0x7f25440cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.054+0000 7f254b400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440cf570 con 0x7f25440cd070 2024-01-31T05:02:17.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.054+0000 7f254b400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25440cf850 con 0x7f254406c940 2024-01-31T05:02:17.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.054+0000 7f254b400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440cfb30 con 0x7f254406ba30 2024-01-31T05:02:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2548a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f25440cd070 0x7f254406c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41956/0 (socket says 172.21.15.97:41956) 2024-01-31T05:02:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2548a00640 1 -- 172.21.15.97:0/2761313864 learned_addr learned my addr 172.21.15.97:0/2761313864 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543e00640 1 --2- 172.21.15.97:0/2761313864 >> v2:172.21.15.97:3300/0 conn(0x7f254406c940 0x7f254409a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2549400640 1 --2- 172.21.15.97:0/2761313864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254406ba30 0x7f25440cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543400640 1 -- 172.21.15.97:0/2761313864 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768949519 0 0) 0x7f25440cf570 con 0x7f25440cd070 2024-01-31T05:02:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543400640 1 -- 172.21.15.97:0/2761313864 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003520 con 0x7f25440cd070 2024-01-31T05:02:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543e00640 1 -- 172.21.15.97:0/2761313864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254406ba30 msgr2=0x7f25440cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543e00640 1 --2- 172.21.15.97:0/2761313864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254406ba30 0x7f25440cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543e00640 1 -- 172.21.15.97:0/2761313864 >> v1:172.21.15.97:6789/0 conn(0x7f25440cd070 legacy=0x7f254406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.055+0000 7f2543e00640 1 -- 172.21.15.97:0/2761313864 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440cfe10 con 0x7f254406c940 2024-01-31T05:02:17.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.056+0000 7f2543e00640 1 --2- 172.21.15.97:0/2761313864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f254409a3a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f252c009d90 tx=0x7f252c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551bb14145b71f55 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:17.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f2543400640 1 -- 172.21.15.97:0/2761313864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c02cc50 con 0x7f254406c940 2024-01-31T05:02:17.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f2543400640 1 -- 172.21.15.97:0/2761313864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c02cdb0 con 0x7f254406c940 2024-01-31T05:02:17.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f2543400640 1 -- 172.21.15.97:0/2761313864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c034660 con 0x7f254406c940 2024-01-31T05:02:17.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f254b400640 1 -- 172.21.15.97:0/2761313864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 msgr2=0x7f254409a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f254b400640 1 --2- 172.21.15.97:0/2761313864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f254409a3a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f252c009d90 tx=0x7f252c02bea0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f254b400640 1 -- 172.21.15.97:0/2761313864 shutdown_connections 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f254b400640 1 --2- 172.21.15.97:0/2761313864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254406ba30 0x7f25440cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.057+0000 7f254b400640 1 --2- 172.21.15.97:0/2761313864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f254409a3a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.058+0000 7f254b400640 1 -- 172.21.15.97:0/2761313864 >> 172.21.15.97:0/2761313864 conn(0x7f25440b2b90 msgr2=0x7f25440b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.058+0000 7f254b400640 1 -- 172.21.15.97:0/2761313864 shutdown_connections 2024-01-31T05:02:17.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.058+0000 7f254b400640 1 -- 172.21.15.97:0/2761313864 wait complete. 2024-01-31T05:02:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.059+0000 7f254b400640 1 Processor -- start 2024-01-31T05:02:17.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.060+0000 7f254b400640 1 -- start start 2024-01-31T05:02:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.060+0000 7f254b400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254406ba30 0x7f254415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.060+0000 7f254b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f25441f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.060+0000 7f254b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25440cd070 0x7f25441f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.060+0000 7f254b400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25441f4e00 con 0x7f254406c940 2024-01-31T05:02:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f254b400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25441f4180 con 0x7f254406ba30 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f254b400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25441f4460 con 0x7f25440cd070 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f25441f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f25441f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45202/0 (socket says 172.21.15.97:45202) 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 -- 172.21.15.97:0/1861676927 learned_addr learned my addr 172.21.15.97:0/1861676927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2549400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25440cd070 0x7f25441f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2548a00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254406ba30 0x7f254415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 -- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25440cd070 msgr2=0x7f25441f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25440cd070 0x7f25441f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 -- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254406ba30 msgr2=0x7f254415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254406ba30 0x7f254415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2543e00640 1 -- 172.21.15.97:0/1861676927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c009a70 con 0x7f254406c940 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2548a00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254406ba30 0x7f254415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.061+0000 7f2549400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25440cd070 0x7f25441f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:17.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.062+0000 7f2543e00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f25441f3610 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f252c009d90 tx=0x7f252c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:17.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.063+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c040650 con 0x7f254406c940 2024-01-31T05:02:17.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.063+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c0407b0 con 0x7f254406c940 2024-01-31T05:02:17.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.063+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c040b60 con 0x7f254406c940 2024-01-31T05:02:17.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.063+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25441f4740 con 0x7f254406c940 2024-01-31T05:02:17.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.063+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25440ae860 con 0x7f254406c940 2024-01-31T05:02:17.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.065+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254406c0a0 con 0x7f254406c940 2024-01-31T05:02:17.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.065+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f252c040cc0 con 0x7f254406c940 2024-01-31T05:02:17.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.066+0000 7f2540c00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f251c06aed0 0x7f251c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.066+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f252c0ac720 con 0x7f254406c940 2024-01-31T05:02:17.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.066+0000 7f2548a00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f251c06aed0 0x7f251c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.067+0000 7f2548a00640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f251c06aed0 0x7f251c06d390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2534005880 tx=0x7f2534009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:17.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.071+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f252c077450 con 0x7f254406c940 2024-01-31T05:02:17.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.304+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2544067dc0 con 0x7f254406c940 2024-01-31T05:02:17.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.305+0000 7f2540c00640 1 -- 172.21.15.97:0/1861676927 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f252c07aa70 con 0x7f254406c940 2024-01-31T05:02:17.304 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:17.305 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:02:17.305 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:02:17.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f251c06aed0 msgr2=0x7f251c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f251c06aed0 0x7f251c06d390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2534005880 tx=0x7f2534009040 comp rx=0 tx=0).stop 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 msgr2=0x7f25441f3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f25441f3610 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f252c009d90 tx=0x7f252c004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 shutdown_connections 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25440cd070 0x7f25441f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f251c06aed0 0x7f251c06d390 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f254406c940 0x7f25441f3610 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 --2- 172.21.15.97:0/1861676927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f254406ba30 0x7f254415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 >> 172.21.15.97:0/1861676927 conn(0x7f25440b2b90 msgr2=0x7f2544160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 shutdown_connections 2024-01-31T05:02:17.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.308+0000 7f254b400640 1 -- 172.21.15.97:0/1861676927 wait complete. 2024-01-31T05:02:17.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:17.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.537+0000 7f8c88e00640 1 Processor -- start 2024-01-31T05:02:17.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.537+0000 7f8c88e00640 1 -- start start 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c88e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c8415d520 0x7f8c840cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c88e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c840abcf0 con 0x7f8c840ce430 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c88e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8406ca40 con 0x7f8c841616d0 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c88e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8415d900 con 0x7f8c8415d520 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c81e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c8415d520 0x7f8c840cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c81e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8c8415d520 0x7f8c840cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45216/0 (socket says 172.21.15.97:45216) 2024-01-31T05:02:17.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.539+0000 7f8c81e00640 1 -- 172.21.15.97:0/1234004888 learned_addr learned my addr 172.21.15.97:0/1234004888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.540+0000 7f8c81e00640 1 -- 172.21.15.97:0/1234004888 >> v1:172.21.15.188:6789/0 conn(0x7f8c841616d0 legacy=0x7f8c8406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.540+0000 7f8c81e00640 1 -- 172.21.15.97:0/1234004888 >> v1:172.21.15.97:6789/0 conn(0x7f8c840ce430 legacy=0x7f8c84160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.540+0000 7f8c81e00640 1 -- 172.21.15.97:0/1234004888 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840ce890 con 0x7f8c8415d520 2024-01-31T05:02:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.541+0000 7f8c81e00640 1 --2- 172.21.15.97:0/1234004888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c840cd710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009870 tx=0x7f8c7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4080c19889e141 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.541+0000 7f8c80a00640 1 -- 172.21.15.97:0/1234004888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74030860 con 0x7f8c8415d520 2024-01-31T05:02:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.541+0000 7f8c80a00640 1 -- 172.21.15.97:0/1234004888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c740309c0 con 0x7f8c8415d520 2024-01-31T05:02:17.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.541+0000 7f8c80a00640 1 -- 172.21.15.97:0/1234004888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74030d50 con 0x7f8c8415d520 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 -- 172.21.15.97:0/1234004888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 msgr2=0x7f8c840cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 --2- 172.21.15.97:0/1234004888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c840cd710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009870 tx=0x7f8c7402b9a0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 -- 172.21.15.97:0/1234004888 shutdown_connections 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 --2- 172.21.15.97:0/1234004888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c840cd710 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 -- 172.21.15.97:0/1234004888 >> 172.21.15.97:0/1234004888 conn(0x7f8c840b0a10 msgr2=0x7f8c840b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 -- 172.21.15.97:0/1234004888 shutdown_connections 2024-01-31T05:02:17.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.542+0000 7f8c88e00640 1 -- 172.21.15.97:0/1234004888 wait complete. 2024-01-31T05:02:17.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.544+0000 7f8c88e00640 1 Processor -- start 2024-01-31T05:02:17.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.544+0000 7f8c88e00640 1 -- start start 2024-01-31T05:02:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.545+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ce430 0x7f8c84158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.545+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c8415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c88e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c841616d0 0x7f8c84159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c88e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8415a690 con 0x7f8c8415d520 2024-01-31T05:02:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c88e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841597c0 con 0x7f8c841616d0 2024-01-31T05:02:17.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c88e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c84159aa0 con 0x7f8c840ce430 2024-01-31T05:02:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c8415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c8415e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45218/0 (socket says 172.21.15.97:45218) 2024-01-31T05:02:17.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 -- 172.21.15.97:0/3025828915 learned_addr learned my addr 172.21.15.97:0/3025828915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c82800640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c841616d0 0x7f8c84159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 -- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ce430 msgr2=0x7f8c84158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ce430 0x7f8c84158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 -- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c841616d0 msgr2=0x7f8c84159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c841616d0 0x7f8c84159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c81400640 1 -- 172.21.15.97:0/3025828915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740094d0 con 0x7f8c8415d520 2024-01-31T05:02:17.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.546+0000 7f8c82800640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c841616d0 0x7f8c84159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.547+0000 7f8c81400640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c8415e660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00c400 tx=0x7f8c6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.547+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0052d0 con 0x7f8c8415d520 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.547+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6c005450 con 0x7f8c8415d520 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.547+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8415c7d0 con 0x7f8c8415d520 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.547+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c007260 con 0x7f8c8415d520 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.547+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8409aa10 con 0x7f8c8415d520 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.548+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c84160dc0 con 0x7f8c8415d520 2024-01-31T05:02:17.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.549+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8c6c0073c0 con 0x7f8c8415d520 2024-01-31T05:02:17.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.550+0000 7f8c7a000640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8c5c06aed0 0x7f8c5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:17.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.550+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8c6c088d20 con 0x7f8c8415d520 2024-01-31T05:02:17.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.550+0000 7f8c81e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8c5c06aed0 0x7f8c5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:17.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.551+0000 7f8c81e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8c5c06aed0 0x7f8c5c06d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c74036000 tx=0x7f8c7403b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:17.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.554+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8c6c053930 con 0x7f8c8415d520 2024-01-31T05:02:17.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.786+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c84159ce0 con 0x7f8c8415d520 2024-01-31T05:02:17.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.787+0000 7f8c7a000640 1 -- 172.21.15.97:0/3025828915 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f8c84159ce0 con 0x7f8c8415d520 2024-01-31T05:02:17.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:17.787 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1407876925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1407876925},{"type":"v1","addr":"172.21.15.188:6833","nonce":1407876925}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4240,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3714536704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3714536704},{"type":"v1","addr":"172.21.15.97:6836","nonce":3714536704}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/536523293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":536523293},{"type":"v1","addr":"172.21.15.188:6835","nonce":536523293}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:00:46.002409+0000","modified":"2024-01-31T05:01:08.303380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4257},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4257":{"gid":4257,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/294460102","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":294460102},{"type":"v1","addr":"172.21.15.97:6837","nonce":294460102}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-31T05:02:17.787 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.789+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8c5c06aed0 msgr2=0x7f8c5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.789+0000 7f8c88e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8c5c06aed0 0x7f8c5c06d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c74036000 tx=0x7f8c7403b040 comp rx=0 tx=0).stop 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.789+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 msgr2=0x7f8c8415e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.789+0000 7f8c88e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c8415e660 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00c400 tx=0x7f8c6c00c8c0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.789+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 shutdown_connections 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c841616d0 0x7f8c84159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8c5c06aed0 0x7f8c5c06d390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c8415d520 0x7f8c8415e660 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 --2- 172.21.15.97:0/3025828915 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c840ce430 0x7f8c84158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:17.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 >> 172.21.15.97:0/3025828915 conn(0x7f8c840b0a10 msgr2=0x7f8c840cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 shutdown_connections 2024-01-31T05:02:17.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:17.790+0000 7f8c88e00640 1 -- 172.21.15.97:0/3025828915 wait complete. 2024-01-31T05:02:17.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:18.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.012+0000 7fc178a00640 1 Processor -- start 2024-01-31T05:02:18.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.012+0000 7fc178a00640 1 -- start start 2024-01-31T05:02:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc178a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc17015aa60 0x7fc17006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc178a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17015aed0 con 0x7fc17015a600 2024-01-31T05:02:18.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc178a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17006c8b0 con 0x7fc1700b1d10 2024-01-31T05:02:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc178a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17015b120 con 0x7fc17015aa60 2024-01-31T05:02:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc175600640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc17015a600 0x7fc17006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41974/0 (socket says 172.21.15.97:41974) 2024-01-31T05:02:18.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc175600640 1 -- 172.21.15.97:0/1047373659 learned_addr learned my addr 172.21.15.97:0/1047373659 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374456204 0 0) 0x7fc17015aed0 con 0x7fc17015a600 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc150003520 con 0x7fc17015a600 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.014+0000 7fc176a00640 1 --2- 172.21.15.97:0/1047373659 >> v2:172.21.15.188:3300/0 conn(0x7fc17015aa60 0x7fc17006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221830902 0 0) 0x7fc150003520 con 0x7fc17015a600 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 >> v2:172.21.15.188:3300/0 conn(0x7fc17015aa60 msgr2=0x7fc17006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 --2- 172.21.15.97:0/1047373659 >> v2:172.21.15.188:3300/0 conn(0x7fc17015aa60 0x7fc17006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 >> v1:172.21.15.188:6789/0 conn(0x7fc1700b1d10 legacy=0x7fc1700b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17015b400 con 0x7fc17015a600 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fc158001670 con 0x7fc17015a600 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 >> v1:172.21.15.97:6789/0 conn(0x7fc17015a600 legacy=0x7fc17006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1500039c0 0x7fc150003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500047c0 0x7fc150004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc15000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17015b120 con 0x7fc150005180 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000c080 con 0x7fc1500047c0 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15000c240 con 0x7fc1500039c0 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.015+0000 7fc176a00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc15000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc175600640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500047c0 0x7fc150004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc176a00640 1 -- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1500039c0 msgr2=0x7fc150003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc176a00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1500039c0 0x7fc150003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc176a00640 1 -- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500047c0 msgr2=0x7fc150004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:18.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc176a00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500047c0 0x7fc150004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc176a00640 1 -- 172.21.15.97:0/1047373659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150003520 con 0x7fc150005180 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.016+0000 7fc175600640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500047c0 0x7fc150004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.017+0000 7fc176a00640 1 -- 172.21.15.97:0/1047373659 reap_dead start 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.017+0000 7fc176a00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc15000b980 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc168009fc0 tx=0x7fc1500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5973aefb0dfc67e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.017+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16802cc60 con 0x7fc150005180 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.017+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16800f040 con 0x7fc150005180 2024-01-31T05:02:18.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.017+0000 7fc174c00640 1 -- 172.21.15.97:0/1047373659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168003990 con 0x7fc150005180 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 -- 172.21.15.97:0/1047373659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 msgr2=0x7fc15000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc15000b980 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc168009fc0 tx=0x7fc1500057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 -- 172.21.15.97:0/1047373659 shutdown_connections 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 --2- 172.21.15.97:0/1047373659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc15000b980 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 -- 172.21.15.97:0/1047373659 >> 172.21.15.97:0/1047373659 conn(0x7fc1700b0a10 msgr2=0x7fc1700b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 -- 172.21.15.97:0/1047373659 shutdown_connections 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.018+0000 7fc178a00640 1 -- 172.21.15.97:0/1047373659 wait complete. 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.020+0000 7fc178a00640 1 Processor -- start 2024-01-31T05:02:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.020+0000 7fc178a00640 1 -- start start 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc178a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc1700b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc178a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc170147020 0x7fc1701447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc178a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1701584e0 0x7fc1700715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc178a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170147ff0 con 0x7fc150005180 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc178a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170144d20 con 0x7fc1701584e0 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc178a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17015c4a0 con 0x7fc170147020 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc1700b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc1700b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45234/0 (socket says 172.21.15.97:45234) 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 -- 172.21.15.97:0/1146013102 learned_addr learned my addr 172.21.15.97:0/1146013102 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1701584e0 0x7fc1700715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 -- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc170147020 msgr2=0x7fc1701447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc170147020 0x7fc1701447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 -- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1701584e0 msgr2=0x7fc1700715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1701584e0 0x7fc1700715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176000640 1 -- 172.21.15.97:0/1146013102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700adb80 con 0x7fc150005180 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.021+0000 7fc176a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1701584e0 0x7fc1700715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.022+0000 7fc176000640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc1700b0390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c400 tx=0x7fc16000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.022+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600052d0 con 0x7fc150005180 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.022+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160005470 con 0x7fc150005180 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.022+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160003830 con 0x7fc150005180 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.022+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc170147480 con 0x7fc150005180 2024-01-31T05:02:18.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.022+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc170147a30 con 0x7fc150005180 2024-01-31T05:02:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.023+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1700677c0 con 0x7fc150005180 2024-01-31T05:02:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.024+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc160003990 con 0x7fc150005180 2024-01-31T05:02:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.025+0000 7fc16e000640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc14806aed0 0x7fc14806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.025+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc160088a80 con 0x7fc150005180 2024-01-31T05:02:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.025+0000 7fc175600640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc14806aed0 0x7fc14806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.026+0000 7fc175600640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc14806aed0 0x7fc14806d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc158000c00 tx=0x7fc158009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.029+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc160057140 con 0x7fc150005180 2024-01-31T05:02:18.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.227+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc170154f70 con 0x7fc150005180 2024-01-31T05:02:18.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.230+0000 7fc16e000640 1 -- 172.21.15.97:0/1146013102 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18492 (secure 0 0 0) 0x7fc160017020 con 0x7fc150005180 2024-01-31T05:02:18.230 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:18.230 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":28,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:01:32.842043+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:00:44.317943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-31T05:00:45.040037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-01-31T05:00:48.648377+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":0,"to":7}]},{"pgid":"4.d","mappings":[{"from":3,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc14806aed0 msgr2=0x7fc14806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc14806aed0 0x7fc14806d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc158000c00 tx=0x7fc158009040 comp rx=0 tx=0).stop 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 msgr2=0x7fc1700b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc1700b0390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c400 tx=0x7fc16000c8c0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 shutdown_connections 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1701584e0 0x7fc1700715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc170147020 0x7fc1701447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc14806aed0 0x7fc14806d390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 --2- 172.21.15.97:0/1146013102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc150005180 0x7fc1700b0390 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.234+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 >> 172.21.15.97:0/1146013102 conn(0x7fc1700b0a10 msgr2=0x7fc1700ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:18.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.235+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 shutdown_connections 2024-01-31T05:02:18.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.235+0000 7fc178a00640 1 -- 172.21.15.97:0/1146013102 wait complete. 2024-01-31T05:02:18.254 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-31T05:02:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.466+0000 7efd43000640 1 Processor -- start 2024-01-31T05:02:18.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.467+0000 7efd43000640 1 -- start start 2024-01-31T05:02:18.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd43000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd43000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd3c0cbf20 0x7efd3c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd43000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c06cd20 con 0x7efd3c0abcf0 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd43000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0ce970 con 0x7efd3c0cbf20 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd43000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c15b850 con 0x7efd3c15b470 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd41000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd3c0cbf20 0x7efd3c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd33200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efd3c0abcf0 0x7efd3c0cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41980/0 (socket says 172.21.15.97:41980) 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd41000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efd3c0cbf20 0x7efd3c0ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45238/0 (socket says 172.21.15.97:45238) 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.468+0000 7efd33200640 1 -- 172.21.15.97:0/2415121359 learned_addr learned my addr 172.21.15.97:0/2415121359 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.469+0000 7efd3be00640 1 --2- 172.21.15.97:0/2415121359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.469+0000 7efd41000640 1 -- 172.21.15.97:0/2415121359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 msgr2=0x7efd3c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.469+0000 7efd41000640 1 --2- 172.21.15.97:0/2415121359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.469+0000 7efd41000640 1 -- 172.21.15.97:0/2415121359 >> v1:172.21.15.97:6789/0 conn(0x7efd3c0abcf0 legacy=0x7efd3c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.469+0000 7efd41000640 1 -- 172.21.15.97:0/2415121359 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0ceb60 con 0x7efd3c0cbf20 2024-01-31T05:02:18.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.469+0000 7efd3b400640 1 -- 172.21.15.97:0/2415121359 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661201489 0 0) 0x7efd3c06cd20 con 0x7efd3c0abcf0 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.470+0000 7efd41000640 1 --2- 172.21.15.97:0/2415121359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c0ce430 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd34009800 tx=0x7efd3402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35057b8ea1d05c60 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.470+0000 7efd3b400640 1 -- 172.21.15.97:0/2415121359 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34031800 con 0x7efd3c0cbf20 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.470+0000 7efd3b400640 1 -- 172.21.15.97:0/2415121359 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34031960 con 0x7efd3c0cbf20 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.470+0000 7efd3b400640 1 -- 172.21.15.97:0/2415121359 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34031cf0 con 0x7efd3c0cbf20 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 -- 172.21.15.97:0/2415121359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 msgr2=0x7efd3c0ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 --2- 172.21.15.97:0/2415121359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c0ce430 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efd34009800 tx=0x7efd3402b8a0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 -- 172.21.15.97:0/2415121359 shutdown_connections 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 --2- 172.21.15.97:0/2415121359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c0ce430 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 --2- 172.21.15.97:0/2415121359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 -- 172.21.15.97:0/2415121359 >> 172.21.15.97:0/2415121359 conn(0x7efd3c0b0a10 msgr2=0x7efd3c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 -- 172.21.15.97:0/2415121359 shutdown_connections 2024-01-31T05:02:18.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.471+0000 7efd43000640 1 -- 172.21.15.97:0/2415121359 wait complete. 2024-01-31T05:02:18.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.473+0000 7efd43000640 1 Processor -- start 2024-01-31T05:02:18.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.473+0000 7efd43000640 1 -- start start 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd43000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd3c0abcf0 0x7efd3c1468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd43000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c09bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd43000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c09c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd43000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c146e20 con 0x7efd3c0cbf20 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd43000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c09a810 con 0x7efd3c0abcf0 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd43000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c09aaf0 con 0x7efd3c15b470 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd33200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c09bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd33200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c09bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45246/0 (socket says 172.21.15.97:45246) 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd33200640 1 -- 172.21.15.97:0/2353837213 learned_addr learned my addr 172.21.15.97:0/2353837213 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:18.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd33200640 1 -- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 msgr2=0x7efd3c09c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.474+0000 7efd33200640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c09c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd33200640 1 -- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd3c0abcf0 msgr2=0x7efd3c1468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd3be00640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd3c0abcf0 0x7efd3c1468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd33200640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd3c0abcf0 0x7efd3c1468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd33200640 1 -- 172.21.15.97:0/2353837213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009400 con 0x7efd3c0cbf20 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd33200640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c09bb00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd2400c4b0 tx=0x7efd2400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd240074e0 con 0x7efd3c0cbf20 2024-01-31T05:02:18.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd24007660 con 0x7efd3c0cbf20 2024-01-31T05:02:18.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24004a20 con 0x7efd3c0cbf20 2024-01-31T05:02:18.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c09add0 con 0x7efd3c0cbf20 2024-01-31T05:02:18.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.475+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c09b390 con 0x7efd3c0cbf20 2024-01-31T05:02:18.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.477+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c151c50 con 0x7efd3c0cbf20 2024-01-31T05:02:18.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.477+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7efd24004b80 con 0x7efd3c0cbf20 2024-01-31T05:02:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.478+0000 7efd38c00640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd1406aed0 0x7efd1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.479+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6547+0+0 (secure 0 0 0) 0x7efd24088f00 con 0x7efd3c0cbf20 2024-01-31T05:02:18.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.479+0000 7efd3be00640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd1406aed0 0x7efd1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.479+0000 7efd3be00640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd1406aed0 0x7efd1406d390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009ee0 tx=0x7efd2c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.483+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7efd24053b40 con 0x7efd3c0cbf20 2024-01-31T05:02:18.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.717+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efd3c067830 con 0x7efd3c0cbf20 2024-01-31T05:02:18.717 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-01-31T05:02:18.717+0000 7f85e2800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:02:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.721+0000 7efd38c00640 1 -- 172.21.15.97:0/2353837213 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd24018020 con 0x7efd3c0cbf20 2024-01-31T05:02:18.720 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-31T05:02:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.724+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd1406aed0 msgr2=0x7efd1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.724+0000 7efd43000640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd1406aed0 0x7efd1406d390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009ee0 tx=0x7efd2c00b040 comp rx=0 tx=0).stop 2024-01-31T05:02:18.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.724+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 msgr2=0x7efd3c09bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.724+0000 7efd43000640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c09bb00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efd2400c4b0 tx=0x7efd2400c970 comp rx=0 tx=0).stop 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 shutdown_connections 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd3c15b470 0x7efd3c09c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7efd1406aed0 0x7efd1406d390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd3c0cbf20 0x7efd3c09bb00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 --2- 172.21.15.97:0/2353837213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd3c0abcf0 0x7efd3c1468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 >> 172.21.15.97:0/2353837213 conn(0x7efd3c0b0a10 msgr2=0x7efd3c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 shutdown_connections 2024-01-31T05:02:18.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.725+0000 7efd43000640 1 -- 172.21.15.97:0/2353837213 wait complete. 2024-01-31T05:02:18.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-31T05:02:18.752 INFO:tasks.ceph.mds.a.smithi097.stderr: -5393> 2024-01-31T05:00:50.334+0000 7f8038e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:02:18.827 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-01-31T05:02:18.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.937+0000 7fba5e200640 1 Processor -- start 2024-01-31T05:02:18.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.937+0000 7fba5e200640 1 -- start start 2024-01-31T05:02:18.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.938+0000 7fba5e200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fba5815d590 0x7fba5815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.938+0000 7fba5e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba580cfae0 0x7fba580cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba5e200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba58161080 0x7fba58161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba5e200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580d0460 con 0x7fba58161080 2024-01-31T05:02:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba5e200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580cded0 con 0x7fba5815d590 2024-01-31T05:02:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba5e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5815d970 con 0x7fba580cfae0 2024-01-31T05:02:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba58161080 0x7fba58161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba58161080 0x7fba58161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45288/0 (socket says 172.21.15.97:45288) 2024-01-31T05:02:18.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 -- 172.21.15.97:0/1742979895 learned_addr learned my addr 172.21.15.97:0/1742979895 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:18.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba56a00640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba580cfae0 0x7fba580cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 -- 172.21.15.97:0/1742979895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba580cfae0 msgr2=0x7fba580cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:18.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba580cfae0 0x7fba580cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 -- 172.21.15.97:0/1742979895 >> v2:172.21.15.188:3300/0 conn(0x7fba5815d590 msgr2=0x7fba5815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57400640 1 --2- 172.21.15.97:0/1742979895 >> v2:172.21.15.188:3300/0 conn(0x7fba5815d590 0x7fba5815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 --2- 172.21.15.97:0/1742979895 >> v2:172.21.15.188:3300/0 conn(0x7fba5815d590 0x7fba5815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.939+0000 7fba57e00640 1 -- 172.21.15.97:0/1742979895 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580cf210 con 0x7fba58161080 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.940+0000 7fba57400640 1 --2- 172.21.15.97:0/1742979895 >> v2:172.21.15.188:3300/0 conn(0x7fba5815d590 0x7fba5815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.940+0000 7fba56a00640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba580cfae0 0x7fba580cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.941+0000 7fba57e00640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba58161080 0x7fba58161510 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fba5808b830 tx=0x7fba5002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed0fe077a6a052d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.941+0000 7fba56000640 1 -- 172.21.15.97:0/1742979895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5002cb00 con 0x7fba58161080 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.941+0000 7fba56000640 1 -- 172.21.15.97:0/1742979895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5002cc60 con 0x7fba58161080 2024-01-31T05:02:18.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.941+0000 7fba56000640 1 -- 172.21.15.97:0/1742979895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500343f0 con 0x7fba58161080 2024-01-31T05:02:18.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 -- 172.21.15.97:0/1742979895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba58161080 msgr2=0x7fba58161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba58161080 0x7fba58161510 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fba5808b830 tx=0x7fba5002bf60 comp rx=0 tx=0).stop 2024-01-31T05:02:18.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 -- 172.21.15.97:0/1742979895 shutdown_connections 2024-01-31T05:02:18.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba58161080 0x7fba58161510 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 --2- 172.21.15.97:0/1742979895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba580cfae0 0x7fba580cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 --2- 172.21.15.97:0/1742979895 >> v2:172.21.15.188:3300/0 conn(0x7fba5815d590 0x7fba5815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 -- 172.21.15.97:0/1742979895 >> 172.21.15.97:0/1742979895 conn(0x7fba5809c560 msgr2=0x7fba580d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 -- 172.21.15.97:0/1742979895 shutdown_connections 2024-01-31T05:02:18.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.942+0000 7fba5e200640 1 -- 172.21.15.97:0/1742979895 wait complete. 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.943+0000 7fba5e200640 1 Processor -- start 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.943+0000 7fba5e200640 1 -- start start 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.944+0000 7fba5e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 0x7fba5815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.944+0000 7fba5e200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5815d590 0x7fba580cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.944+0000 7fba5e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba58161080 0x7fba580ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.944+0000 7fba5e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba581f6380 con 0x7fba580cfae0 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.944+0000 7fba5e200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba581f6660 con 0x7fba5815d590 2024-01-31T05:02:18.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.944+0000 7fba5e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba581f6940 con 0x7fba58161080 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 0x7fba5815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 0x7fba5815dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45304/0 (socket says 172.21.15.97:45304) 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 -- 172.21.15.97:0/1304184542 learned_addr learned my addr 172.21.15.97:0/1304184542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 -- 172.21.15.97:0/1304184542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba58161080 msgr2=0x7fba580ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba58161080 0x7fba580ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 -- 172.21.15.97:0/1304184542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5815d590 msgr2=0x7fba580cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5815d590 0x7fba580cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 -- 172.21.15.97:0/1304184542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba50009b30 con 0x7fba580cfae0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba57400640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 0x7fba5815dfa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fba4800c7f0 tx=0x7fba4800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48007a30 con 0x7fba580cfae0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba48007b90 con 0x7fba580cfae0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.945+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4800ce40 con 0x7fba580cfae0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.946+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba581f6c20 con 0x7fba580cfae0 2024-01-31T05:02:18.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.946+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba580c7a70 con 0x7fba580cfae0 2024-01-31T05:02:18.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.947+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba580ce9d0 con 0x7fba580cfae0 2024-01-31T05:02:18.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.948+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fba480113f0 con 0x7fba580cfae0 2024-01-31T05:02:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.948+0000 7fba4f400640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3006aed0 0x7fba3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.949+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6633+0+0 (secure 0 0 0) 0x7fba48089b00 con 0x7fba580cfae0 2024-01-31T05:02:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.949+0000 7fba56a00640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3006aed0 0x7fba3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:18.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.949+0000 7fba56a00640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3006aed0 0x7fba3006d390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba40006fd0 tx=0x7fba40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:18.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:18.953+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fba48054630 con 0x7fba580cfae0 2024-01-31T05:02:19.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.167+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fba581f64c0 con 0x7fba580cfae0 2024-01-31T05:02:19.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.724+0000 7fba4f400640 1 -- 172.21.15.97:0/1304184542 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fba581f64c0 con 0x7fba580cfae0 2024-01-31T05:02:19.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3006aed0 msgr2=0x7fba3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:19.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3006aed0 0x7fba3006d390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba40006fd0 tx=0x7fba40008040 comp rx=0 tx=0).stop 2024-01-31T05:02:19.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 msgr2=0x7fba5815dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 0x7fba5815dfa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fba4800c7f0 tx=0x7fba4800cbf0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 shutdown_connections 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba58161080 0x7fba580ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba5815d590 0x7fba580cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fba3006aed0 0x7fba3006d390 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 --2- 172.21.15.97:0/1304184542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba580cfae0 0x7fba5815dfa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.728+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 >> 172.21.15.97:0/1304184542 conn(0x7fba5809c560 msgr2=0x7fba5815f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.729+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 shutdown_connections 2024-01-31T05:02:19.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.729+0000 7fba5e200640 1 -- 172.21.15.97:0/1304184542 wait complete. 2024-01-31T05:02:19.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-31T05:02:19.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.948+0000 7f357e800640 1 Processor -- start 2024-01-31T05:02:19.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.948+0000 7f357e800640 1 -- start start 2024-01-31T05:02:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.949+0000 7f357e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.949+0000 7f357e800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35780c9430 0x7f3578071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.949+0000 7f357e800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35780cab10 con 0x7f357815a230 2024-01-31T05:02:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.949+0000 7f357e800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35780c99f0 con 0x7f35780c9430 2024-01-31T05:02:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.949+0000 7f357e800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357806c720 con 0x7f357806e1e0 2024-01-31T05:02:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.950+0000 7f356e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.950+0000 7f356e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:42732/0 (socket says 172.21.15.97:42732) 2024-01-31T05:02:19.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.950+0000 7f356e800640 1 -- 172.21.15.97:0/3528606449 learned_addr learned my addr 172.21.15.97:0/3528606449 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.950+0000 7f3577e00640 1 --2- 172.21.15.97:0/3528606449 >> v2:172.21.15.188:3300/0 conn(0x7f35780c9430 0x7f3578071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.951+0000 7f356e800640 1 -- 172.21.15.97:0/3528606449 >> v2:172.21.15.188:3300/0 conn(0x7f35780c9430 msgr2=0x7f3578071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.951+0000 7f356e800640 1 --2- 172.21.15.97:0/3528606449 >> v2:172.21.15.188:3300/0 conn(0x7f35780c9430 0x7f3578071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.951+0000 7f356e800640 1 -- 172.21.15.97:0/3528606449 >> v1:172.21.15.188:6789/0 conn(0x7f357815a230 legacy=0x7f357806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.951+0000 7f356e800640 1 -- 172.21.15.97:0/3528606449 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35780ae050 con 0x7f357806e1e0 2024-01-31T05:02:19.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.952+0000 7f356e800640 1 --2- 172.21.15.97:0/3528606449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780ca5d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3560009870 tx=0x7f356002db00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ded7d3c0dbe4478 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.953+0000 7f3576a00640 1 -- 172.21.15.97:0/3528606449 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356002f8c0 con 0x7f357806e1e0 2024-01-31T05:02:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.953+0000 7f3576a00640 1 -- 172.21.15.97:0/3528606449 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356002fa20 con 0x7f357806e1e0 2024-01-31T05:02:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.953+0000 7f3576a00640 1 -- 172.21.15.97:0/3528606449 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356002fdb0 con 0x7f357806e1e0 2024-01-31T05:02:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.953+0000 7f357e800640 1 -- 172.21.15.97:0/3528606449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 msgr2=0x7f35780ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.953+0000 7f357e800640 1 --2- 172.21.15.97:0/3528606449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780ca5d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3560009870 tx=0x7f356002db00 comp rx=0 tx=0).stop 2024-01-31T05:02:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.954+0000 7f357e800640 1 -- 172.21.15.97:0/3528606449 shutdown_connections 2024-01-31T05:02:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.954+0000 7f357e800640 1 --2- 172.21.15.97:0/3528606449 >> v2:172.21.15.188:3300/0 conn(0x7f35780c9430 0x7f3578071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.954+0000 7f357e800640 1 --2- 172.21.15.97:0/3528606449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780ca5d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.954+0000 7f357e800640 1 -- 172.21.15.97:0/3528606449 >> 172.21.15.97:0/3528606449 conn(0x7f35780cede0 msgr2=0x7f35780cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.954+0000 7f357e800640 1 -- 172.21.15.97:0/3528606449 shutdown_connections 2024-01-31T05:02:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.954+0000 7f357e800640 1 -- 172.21.15.97:0/3528606449 wait complete. 2024-01-31T05:02:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.955+0000 7f357e800640 1 Processor -- start 2024-01-31T05:02:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.956+0000 7f357e800640 1 -- start start 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f357e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f357e800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35780c9430 0x7f35780b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f357e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 0x7f3578151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f357e800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357809a400 con 0x7f357815a230 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f357e800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35780b3d70 con 0x7f35780c9430 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f357e800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3578099a60 con 0x7f357806e1e0 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f3577e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 0x7f3578151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f3577e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 0x7f3578151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45306/0 (socket says 172.21.15.97:45306) 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f3577e00640 1 -- 172.21.15.97:0/2614867519 learned_addr learned my addr 172.21.15.97:0/2614867519 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f3577400640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.957+0000 7f356e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35780c9430 0x7f35780b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f3577e00640 1 -- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 msgr2=0x7f35780c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:19.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f3577e00640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f3577e00640 1 -- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35780c9430 msgr2=0x7f35780b32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f3577e00640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35780c9430 0x7f35780b32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f3577e00640 1 -- 172.21.15.97:0/2614867519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600094d0 con 0x7f357815a230 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f3577e00640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 0x7f3578151350 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f357000efc0 tx=0x7f357000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570009b10 con 0x7f357815a230 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357000f040 con 0x7f357815a230 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35700146c0 con 0x7f357815a230 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3578099d40 con 0x7f357815a230 2024-01-31T05:02:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.958+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35780c85a0 con 0x7f357815a230 2024-01-31T05:02:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.959+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357809a540 con 0x7f357815a230 2024-01-31T05:02:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.960+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3570014820 con 0x7f357815a230 2024-01-31T05:02:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.961+0000 7f356fe00640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f355006aed0 0x7f355006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.961+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6633+0+0 (secure 0 0 0) 0x7f357008a540 con 0x7f357815a230 2024-01-31T05:02:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.961+0000 7f3577400640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f355006aed0 0x7f355006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:19.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.962+0000 7f3577400640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f355006aed0 0x7f355006d390 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3568007fd0 tx=0x7f3568009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:19.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:19.965+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3570055060 con 0x7f357815a230 2024-01-31T05:02:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:20.160+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f35780ce740 con 0x7f357815a230 2024-01-31T05:02:21.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.083+0000 7f356fe00640 1 -- 172.21.15.97:0/2614867519 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3570058680 con 0x7f357815a230 2024-01-31T05:02:21.084 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-01-31T05:02:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.087+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f355006aed0 msgr2=0x7f355006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.087+0000 7f357e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f355006aed0 0x7f355006d390 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3568007fd0 tx=0x7f3568009040 comp rx=0 tx=0).stop 2024-01-31T05:02:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.087+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 msgr2=0x7f3578151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.087+0000 7f357e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 0x7f3578151350 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f357000efc0 tx=0x7f357000c490 comp rx=0 tx=0).stop 2024-01-31T05:02:21.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 shutdown_connections 2024-01-31T05:02:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f355006aed0 0x7f355006d390 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357815a230 0x7f3578151350 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35780c9430 0x7f35780b32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 --2- 172.21.15.97:0/2614867519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f357806e1e0 0x7f35780c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 >> 172.21.15.97:0/2614867519 conn(0x7f35780cede0 msgr2=0x7f35780abdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:21.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.088+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 shutdown_connections 2024-01-31T05:02:21.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.089+0000 7f357e800640 1 -- 172.21.15.97:0/2614867519 wait complete. 2024-01-31T05:02:21.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-31T05:02:21.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.321+0000 7f6a0ba00640 1 Processor -- start 2024-01-31T05:02:21.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.321+0000 7f6a0ba00640 1 -- start start 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.322+0000 7f6a0ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a040ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.322+0000 7f6a0ba00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a040c9430 0x7f6a0406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.322+0000 7f6a0ba00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a0406e010 0x7f6a0415de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.322+0000 7f6a0ba00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a040cab10 con 0x7f6a040c9430 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.322+0000 7f6a0ba00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0406e460 con 0x7f6a0406e010 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.322+0000 7f6a0ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a040c99f0 con 0x7f6a0415a230 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a040c9430 0x7f6a0406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6a040c9430 0x7f6a0406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45310/0 (socket says 172.21.15.97:45310) 2024-01-31T05:02:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 -- 172.21.15.97:0/1179260679 learned_addr learned my addr 172.21.15.97:0/1179260679 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 -- 172.21.15.97:0/1179260679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 msgr2=0x7f6a040ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 --2- 172.21.15.97:0/1179260679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a040ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 -- 172.21.15.97:0/1179260679 >> v2:172.21.15.188:3300/0 conn(0x7f6a0406e010 msgr2=0x7f6a0415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 --2- 172.21.15.97:0/1179260679 >> v2:172.21.15.188:3300/0 conn(0x7f6a0406e010 0x7f6a0415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.323+0000 7f69fbc00640 1 -- 172.21.15.97:0/1179260679 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0406c720 con 0x7f6a040c9430 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.324+0000 7f69fbc00640 1 --2- 172.21.15.97:0/1179260679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a040c9430 0x7f6a0406dad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a04070170 tx=0x7f69ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906cd00dededadbc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.324+0000 7f6a03e00640 1 -- 172.21.15.97:0/1179260679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec0317e0 con 0x7f6a040c9430 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.324+0000 7f6a03e00640 1 -- 172.21.15.97:0/1179260679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec031940 con 0x7f6a040c9430 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a03e00640 1 -- 172.21.15.97:0/1179260679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec031cd0 con 0x7f6a040c9430 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 -- 172.21.15.97:0/1179260679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a040c9430 msgr2=0x7f6a0406dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/1179260679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a040c9430 0x7f6a0406dad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6a04070170 tx=0x7f69ec02b880 comp rx=0 tx=0).stop 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 -- 172.21.15.97:0/1179260679 shutdown_connections 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/1179260679 >> v2:172.21.15.188:3300/0 conn(0x7f6a0406e010 0x7f6a0415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/1179260679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a040c9430 0x7f6a0406dad0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/1179260679 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a040ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 -- 172.21.15.97:0/1179260679 >> 172.21.15.97:0/1179260679 conn(0x7f6a040cede0 msgr2=0x7f6a040cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:21.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 -- 172.21.15.97:0/1179260679 shutdown_connections 2024-01-31T05:02:21.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.325+0000 7f6a0ba00640 1 -- 172.21.15.97:0/1179260679 wait complete. 2024-01-31T05:02:21.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.327+0000 7f6a0ba00640 1 Processor -- start 2024-01-31T05:02:21.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.327+0000 7f6a0ba00640 1 -- start start 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a0ba00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 0x7f6a040cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a0ba00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a040c9430 0x7f6a04150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a0ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a041f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a0ba00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04157640 con 0x7f6a0406e010 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a0ba00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04157920 con 0x7f6a040c9430 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a0ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a04150860 con 0x7f6a0415a230 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a09000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 0x7f6a040cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a09000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 0x7f6a040cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45312/0 (socket says 172.21.15.97:45312) 2024-01-31T05:02:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.328+0000 7f6a09000640 1 -- 172.21.15.97:0/3713545005 learned_addr learned my addr 172.21.15.97:0/3713545005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09a00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a041f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f69fbc00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a040c9430 0x7f6a04150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09000640 1 -- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 msgr2=0x7f6a041f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09000640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a041f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09000640 1 -- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a040c9430 msgr2=0x7f6a04150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09000640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a040c9430 0x7f6a04150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09000640 1 -- 172.21.15.97:0/3713545005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec0094d0 con 0x7f6a0406e010 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a09000640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 0x7f6a040cced0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f69f400b440 tx=0x7f69f400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4011890 con 0x7f6a0406e010 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f40119f0 con 0x7f6a0406e010 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a041f40d0 con 0x7f6a0406e010 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4011da0 con 0x7f6a0406e010 2024-01-31T05:02:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.329+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a041f4690 con 0x7f6a0406e010 2024-01-31T05:02:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.330+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a04067830 con 0x7f6a0406e010 2024-01-31T05:02:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.331+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f69f401a400 con 0x7f6a0406e010 2024-01-31T05:02:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.332+0000 7f6a01600640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69dc06aed0 0x7f69dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:21.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.332+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f69f4089b10 con 0x7f6a0406e010 2024-01-31T05:02:21.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.333+0000 7f69fbc00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69dc06aed0 0x7f69dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:21.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.333+0000 7f69fbc00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69dc06aed0 0x7f69dc06d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f69ec036000 tx=0x7f69ec03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:21.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.337+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f69f4054860 con 0x7f6a0406e010 2024-01-31T05:02:21.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:21.524+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a040abd90 con 0x7f6a0406e010 2024-01-31T05:02:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.109+0000 7f6a01600640 1 -- 172.21.15.97:0/3713545005 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7f69f4015020 con 0x7f6a0406e010 2024-01-31T05:02:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-01-31T05:02:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.112+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69dc06aed0 msgr2=0x7f69dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:22.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.112+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69dc06aed0 0x7f69dc06d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f69ec036000 tx=0x7f69ec03d040 comp rx=0 tx=0).stop 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.112+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 msgr2=0x7f6a040cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.112+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 0x7f6a040cced0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f69f400b440 tx=0x7f69f400b900 comp rx=0 tx=0).stop 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 shutdown_connections 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a0415a230 0x7f6a041f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a040c9430 0x7f6a04150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69dc06aed0 0x7f69dc06d390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 --2- 172.21.15.97:0/3713545005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a0406e010 0x7f6a040cced0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 >> 172.21.15.97:0/3713545005 conn(0x7f6a040cede0 msgr2=0x7f6a0406c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 shutdown_connections 2024-01-31T05:02:22.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.113+0000 7f6a0ba00640 1 -- 172.21.15.97:0/3713545005 wait complete. 2024-01-31T05:02:22.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-01-31T05:02:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.340+0000 7f3644600640 1 Processor -- start 2024-01-31T05:02:22.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.340+0000 7f3644600640 1 -- start start 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.341+0000 7f3644600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f363c06e1e0 0x7f363c0ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.341+0000 7f3644600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f363c0c9430 0x7f363c071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.341+0000 7f3644600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0cab10 con 0x7f363c15a230 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.341+0000 7f3644600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0c99f0 con 0x7f363c06e1e0 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.341+0000 7f3644600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c06c720 con 0x7f363c0c9430 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f363c15a230 0x7f363c06dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41990/0 (socket says 172.21.15.97:41990) 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641c00640 1 -- 172.21.15.97:0/1941269687 learned_addr learned my addr 172.21.15.97:0/1941269687 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641200640 1 --2- 172.21.15.97:0/1941269687 >> v2:172.21.15.97:3300/0 conn(0x7f363c06e1e0 0x7f363c0ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f363be00640 1 -- 172.21.15.97:0/1941269687 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224860122 0 0) 0x7f363c0cab10 con 0x7f363c15a230 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f363be00640 1 -- 172.21.15.97:0/1941269687 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f361c003520 con 0x7f363c15a230 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3642600640 1 --2- 172.21.15.97:0/1941269687 >> v2:172.21.15.188:3300/0 conn(0x7f363c0c9430 0x7f363c071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:22.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641200640 1 -- 172.21.15.97:0/1941269687 >> v2:172.21.15.188:3300/0 conn(0x7f363c0c9430 msgr2=0x7f363c071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641200640 1 --2- 172.21.15.97:0/1941269687 >> v2:172.21.15.188:3300/0 conn(0x7f363c0c9430 0x7f363c071450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641200640 1 -- 172.21.15.97:0/1941269687 >> v1:172.21.15.97:6789/0 conn(0x7f363c15a230 legacy=0x7f363c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.342+0000 7f3641200640 1 -- 172.21.15.97:0/1941269687 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0ae050 con 0x7f363c06e1e0 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.344+0000 7f3641200640 1 --2- 172.21.15.97:0/1941269687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c06e1e0 0x7f363c0ca5d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f363c04faf0 tx=0x7f362402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9829ab744eca94cd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.344+0000 7f363be00640 1 -- 172.21.15.97:0/1941269687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362402e070 con 0x7f363c06e1e0 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.344+0000 7f363be00640 1 -- 172.21.15.97:0/1941269687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36240315a0 con 0x7f363c06e1e0 2024-01-31T05:02:22.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.344+0000 7f363be00640 1 -- 172.21.15.97:0/1941269687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624031930 con 0x7f363c06e1e0 2024-01-31T05:02:22.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 -- 172.21.15.97:0/1941269687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c06e1e0 msgr2=0x7f363c0ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:22.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 --2- 172.21.15.97:0/1941269687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c06e1e0 0x7f363c0ca5d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f363c04faf0 tx=0x7f362402b840 comp rx=0 tx=0).stop 2024-01-31T05:02:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 -- 172.21.15.97:0/1941269687 shutdown_connections 2024-01-31T05:02:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 --2- 172.21.15.97:0/1941269687 >> v2:172.21.15.188:3300/0 conn(0x7f363c0c9430 0x7f363c071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 --2- 172.21.15.97:0/1941269687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c06e1e0 0x7f363c0ca5d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 -- 172.21.15.97:0/1941269687 >> 172.21.15.97:0/1941269687 conn(0x7f363c0cede0 msgr2=0x7f363c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 -- 172.21.15.97:0/1941269687 shutdown_connections 2024-01-31T05:02:22.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.345+0000 7f3644600640 1 -- 172.21.15.97:0/1941269687 wait complete. 2024-01-31T05:02:22.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.346+0000 7f3644600640 1 Processor -- start 2024-01-31T05:02:22.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 -- start start 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f363c06e1e0 0x7f363c0ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 0x7f363c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c15a230 0x7f363c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c14f3c0 con 0x7f363c0c9430 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c14e800 con 0x7f363c15a230 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.347+0000 7f3644600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c09c600 con 0x7f363c06e1e0 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 0x7f363c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 0x7f363c14e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45324/0 (socket says 172.21.15.97:45324) 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 -- 172.21.15.97:0/3725050466 learned_addr learned my addr 172.21.15.97:0/3725050466 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 -- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f363c06e1e0 msgr2=0x7f363c0ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f363c06e1e0 0x7f363c0ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 -- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c15a230 msgr2=0x7f363c0cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:22.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3642600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c15a230 0x7f363c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:22.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c15a230 0x7f363c0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:22.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.348+0000 7f3641200640 1 -- 172.21.15.97:0/3725050466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3624009400 con 0x7f363c0c9430 2024-01-31T05:02:22.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.349+0000 7f3641200640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 0x7f363c14e2c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f36240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:22.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.349+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624032840 con 0x7f363c0c9430 2024-01-31T05:02:22.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.349+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36240329c0 con 0x7f363c0c9430 2024-01-31T05:02:22.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.349+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624032d70 con 0x7f363c0c9430 2024-01-31T05:02:22.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.349+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0cd680 con 0x7f363c0c9430 2024-01-31T05:02:22.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.349+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c151100 con 0x7f363c0c9430 2024-01-31T05:02:22.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.350+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c0ce4d0 con 0x7f363c0c9430 2024-01-31T05:02:22.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.351+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f362404d3f0 con 0x7f363c0c9430 2024-01-31T05:02:22.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.352+0000 7f3639600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f361406aed0 0x7f361406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:22.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.352+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f36240ab230 con 0x7f363c0c9430 2024-01-31T05:02:22.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.352+0000 7f3641c00640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f361406aed0 0x7f361406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:22.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.353+0000 7f3641c00640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f361406aed0 0x7f361406d390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f362c000f60 tx=0x7f362c0078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:22.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.356+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3624076030 con 0x7f363c0c9430 2024-01-31T05:02:22.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:22.549+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f363c067830 con 0x7f363c0c9430 2024-01-31T05:02:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.123+0000 7f3639600640 1 -- 172.21.15.97:0/3725050466 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7f3624079650 con 0x7f363c0c9430 2024-01-31T05:02:23.123 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data_ec' removed 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f361406aed0 msgr2=0x7f361406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f361406aed0 0x7f361406d390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f362c000f60 tx=0x7f362c0078f0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 msgr2=0x7f363c14e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 0x7f363c14e2c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3624000c00 tx=0x7f36240046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 shutdown_connections 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f363c15a230 0x7f363c0cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f361406aed0 0x7f361406d390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f363c0c9430 0x7f363c14e2c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 --2- 172.21.15.97:0/3725050466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f363c06e1e0 0x7f363c0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 >> 172.21.15.97:0/3725050466 conn(0x7f363c0cede0 msgr2=0x7f363c06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 shutdown_connections 2024-01-31T05:02:23.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.127+0000 7f3644600640 1 -- 172.21.15.97:0/3725050466 wait complete. 2024-01-31T05:02:23.149 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-31T05:02:23.150 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-31T05:02:23.150 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-31T05:02:23.150 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:23.150 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-31T05:02:23.150 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-31T05:02:23.150 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-31T05:02:23.151 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:23.151 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-31T05:02:23.151 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-31T05:02:23.151 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-31T05:02:23.151 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:23.151 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-31T05:02:23.151 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-31T05:02:23.152 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-31T05:02:23.152 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:23.168 INFO:tasks.ceph.mds.c:Stopped 2024-01-31T05:02:23.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-31T05:02:23.171 INFO:tasks.ceph.mds.a:Stopped 2024-01-31T05:02:23.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-31T05:02:23.173 INFO:tasks.ceph.mds.d:Stopped 2024-01-31T05:02:23.173 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-31T05:02:23.173 INFO:tasks.ceph.mds.b:Stopped 2024-01-31T05:02:23.174 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-31T05:02:23.218 INFO:tasks.ceph.mds.c:Started 2024-01-31T05:02:23.218 DEBUG:teuthology.parallel:result is None 2024-01-31T05:02:23.219 INFO:tasks.ceph.mds.d:Started 2024-01-31T05:02:23.219 DEBUG:teuthology.parallel:result is None 2024-01-31T05:02:23.224 INFO:tasks.ceph.mds.a:Started 2024-01-31T05:02:23.224 DEBUG:teuthology.parallel:result is None 2024-01-31T05:02:23.225 INFO:tasks.ceph.mds.b:Started 2024-01-31T05:02:23.225 DEBUG:teuthology.parallel:result is None 2024-01-31T05:02:23.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-31T05:02:23.376 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-01-31T05:02:23.376 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-31T05:02:23.376 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-01-31T05:02:23.376 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-31T05:02:23.377 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-01-31T05:02:23.377+0000 7f5ff5e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-01-31T05:02:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.414+0000 7f2a31400640 1 Processor -- start 2024-01-31T05:02:23.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.414+0000 7f2a31400640 1 -- start start 2024-01-31T05:02:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.415+0000 7f2a31400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a2c0cbf20 0x7f2a2c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.415+0000 7f2a31400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.415+0000 7f2a31400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2c06cd20 con 0x7f2a2c15b470 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.415+0000 7f2a31400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2c0ce970 con 0x7f2a2c0cbf20 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.415+0000 7f2a31400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c15b850 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a2c0cbf20 0x7f2a2c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a2c0cbf20 0x7f2a2c0722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45364/0 (socket says 172.21.15.97:45364) 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 -- 172.21.15.97:0/2396639558 learned_addr learned my addr 172.21.15.97:0/2396639558 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a2b200640 1 --2- 172.21.15.97:0/2396639558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 -- 172.21.15.97:0/2396639558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0abcf0 msgr2=0x7f2a2c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 --2- 172.21.15.97:0/2396639558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 -- 172.21.15.97:0/2396639558 >> v1:172.21.15.188:6789/0 conn(0x7f2a2c15b470 legacy=0x7f2a2c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.416+0000 7f2a29e00640 1 -- 172.21.15.97:0/2396639558 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0ceb60 con 0x7f2a2c0cbf20 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.417+0000 7f2a29e00640 1 --2- 172.21.15.97:0/2396639558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c0722c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a140098a0 tx=0x7f2a1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea28f446a50dbbf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.417+0000 7f2a29400640 1 -- 172.21.15.97:0/2396639558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14031640 con 0x7f2a2c0cbf20 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a29400640 1 -- 172.21.15.97:0/2396639558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a140317a0 con 0x7f2a2c0cbf20 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a29400640 1 -- 172.21.15.97:0/2396639558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14031b30 con 0x7f2a2c0cbf20 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 -- 172.21.15.97:0/2396639558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0cbf20 msgr2=0x7f2a2c0722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 --2- 172.21.15.97:0/2396639558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c0722c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a140098a0 tx=0x7f2a1402b8f0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 -- 172.21.15.97:0/2396639558 shutdown_connections 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 --2- 172.21.15.97:0/2396639558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c0722c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 --2- 172.21.15.97:0/2396639558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 -- 172.21.15.97:0/2396639558 >> 172.21.15.97:0/2396639558 conn(0x7f2a2c0b0a10 msgr2=0x7f2a2c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 -- 172.21.15.97:0/2396639558 shutdown_connections 2024-01-31T05:02:23.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.418+0000 7f2a31400640 1 -- 172.21.15.97:0/2396639558 wait complete. 2024-01-31T05:02:23.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.420+0000 7f2a31400640 1 Processor -- start 2024-01-31T05:02:23.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.420+0000 7f2a31400640 1 -- start start 2024-01-31T05:02:23.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a31400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:23.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a31400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:23.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a31400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c15b470 0x7f2a2c09ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:23.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a31400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0aef80 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a31400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0af260 con 0x7f2a2c0cbf20 2024-01-31T05:02:23.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a31400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c0af540 con 0x7f2a2c15b470 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a2a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a2a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45372/0 (socket says 172.21.15.97:45372) 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a2a800640 1 -- 172.21.15.97:0/2306122975 learned_addr learned my addr 172.21.15.97:0/2306122975 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a29e00640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.421+0000 7f2a2b200640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c15b470 0x7f2a2c09ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a2a800640 1 -- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c15b470 msgr2=0x7f2a2c09ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a2a800640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c15b470 0x7f2a2c09ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a2a800640 1 -- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c0cbf20 msgr2=0x7f2a2c1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:23.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a2a800640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a2a800640 1 -- 172.21.15.97:0/2306122975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a140094d0 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a29e00640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a2a800640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ae8d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00b3e0 tx=0x7f2a1c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c011830 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c0119b0 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c011da0 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.422+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c0af820 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.423+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c157a00 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.424+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c06d190 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.424+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2a1c01a410 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.425+0000 7f2a22a00640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a0406aed0 0x7f2a0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.425+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2a1c0893c0 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.426+0000 7f2a29e00640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a0406aed0 0x7f2a0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:23.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.426+0000 7f2a29e00640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a0406aed0 0x7f2a0406d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a14036000 tx=0x7f2a1403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:23.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.430+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2a2c06d190 con 0x7f2a2c0abcf0 2024-01-31T05:02:23.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:23.635+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2a2c06d190 con 0x7f2a2c0abcf0 2024-01-31T05:02:24.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.152+0000 7f2a22a00640 1 -- 172.21.15.97:0/2306122975 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2a2c06d190 con 0x7f2a2c0abcf0 2024-01-31T05:02:24.153 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a0406aed0 msgr2=0x7f2a0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a0406aed0 0x7f2a0406d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a14036000 tx=0x7f2a1403c040 comp rx=0 tx=0).stop 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 msgr2=0x7f2a2c0ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ae8d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00b3e0 tx=0x7f2a1c00b8a0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 shutdown_connections 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c15b470 0x7f2a2c09ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c0cbf20 0x7f2a2c1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a0406aed0 0x7f2a0406d390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 --2- 172.21.15.97:0/2306122975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a2c0abcf0 0x7f2a2c0ae8d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 >> 172.21.15.97:0/2306122975 conn(0x7f2a2c0b0a10 msgr2=0x7f2a2c15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 shutdown_connections 2024-01-31T05:02:24.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.158+0000 7f2a31400640 1 -- 172.21.15.97:0/2306122975 wait complete. 2024-01-31T05:02:24.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-31T05:02:24.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.381+0000 7fd402200640 1 Processor -- start 2024-01-31T05:02:24.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.381+0000 7fd402200640 1 -- start start 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd402200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3fc0cb810 0x7fd3fc0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd402200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3fc15b400 0x7fd3fc06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd402200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3fc06d1a0 con 0x7fd3fc06db70 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd402200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3fc0cc960 con 0x7fd3fc0cb810 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd402200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3fc06c600 con 0x7fd3fc15b400 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fb400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3fc0cb810 0x7fd3fc0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fbe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd3fc06db70 0x7fd3fc0ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42002/0 (socket says 172.21.15.97:42002) 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fbe00640 1 -- 172.21.15.97:0/1772811720 learned_addr learned my addr 172.21.15.97:0/1772811720 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fb400640 1 -- 172.21.15.97:0/1772811720 >> v2:172.21.15.188:3300/0 conn(0x7fd3fc15b400 msgr2=0x7fd3fc06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fb400640 1 --2- 172.21.15.97:0/1772811720 >> v2:172.21.15.188:3300/0 conn(0x7fd3fc15b400 0x7fd3fc06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fb400640 1 -- 172.21.15.97:0/1772811720 >> v1:172.21.15.97:6789/0 conn(0x7fd3fc06db70 legacy=0x7fd3fc0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3fb400640 1 -- 172.21.15.97:0/1772811720 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0cc130 con 0x7fd3fc0cb810 2024-01-31T05:02:24.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.383+0000 7fd3faa00640 1 -- 172.21.15.97:0/1772811720 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839212056 0 0) 0x7fd3fc06d1a0 con 0x7fd3fc06db70 2024-01-31T05:02:24.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd3fb400640 1 --2- 172.21.15.97:0/1772811720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0cbbf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0ae830 tx=0x7fd3ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a97d39fc14945e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:24.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd3faa00640 1 -- 172.21.15.97:0/1772811720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec031830 con 0x7fd3fc0cb810 2024-01-31T05:02:24.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd3faa00640 1 -- 172.21.15.97:0/1772811720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec031990 con 0x7fd3fc0cb810 2024-01-31T05:02:24.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd3faa00640 1 -- 172.21.15.97:0/1772811720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec031d20 con 0x7fd3fc0cb810 2024-01-31T05:02:24.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 -- 172.21.15.97:0/1772811720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 msgr2=0x7fd3fc0cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 --2- 172.21.15.97:0/1772811720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0cbbf0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0ae830 tx=0x7fd3ec02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 -- 172.21.15.97:0/1772811720 shutdown_connections 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 --2- 172.21.15.97:0/1772811720 >> v2:172.21.15.188:3300/0 conn(0x7fd3fc15b400 0x7fd3fc06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 --2- 172.21.15.97:0/1772811720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0cbbf0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 -- 172.21.15.97:0/1772811720 >> 172.21.15.97:0/1772811720 conn(0x7fd3fc0b0a10 msgr2=0x7fd3fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.385+0000 7fd402200640 1 -- 172.21.15.97:0/1772811720 shutdown_connections 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.386+0000 7fd402200640 1 -- 172.21.15.97:0/1772811720 wait complete. 2024-01-31T05:02:24.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.387+0000 7fd402200640 1 Processor -- start 2024-01-31T05:02:24.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.387+0000 7fd402200640 1 -- start start 2024-01-31T05:02:24.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.388+0000 7fd402200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3fc06db70 0x7fd3fc1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.388+0000 7fd402200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.388+0000 7fd402200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3fc15b400 0x7fd3fc0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.388+0000 7fd402200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b2e20 con 0x7fd3fc0cb810 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.388+0000 7fd402200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0ca670 con 0x7fd3fc15b400 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.388+0000 7fd402200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b1e10 con 0x7fd3fc06db70 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45398/0 (socket says 172.21.15.97:45398) 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 -- 172.21.15.97:0/2418336554 learned_addr learned my addr 172.21.15.97:0/2418336554 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3fbe00640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3fc15b400 0x7fd3fc0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 -- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3fc06db70 msgr2=0x7fd3fc1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3fc06db70 0x7fd3fc1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 -- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3fc15b400 msgr2=0x7fd3fc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3fc15b400 0x7fd3fc0ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 -- 172.21.15.97:0/2418336554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec009400 con 0x7fd3fc0cb810 2024-01-31T05:02:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.389+0000 7fd3f3e00640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0ce7f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b440 tx=0x7fd3e400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:24.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.390+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4011890 con 0x7fd3fc0cb810 2024-01-31T05:02:24.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.390+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e40119f0 con 0x7fd3fc0cb810 2024-01-31T05:02:24.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.390+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4011da0 con 0x7fd3fc0cb810 2024-01-31T05:02:24.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.390+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc0b2060 con 0x7fd3fc0cb810 2024-01-31T05:02:24.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.390+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3fc0b2620 con 0x7fd3fc0cb810 2024-01-31T05:02:24.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.391+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3fc0677c0 con 0x7fd3fc0cb810 2024-01-31T05:02:24.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.392+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd3e401a400 con 0x7fd3fc0cb810 2024-01-31T05:02:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.393+0000 7fd3f3400640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d406aed0 0x7fd3d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.393+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd3e40897c0 con 0x7fd3fc0cb810 2024-01-31T05:02:24.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.393+0000 7fd3fb400640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d406aed0 0x7fd3d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.394+0000 7fd3fb400640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d406aed0 0x7fd3d406d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec036000 tx=0x7fd3ec030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:24.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.397+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd3e4054800 con 0x7fd3fc0cb810 2024-01-31T05:02:24.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.657+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd3fc0706f0 con 0x7fd3fc0cb810 2024-01-31T05:02:24.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.659+0000 7fd3f3400640 1 -- 172.21.15.97:0/2418336554 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd3e4015090 con 0x7fd3fc0cb810 2024-01-31T05:02:24.658 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDB07llYd8zLRAAXY4glNCfVX1dTNGZUbfFmQ==", 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llbUoqERAAS5j0bqhOLOEBYeJJAcw1gg==", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llaOqoARAAR5IabzNA8GwYKZPChWBslQ==", 2024-01-31T05:02:24.660 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llaDyVHhAAxCxPKz8CdUNYF2635sVSPg==", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:02:24.661 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDH07llltQODBAAlc7RryuyTlHz0D9phNkn2w==", 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.662 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDK07llEPFhJRAABOCmKBieoS3MS1W+SZKB3A==", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDN07llKeEkLxAAJx3yznrb9PPxr2ok3pcXAw==", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.663 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDQ07ll1vrzOBAAO9IwnSQp37YmgtkSvJ9pTA==", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDY07llYzg7DRAAcbb39NsWEVWRdIornxlExQ==", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.664 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDb07llNPq2FhAASqQSEA/5vGxqRv3LOVIW8Q==", 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-01-31T05:02:24.665 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDe07ll9GdBIBAAW+tnotDCbokaSSouTInb/w==", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDh07llDRo1ORAASF67CwBasg4V3RLF0lvQjw==", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A==", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDD07llR2txBBAA7So8sHU4Wrev8Jt381SvFA==", 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.667 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07ll2MQRNhAAOVu6KYtnRga+FC4ViTTGzA==", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:02:24.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDD07llpqYODxAAKYaORRDDq/CCur8PKSPKNw==", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDA07llafWAIBAAyGsoMQmBd7Wf2ZwG5QCPeA==", 2024-01-31T05:02:24.669 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mds", 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDq07llciMAMRAAoiPFb/y3qUeqgO1lsmdN6w==", 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mds" 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mgr", 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDq07ll9y4AMRAAxVot9ZklbHmlr0rb4irzlw==", 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-osd", 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDq07llXTkAMRAAsOUhg+saKagikRvWuKyLQg==", 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-osd" 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd", 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDq07ll7EEAMRAAZQtUoSmzU4nljFFveKI/WQ==", 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDq07llh0oAMRAA/K4WfF+rnCtoO1WUxqGl8A==", 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rgw", 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDq07llZlMAMRAAykF6OoSPTC22bXilPfBqIg==", 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDB07llSQTaIRAAGEtHw/4lHnjbsB87JXmqYQ==", 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-01-31T05:02:24.673 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDB07llWApVFhAAsTCld+Pxo/BYtCHNVlnvPQ==", 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-01-31T05:02:24.674 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:02:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d406aed0 msgr2=0x7fd3d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d406aed0 0x7fd3d406d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec036000 tx=0x7fd3ec030000 comp rx=0 tx=0).stop 2024-01-31T05:02:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 msgr2=0x7fd3fc0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0ce7f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b440 tx=0x7fd3e400b900 comp rx=0 tx=0).stop 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 shutdown_connections 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3fc15b400 0x7fd3fc0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d406aed0 0x7fd3d406d390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3fc0cb810 0x7fd3fc0ce7f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 --2- 172.21.15.97:0/2418336554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3fc06db70 0x7fd3fc1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.661+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 >> 172.21.15.97:0/2418336554 conn(0x7fd3fc0b0a10 msgr2=0x7fd3fc0abf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.662+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 shutdown_connections 2024-01-31T05:02:24.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.662+0000 7fd402200640 1 -- 172.21.15.97:0/2418336554 wait complete. 2024-01-31T05:02:24.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-31T05:02:24.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.891+0000 7f9a52400640 1 Processor -- start 2024-01-31T05:02:24.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.892+0000 7f9a52400640 1 -- start start 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.893+0000 7f9a52400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c15a600 0x7f9a4c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.893+0000 7f9a52400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a4c06d090 0x7f9a4c06d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.893+0000 7f9a52400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c06c8b0 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.893+0000 7f9a52400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c15b120 con 0x7f9a4c06d090 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.893+0000 7f9a52400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c15b400 con 0x7f9a4c15a600 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.893+0000 7f9a4be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a4c06d090 0x7f9a4c06d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a4c06d090 0x7f9a4c06d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45414/0 (socket says 172.21.15.97:45414) 2024-01-31T05:02:24.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4be00640 1 -- 172.21.15.97:0/1120101106 learned_addr learned my addr 172.21.15.97:0/1120101106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:24.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4aa00640 1 --2- 172.21.15.97:0/1120101106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c15a600 0x7f9a4c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4be00640 1 -- 172.21.15.97:0/1120101106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c15a600 msgr2=0x7f9a4c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:24.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4be00640 1 --2- 172.21.15.97:0/1120101106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c15a600 0x7f9a4c15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4be00640 1 -- 172.21.15.97:0/1120101106 >> v1:172.21.15.188:6789/0 conn(0x7f9a4c0b1d10 legacy=0x7f9a4c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:24.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.894+0000 7f9a4be00640 1 -- 172.21.15.97:0/1120101106 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c06cb70 con 0x7f9a4c06d090 2024-01-31T05:02:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.896+0000 7f9a4be00640 1 --2- 172.21.15.97:0/1120101106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c06d090 0x7f9a4c06d520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009870 tx=0x7f9a4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f0566c49b547a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.896+0000 7f9a4a000640 1 -- 172.21.15.97:0/1120101106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44031640 con 0x7f9a4c06d090 2024-01-31T05:02:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.896+0000 7f9a4a000640 1 -- 172.21.15.97:0/1120101106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a440317a0 con 0x7f9a4c06d090 2024-01-31T05:02:24.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.896+0000 7f9a4a000640 1 -- 172.21.15.97:0/1120101106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a44031b30 con 0x7f9a4c06d090 2024-01-31T05:02:24.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 -- 172.21.15.97:0/1120101106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c06d090 msgr2=0x7f9a4c06d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 --2- 172.21.15.97:0/1120101106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c06d090 0x7f9a4c06d520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009870 tx=0x7f9a4402b8f0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 -- 172.21.15.97:0/1120101106 shutdown_connections 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 --2- 172.21.15.97:0/1120101106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c06d090 0x7f9a4c06d520 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 --2- 172.21.15.97:0/1120101106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c15a600 0x7f9a4c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 -- 172.21.15.97:0/1120101106 >> 172.21.15.97:0/1120101106 conn(0x7f9a4c0b0a10 msgr2=0x7f9a4c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 -- 172.21.15.97:0/1120101106 shutdown_connections 2024-01-31T05:02:24.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.897+0000 7f9a52400640 1 -- 172.21.15.97:0/1120101106 wait complete. 2024-01-31T05:02:24.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.899+0000 7f9a52400640 1 Processor -- start 2024-01-31T05:02:24.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.899+0000 7f9a52400640 1 -- start start 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.900+0000 7f9a52400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c06d090 0x7f9a4c143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.900+0000 7f9a52400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 0x7f9a4c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.900+0000 7f9a52400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c15a600 0x7f9a4c0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.900+0000 7f9a52400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c146ff0 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.900+0000 7f9a52400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1f0d80 con 0x7f9a4c15a600 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.900+0000 7f9a52400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1f1060 con 0x7f9a4c06d090 2024-01-31T05:02:24.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 0x7f9a4c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 0x7f9a4c0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45416/0 (socket says 172.21.15.97:45416) 2024-01-31T05:02:24.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 -- 172.21.15.97:0/1718847741 learned_addr learned my addr 172.21.15.97:0/1718847741 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 -- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c06d090 msgr2=0x7f9a4c143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4b400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c06d090 0x7f9a4c143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4be00640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c15a600 0x7f9a4c0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c06d090 0x7f9a4c143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 -- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c15a600 msgr2=0x7f9a4c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c15a600 0x7f9a4c0ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 -- 172.21.15.97:0/1718847741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440094d0 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.901+0000 7f9a4aa00640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 0x7f9a4c0abcf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400ccd0 tx=0x7f9a3400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.902+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34003990 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.902+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a34003af0 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.902+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a34004020 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.902+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c1f1340 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.902+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c1f1b40 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.903+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c1f1860 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.904+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9a34004180 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.905+0000 7f9a43400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9a2406af00 0x7f9a2406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.905+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9a340897e0 con 0x7f9a4c0b1d10 2024-01-31T05:02:24.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.905+0000 7f9a4b400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9a2406af00 0x7f9a2406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:24.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.906+0000 7f9a4b400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9a2406af00 0x7f9a2406d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c003d90 tx=0x7f9a3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:24.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:24.909+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9a4c1f1860 con 0x7f9a4c0b1d10 2024-01-31T05:02:25.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.181+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9a4c1f1860 con 0x7f9a4c0b1d10 2024-01-31T05:02:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.185+0000 7f9a43400640 1 -- 172.21.15.97:0/1718847741 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9a4c1f1860 con 0x7f9a4c0b1d10 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9a2406af00 msgr2=0x7f9a2406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9a2406af00 0x7f9a2406d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c003d90 tx=0x7f9a3c009040 comp rx=0 tx=0).stop 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 msgr2=0x7f9a4c0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 0x7f9a4c0abcf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400ccd0 tx=0x7f9a3400a5c0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 shutdown_connections 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a4c15a600 0x7f9a4c0ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9a2406af00 0x7f9a2406d3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a4c0b1d10 0x7f9a4c0abcf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 --2- 172.21.15.97:0/1718847741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a4c06d090 0x7f9a4c143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 >> 172.21.15.97:0/1718847741 conn(0x7f9a4c0b0a10 msgr2=0x7f9a4c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:25.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 shutdown_connections 2024-01-31T05:02:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.188+0000 7f9a52400640 1 -- 172.21.15.97:0/1718847741 wait complete. 2024-01-31T05:02:25.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-31T05:02:25.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.396+0000 7f7abd000640 1 Processor -- start 2024-01-31T05:02:25.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.396+0000 7f7abd000640 1 -- start start 2024-01-31T05:02:25.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.398+0000 7f7abd000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7ab815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.398+0000 7f7abd000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab8161ea0 con 0x7f7ab80cfae0 2024-01-31T05:02:25.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.398+0000 7f7abd000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab80cded0 con 0x7f7ab80cff20 2024-01-31T05:02:25.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.398+0000 7f7abd000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab815d970 con 0x7f7ab815d590 2024-01-31T05:02:25.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.398+0000 7f7ab5400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ab80cfae0 0x7f7ab8161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60388/0 (socket says 172.21.15.97:60388) 2024-01-31T05:02:25.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.398+0000 7f7ab5400640 1 -- 172.21.15.97:0/1018357619 learned_addr learned my addr 172.21.15.97:0/1018357619 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424451798 0 0) 0x7f7ab8161ea0 con 0x7f7ab80cfae0 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7ab815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a98003520 con 0x7f7ab80cfae0 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1440845860 0 0) 0x7f7a98003520 con 0x7f7ab80cfae0 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 msgr2=0x7f7ab815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7ab815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 >> v1:172.21.15.188:6789/0 conn(0x7f7ab80cff20 legacy=0x7f7ab8161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80cf210 con 0x7f7ab80cfae0 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f7aa80019f0 con 0x7f7ab80cfae0 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 >> v1:172.21.15.97:6789/0 conn(0x7f7ab80cfae0 legacy=0x7f7ab8161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a980039a0 0x7f7a98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7a980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a98004d00 0x7f7a98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab815d970 con 0x7f7a980039a0 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9800bac0 con 0x7f7a98004d00 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9800bc80 con 0x7f7ab815d590 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.399+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a980039a0 0x7f7a98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5400640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7a980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5e00640 1 -- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 msgr2=0x7f7a980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7a980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5e00640 1 -- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a98004d00 msgr2=0x7f7a98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab6800640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a98004d00 0x7f7a98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a98004d00 0x7f7a98005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5e00640 1 -- 172.21.15.97:0/1018357619 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98003520 con 0x7f7a980039a0 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab5400640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab815d590 0x7f7a980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:25.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.400+0000 7f7ab6800640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a98004d00 0x7f7a98005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.401+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a980039a0 0x7f7a98003dc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009a90 tx=0x7f7a980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891d26af8d3b556b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.401+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa002c680 con 0x7f7a980039a0 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.401+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa002c7e0 con 0x7f7a980039a0 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.401+0000 7f7ab4a00640 1 -- 172.21.15.97:0/1018357619 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa002cb70 con 0x7f7a980039a0 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7abd000640 1 -- 172.21.15.97:0/1018357619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a980039a0 msgr2=0x7f7a98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7abd000640 1 --2- 172.21.15.97:0/1018357619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a980039a0 0x7f7a98003dc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009a90 tx=0x7f7a980057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7ab6800640 1 -- 172.21.15.97:0/1018357619 reap_dead start 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7abd000640 1 -- 172.21.15.97:0/1018357619 shutdown_connections 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7abd000640 1 -- 172.21.15.97:0/1018357619 >> 172.21.15.97:0/1018357619 conn(0x7f7ab809c560 msgr2=0x7f7ab80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7abd000640 1 -- 172.21.15.97:0/1018357619 shutdown_connections 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.402+0000 7f7abd000640 1 -- 172.21.15.97:0/1018357619 wait complete. 2024-01-31T05:02:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.403+0000 7f7abd000640 1 Processor -- start 2024-01-31T05:02:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.404+0000 7f7abd000640 1 -- start start 2024-01-31T05:02:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.404+0000 7f7abd000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a980039a0 0x7f7ab80b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7abd000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 0x7f7ab8147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7abd000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab814a590 0x7f7ab814aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7abd000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab814af60 con 0x7f7ab80b6770 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7abd000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80b81d0 con 0x7f7a980039a0 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7abd000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8148250 con 0x7f7ab814a590 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 0x7f7ab8147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 0x7f7ab8147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36668/0 (socket says 172.21.15.97:36668) 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 -- 172.21.15.97:0/2701982549 learned_addr learned my addr 172.21.15.97:0/2701982549 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 -- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab814a590 msgr2=0x7f7ab814aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab814a590 0x7f7ab814aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 -- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a980039a0 msgr2=0x7f7ab80b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a980039a0 0x7f7ab80b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a980039a0 0x7f7ab80b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.405+0000 7f7ab5400640 1 -- 172.21.15.97:0/2701982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80b6bf0 con 0x7f7ab80b6770 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.406+0000 7f7ab5400640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 0x7f7ab8147d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80021f0 tx=0x7f7aa800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.406+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8004d60 con 0x7f7ab80b6770 2024-01-31T05:02:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.406+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa80070d0 con 0x7f7ab80b6770 2024-01-31T05:02:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.406+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab80b6ed0 con 0x7f7ab80b6770 2024-01-31T05:02:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.406+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8007440 con 0x7f7ab80b6770 2024-01-31T05:02:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.406+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab80b7490 con 0x7f7ab80b6770 2024-01-31T05:02:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.407+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab80677c0 con 0x7f7ab80b6770 2024-01-31T05:02:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.408+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7aa80075a0 con 0x7f7ab80b6770 2024-01-31T05:02:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.409+0000 7f7aae000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7a9006aed0 0x7f7a9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.409+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7aa8089940 con 0x7f7ab80b6770 2024-01-31T05:02:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.409+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7a9006aed0 0x7f7a9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.410+0000 7f7ab5e00640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7a9006aed0 0x7f7a9006d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0036000 tx=0x7f7aa0030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.414+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7aa8054a00 con 0x7f7ab80b6770 2024-01-31T05:02:25.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.674+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f7ab80cece0 con 0x7f7ab80b6770 2024-01-31T05:02:25.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.679+0000 7f7aae000640 1 -- 172.21.15.97:0/2701982549 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7aa8019020 con 0x7f7ab80b6770 2024-01-31T05:02:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.681+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7a9006aed0 msgr2=0x7f7a9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.681+0000 7f7abd000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7a9006aed0 0x7f7a9006d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0036000 tx=0x7f7aa0030000 comp rx=0 tx=0).stop 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 msgr2=0x7f7ab8147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 0x7f7ab8147d10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80021f0 tx=0x7f7aa800b560 comp rx=0 tx=0).stop 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 shutdown_connections 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab814a590 0x7f7ab814aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7a9006aed0 0x7f7a9006d390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab80b6770 0x7f7ab8147d10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 --2- 172.21.15.97:0/2701982549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a980039a0 0x7f7ab80b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 >> 172.21.15.97:0/2701982549 conn(0x7f7ab809c560 msgr2=0x7f7ab815b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 shutdown_connections 2024-01-31T05:02:25.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.682+0000 7f7abd000640 1 -- 172.21.15.97:0/2701982549 wait complete. 2024-01-31T05:02:25.698 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-31T05:02:25.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.903+0000 7fa20fe00640 1 Processor -- start 2024-01-31T05:02:25.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.904+0000 7fa20fe00640 1 -- start start 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2080b1d10 0x7fa2080b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa20815a600 0x7fa20815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20815dc50 0x7fa20815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20fe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2080b2630 con 0x7fa2080b1d10 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20806c8b0 con 0x7fa20815a600 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20815b120 con 0x7fa20815dc50 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2080b1d10 0x7fa2080b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2080b1d10 0x7fa2080b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36672/0 (socket says 172.21.15.97:36672) 2024-01-31T05:02:25.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.905+0000 7fa20d400640 1 -- 172.21.15.97:0/158345894 learned_addr learned my addr 172.21.15.97:0/158345894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:25.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20d400640 1 -- 172.21.15.97:0/158345894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20815dc50 msgr2=0x7fa20815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:25.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20ca00640 1 --2- 172.21.15.97:0/158345894 >> v2:172.21.15.188:3300/0 conn(0x7fa20815a600 0x7fa20815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20de00640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20815dc50 0x7fa20815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20d400640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20815dc50 0x7fa20815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20d400640 1 -- 172.21.15.97:0/158345894 >> v2:172.21.15.188:3300/0 conn(0x7fa20815a600 msgr2=0x7fa20815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20d400640 1 --2- 172.21.15.97:0/158345894 >> v2:172.21.15.188:3300/0 conn(0x7fa20815a600 0x7fa20815aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20d400640 1 -- 172.21.15.97:0/158345894 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20815b400 con 0x7fa2080b1d10 2024-01-31T05:02:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20ca00640 1 --2- 172.21.15.97:0/158345894 >> v2:172.21.15.188:3300/0 conn(0x7fa20815a600 0x7fa20815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.906+0000 7fa20de00640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20815dc50 0x7fa20815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:25.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.907+0000 7fa20d400640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2080b1d10 0x7fa2080b20f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa208158250 tx=0x7fa1f002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871173e7795d4d82 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.907+0000 7fa207e00640 1 -- 172.21.15.97:0/158345894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0031880 con 0x7fa2080b1d10 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.907+0000 7fa207e00640 1 -- 172.21.15.97:0/158345894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f00319e0 con 0x7fa2080b1d10 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.907+0000 7fa207e00640 1 -- 172.21.15.97:0/158345894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0031d70 con 0x7fa2080b1d10 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 -- 172.21.15.97:0/158345894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2080b1d10 msgr2=0x7fa2080b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2080b1d10 0x7fa2080b20f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa208158250 tx=0x7fa1f002b920 comp rx=0 tx=0).stop 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 -- 172.21.15.97:0/158345894 shutdown_connections 2024-01-31T05:02:25.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa20815dc50 0x7fa20815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 --2- 172.21.15.97:0/158345894 >> v2:172.21.15.188:3300/0 conn(0x7fa20815a600 0x7fa20815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 --2- 172.21.15.97:0/158345894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2080b1d10 0x7fa2080b20f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 -- 172.21.15.97:0/158345894 >> 172.21.15.97:0/158345894 conn(0x7fa2080b0a10 msgr2=0x7fa2080b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 -- 172.21.15.97:0/158345894 shutdown_connections 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.908+0000 7fa20fe00640 1 -- 172.21.15.97:0/158345894 wait complete. 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.910+0000 7fa20fe00640 1 Processor -- start 2024-01-31T05:02:25.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.910+0000 7fa20fe00640 1 -- start start 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2080b1d10 0x7fa208149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20815a600 0x7fa208071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 0x7fa2080af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa208070420 con 0x7fa20815dc50 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa208099bc0 con 0x7fa20815a600 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa208070670 con 0x7fa2080b1d10 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 0x7fa2080af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 0x7fa2080af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36686/0 (socket says 172.21.15.97:36686) 2024-01-31T05:02:25.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.911+0000 7fa20de00640 1 -- 172.21.15.97:0/1022425499 learned_addr learned my addr 172.21.15.97:0/1022425499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20d400640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2080b1d10 0x7fa208149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20ca00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20815a600 0x7fa208071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20de00640 1 -- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2080b1d10 msgr2=0x7fa208149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20de00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2080b1d10 0x7fa208149830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20de00640 1 -- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20815a600 msgr2=0x7fa208071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20de00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20815a600 0x7fa208071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20de00640 1 -- 172.21.15.97:0/1022425499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00094d0 con 0x7fa20815dc50 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20de00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 0x7fa2080af9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c7c0 tx=0x7fa20000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200004230 con 0x7fa20815dc50 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200004390 con 0x7fa20815dc50 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200007860 con 0x7fa20815dc50 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.912+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2080adb80 con 0x7fa20815dc50 2024-01-31T05:02:25.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.913+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa208070a00 con 0x7fa20815dc50 2024-01-31T05:02:25.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.914+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2080677c0 con 0x7fa20815dc50 2024-01-31T05:02:25.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.915+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa2000079c0 con 0x7fa20815dc50 2024-01-31T05:02:25.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.916+0000 7fa205600640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa1e006aed0 0x7fa1e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:25.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.916+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa200089690 con 0x7fa20815dc50 2024-01-31T05:02:25.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.916+0000 7fa20d400640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa1e006aed0 0x7fa1e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:25.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.917+0000 7fa20d400640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa1e006aed0 0x7fa1e006d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0036000 tx=0x7fa1f003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:25.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:25.920+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa200054830 con 0x7fa20815dc50 2024-01-31T05:02:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.182+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa20806e740 con 0x7fa20815dc50 2024-01-31T05:02:26.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.185+0000 7fa205600640 1 -- 172.21.15.97:0/1022425499 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa200057e50 con 0x7fa20815dc50 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa1e006aed0 msgr2=0x7fa1e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa1e006aed0 0x7fa1e006d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0036000 tx=0x7fa1f003c040 comp rx=0 tx=0).stop 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 msgr2=0x7fa2080af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 0x7fa2080af9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c7c0 tx=0x7fa20000cc80 comp rx=0 tx=0).stop 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 shutdown_connections 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa1e006aed0 0x7fa1e006d390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa20815dc50 0x7fa2080af9d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa20815a600 0x7fa208071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 --2- 172.21.15.97:0/1022425499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2080b1d10 0x7fa208149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 >> 172.21.15.97:0/1022425499 conn(0x7fa2080b0a10 msgr2=0x7fa2080ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:26.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 shutdown_connections 2024-01-31T05:02:26.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.188+0000 7fa20fe00640 1 -- 172.21.15.97:0/1022425499 wait complete. 2024-01-31T05:02:26.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-31T05:02:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.411+0000 7f3747000640 1 Processor -- start 2024-01-31T05:02:26.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.411+0000 7f3747000640 1 -- start start 2024-01-31T05:02:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.413+0000 7f3747000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37400cb810 0x7f37400cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.413+0000 7f3747000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 0x7f37400cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.413+0000 7f3747000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374006d1a0 con 0x7f374015b400 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.413+0000 7f3747000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37400cc960 con 0x7f37400cb810 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.413+0000 7f3747000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37400cd6f0 con 0x7f37400ccd00 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f3745000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 0x7f37400cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f373fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37400cb810 0x7f37400cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f3745000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 0x7f37400cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59082/0 (socket says 172.21.15.97:59082) 2024-01-31T05:02:26.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f3745000640 1 -- 172.21.15.97:0/2790366438 learned_addr learned my addr 172.21.15.97:0/2790366438 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f373fe00640 1 -- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 msgr2=0x7f37400cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f373fe00640 1 --2- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 0x7f37400cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f373fe00640 1 -- 172.21.15.97:0/2790366438 >> v1:172.21.15.188:6789/0 conn(0x7f374015b400 legacy=0x7f374006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f373fe00640 1 -- 172.21.15.97:0/2790366438 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374006c600 con 0x7f37400cb810 2024-01-31T05:02:26.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.414+0000 7f3745000640 1 --2- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 0x7f37400cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:02:26.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.416+0000 7f373fe00640 1 --2- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400cb810 0x7f37400cbbf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f37300097a0 tx=0x7f373002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2216711fd66077de server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.416+0000 7f373f400640 1 -- 172.21.15.97:0/2790366438 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373002d040 con 0x7f37400cb810 2024-01-31T05:02:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.416+0000 7f373f400640 1 -- 172.21.15.97:0/2790366438 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3730031810 con 0x7f37400cb810 2024-01-31T05:02:26.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.416+0000 7f373f400640 1 -- 172.21.15.97:0/2790366438 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730031ba0 con 0x7f37400cb810 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 -- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400cb810 msgr2=0x7f37400cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 --2- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400cb810 0x7f37400cbbf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f37300097a0 tx=0x7f373002b8b0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 -- 172.21.15.97:0/2790366438 shutdown_connections 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 --2- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400ccd00 0x7f37400cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 --2- 172.21.15.97:0/2790366438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400cb810 0x7f37400cbbf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 -- 172.21.15.97:0/2790366438 >> 172.21.15.97:0/2790366438 conn(0x7f37400b0a10 msgr2=0x7f37400b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 -- 172.21.15.97:0/2790366438 shutdown_connections 2024-01-31T05:02:26.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.417+0000 7f3747000640 1 -- 172.21.15.97:0/2790366438 wait complete. 2024-01-31T05:02:26.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.419+0000 7f3747000640 1 Processor -- start 2024-01-31T05:02:26.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.419+0000 7f3747000640 1 -- start start 2024-01-31T05:02:26.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3747000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400cb810 0x7f374014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3747000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400ccd00 0x7f374015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3747000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 0x7f374015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3747000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374015c720 con 0x7f374015b400 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3747000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374015ca00 con 0x7f37400ccd00 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3747000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374015cce0 con 0x7f37400cb810 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3745000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 0x7f374015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3745000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 0x7f374015dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36696/0 (socket says 172.21.15.97:36696) 2024-01-31T05:02:26.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3745000640 1 -- 172.21.15.97:0/3027578329 learned_addr learned my addr 172.21.15.97:0/3027578329 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:26.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3745000640 1 -- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400cb810 msgr2=0x7f374014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3745000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400cb810 0x7f374014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3745000640 1 -- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400ccd00 msgr2=0x7f374015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.420+0000 7f3737200640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400ccd00 0x7f374015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3745000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400ccd00 0x7f374015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3745000640 1 -- 172.21.15.97:0/3027578329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009b30 con 0x7f374015b400 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3737200640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400ccd00 0x7f374015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3745000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 0x7f374015dbc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f373800efc0 tx=0x7f373800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738018070 con 0x7f374015b400 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373800f040 con 0x7f374015b400 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3730009400 con 0x7f374015b400 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800ce00 con 0x7f374015b400 2024-01-31T05:02:26.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.421+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374015d2a0 con 0x7f374015b400 2024-01-31T05:02:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.422+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374015d460 con 0x7f374015b400 2024-01-31T05:02:26.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.423+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3738014480 con 0x7f374015b400 2024-01-31T05:02:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.424+0000 7f373cc00640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f371806aed0 0x7f371806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.424+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3738089770 con 0x7f374015b400 2024-01-31T05:02:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.424+0000 7f373fe00640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f371806aed0 0x7f371806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.425+0000 7f373fe00640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f371806aed0 0x7f371806d390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3730036000 tx=0x7f3730030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:26.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.428+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3738054840 con 0x7f374015b400 2024-01-31T05:02:26.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.688+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f3740071ed0 con 0x7f374015b400 2024-01-31T05:02:26.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.693+0000 7f373cc00640 1 -- 172.21.15.97:0/3027578329 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3738057e60 con 0x7f374015b400 2024-01-31T05:02:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.695+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f371806aed0 msgr2=0x7f371806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.695+0000 7f3747000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f371806aed0 0x7f371806d390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3730036000 tx=0x7f3730030000 comp rx=0 tx=0).stop 2024-01-31T05:02:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.695+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 msgr2=0x7f374015dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.695+0000 7f3747000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 0x7f374015dbc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f373800efc0 tx=0x7f373800c490 comp rx=0 tx=0).stop 2024-01-31T05:02:26.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 shutdown_connections 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f371806aed0 0x7f371806d390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374015b400 0x7f374015dbc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37400ccd00 0x7f374015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 --2- 172.21.15.97:0/3027578329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37400cb810 0x7f374014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 >> 172.21.15.97:0/3027578329 conn(0x7f37400b0a10 msgr2=0x7f3740071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 shutdown_connections 2024-01-31T05:02:26.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.696+0000 7f3747000640 1 -- 172.21.15.97:0/3027578329 wait complete. 2024-01-31T05:02:26.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-31T05:02:26.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.918+0000 7f4e1a400640 1 Processor -- start 2024-01-31T05:02:26.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.919+0000 7f4e1a400640 1 -- start start 2024-01-31T05:02:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e1a400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e140abcf0 0x7f4e140ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e1a400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1406cd20 con 0x7f4e1415b470 2024-01-31T05:02:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e1a400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140ce970 con 0x7f4e140abcf0 2024-01-31T05:02:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e1a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1415b850 con 0x7f4e140cbf20 2024-01-31T05:02:26.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e13400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4e1415b470 0x7f4e140cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60404/0 (socket says 172.21.15.97:60404) 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e13400640 1 -- 172.21.15.97:0/2894027505 learned_addr learned my addr 172.21.15.97:0/2894027505 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.920+0000 7f4e13e00640 1 --2- 172.21.15.97:0/2894027505 >> v2:172.21.15.97:3300/0 conn(0x7f4e140abcf0 0x7f4e140ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e12a00640 1 --2- 172.21.15.97:0/2894027505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e12000640 1 -- 172.21.15.97:0/2894027505 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733717432 0 0) 0x7f4e1406cd20 con 0x7f4e1415b470 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e12000640 1 -- 172.21.15.97:0/2894027505 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df4003520 con 0x7f4e1415b470 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e13e00640 1 -- 172.21.15.97:0/2894027505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 msgr2=0x7f4e140722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e13e00640 1 --2- 172.21.15.97:0/2894027505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e13e00640 1 -- 172.21.15.97:0/2894027505 >> v1:172.21.15.97:6789/0 conn(0x7f4e1415b470 legacy=0x7f4e140cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.921+0000 7f4e13e00640 1 -- 172.21.15.97:0/2894027505 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140ceb60 con 0x7f4e140abcf0 2024-01-31T05:02:26.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.922+0000 7f4e13e00640 1 --2- 172.21.15.97:0/2894027505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e140ce430 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009800 tx=0x7f4e0c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1c4e6c8651e907 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:26.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.922+0000 7f4e12000640 1 -- 172.21.15.97:0/2894027505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c031800 con 0x7f4e140abcf0 2024-01-31T05:02:26.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.922+0000 7f4e12000640 1 -- 172.21.15.97:0/2894027505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c031960 con 0x7f4e140abcf0 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.922+0000 7f4e12000640 1 -- 172.21.15.97:0/2894027505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c031cf0 con 0x7f4e140abcf0 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 -- 172.21.15.97:0/2894027505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 msgr2=0x7f4e140ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 --2- 172.21.15.97:0/2894027505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e140ce430 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009800 tx=0x7f4e0c02b8a0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 -- 172.21.15.97:0/2894027505 shutdown_connections 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 --2- 172.21.15.97:0/2894027505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e140ce430 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 --2- 172.21.15.97:0/2894027505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 -- 172.21.15.97:0/2894027505 >> 172.21.15.97:0/2894027505 conn(0x7f4e140b0a10 msgr2=0x7f4e140b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 -- 172.21.15.97:0/2894027505 shutdown_connections 2024-01-31T05:02:26.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.923+0000 7f4e1a400640 1 -- 172.21.15.97:0/2894027505 wait complete. 2024-01-31T05:02:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.924+0000 7f4e1a400640 1 Processor -- start 2024-01-31T05:02:26.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.925+0000 7f4e1a400640 1 -- start start 2024-01-31T05:02:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.925+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e14144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.925+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.925+0000 7f4e1a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415b470 0x7f4e1415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e1a400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14146140 con 0x7f4e140abcf0 2024-01-31T05:02:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e1a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14144eb0 con 0x7f4e1415b470 2024-01-31T05:02:26.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e1a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14145530 con 0x7f4e140cbf20 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e14144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e14144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36726/0 (socket says 172.21.15.97:36726) 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 -- 172.21.15.97:0/48819397 learned_addr learned my addr 172.21.15.97:0/48819397 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 -- 172.21.15.97:0/48819397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 msgr2=0x7f4e140999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 -- 172.21.15.97:0/48819397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415b470 msgr2=0x7f4e1415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415b470 0x7f4e1415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 -- 172.21.15.97:0/48819397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009400 con 0x7f4e140abcf0 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.926+0000 7f4e13400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e14144970 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400c400 tx=0x7f4e0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.927+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040074e0 con 0x7f4e140abcf0 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.927+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e040057a0 con 0x7f4e140abcf0 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.927+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04007660 con 0x7f4e140abcf0 2024-01-31T05:02:26.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.927+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e14145810 con 0x7f4e140abcf0 2024-01-31T05:02:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.927+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e14145dd0 con 0x7f4e140abcf0 2024-01-31T05:02:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.928+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e14067830 con 0x7f4e140abcf0 2024-01-31T05:02:26.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.929+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4e04005b50 con 0x7f4e140abcf0 2024-01-31T05:02:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.929+0000 7f4e0b400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4dec06af00 0x7f4dec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.930+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4e04088a10 con 0x7f4e140abcf0 2024-01-31T05:02:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.930+0000 7f4e12a00640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4dec06af00 0x7f4dec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:26.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.930+0000 7f4e12a00640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4dec06af00 0x7f4dec06d3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0070b0 tx=0x7f4dfc007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:26.934+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4e04053ae0 con 0x7f4e140abcf0 2024-01-31T05:02:27.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.197+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f4e140b0780 con 0x7f4e140abcf0 2024-01-31T05:02:27.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.202+0000 7f4e0b400640 1 -- 172.21.15.97:0/48819397 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4e04017020 con 0x7f4e140abcf0 2024-01-31T05:02:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4dec06af00 msgr2=0x7f4dec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4dec06af00 0x7f4dec06d3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0070b0 tx=0x7f4dfc007040 comp rx=0 tx=0).stop 2024-01-31T05:02:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 msgr2=0x7f4e14144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e14144970 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400c400 tx=0x7f4e0400c8c0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 shutdown_connections 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415b470 0x7f4e1415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cbf20 0x7f4e140999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4dec06af00 0x7f4dec06d3c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 --2- 172.21.15.97:0/48819397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140abcf0 0x7f4e14144970 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 >> 172.21.15.97:0/48819397 conn(0x7f4e140b0a10 msgr2=0x7f4e14159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:27.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 shutdown_connections 2024-01-31T05:02:27.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.205+0000 7f4e1a400640 1 -- 172.21.15.97:0/48819397 wait complete. 2024-01-31T05:02:27.223 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-31T05:02:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.424+0000 7fbf90000640 1 Processor -- start 2024-01-31T05:02:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.425+0000 7fbf90000640 1 -- start start 2024-01-31T05:02:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.426+0000 7fbf90000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf880cb810 0x7fbf880cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.426+0000 7fbf90000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf8815b400 0x7fbf8806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.426+0000 7fbf90000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbf8806db70 0x7fbf880ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.426+0000 7fbf90000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf880cc960 con 0x7fbf8815b400 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.426+0000 7fbf90000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8806d1a0 con 0x7fbf8806db70 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.426+0000 7fbf90000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8806c600 con 0x7fbf880cb810 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf8815b400 0x7fbf8806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf8815b400 0x7fbf8806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36732/0 (socket says 172.21.15.97:36732) 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 -- 172.21.15.97:0/1422080849 learned_addr learned my addr 172.21.15.97:0/1422080849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 -- 172.21.15.97:0/1422080849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf880cb810 msgr2=0x7fbf880cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/1422080849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf880cb810 0x7fbf880cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 -- 172.21.15.97:0/1422080849 >> v2:172.21.15.188:3300/0 conn(0x7fbf8806db70 msgr2=0x7fbf880ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/1422080849 >> v2:172.21.15.188:3300/0 conn(0x7fbf8806db70 0x7fbf880ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.427+0000 7fbf8cc00640 1 -- 172.21.15.97:0/1422080849 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880cc130 con 0x7fbf8815b400 2024-01-31T05:02:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.428+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/1422080849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8815b400 0x7fbf8806d630 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbf8806c1c0 tx=0x7fbf7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e604eae901bb2fe server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf87e00640 1 -- 172.21.15.97:0/1422080849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70031870 con 0x7fbf8815b400 2024-01-31T05:02:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf87e00640 1 -- 172.21.15.97:0/1422080849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf700319d0 con 0x7fbf8815b400 2024-01-31T05:02:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf87e00640 1 -- 172.21.15.97:0/1422080849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70031d60 con 0x7fbf8815b400 2024-01-31T05:02:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 -- 172.21.15.97:0/1422080849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8815b400 msgr2=0x7fbf8806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 --2- 172.21.15.97:0/1422080849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8815b400 0x7fbf8806d630 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbf8806c1c0 tx=0x7fbf7002b910 comp rx=0 tx=0).stop 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 -- 172.21.15.97:0/1422080849 shutdown_connections 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 --2- 172.21.15.97:0/1422080849 >> v2:172.21.15.188:3300/0 conn(0x7fbf8806db70 0x7fbf880ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 --2- 172.21.15.97:0/1422080849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8815b400 0x7fbf8806d630 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 --2- 172.21.15.97:0/1422080849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf880cb810 0x7fbf880cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 -- 172.21.15.97:0/1422080849 >> 172.21.15.97:0/1422080849 conn(0x7fbf880b0a10 msgr2=0x7fbf880b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.429+0000 7fbf90000640 1 -- 172.21.15.97:0/1422080849 shutdown_connections 2024-01-31T05:02:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.430+0000 7fbf90000640 1 -- 172.21.15.97:0/1422080849 wait complete. 2024-01-31T05:02:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.431+0000 7fbf90000640 1 Processor -- start 2024-01-31T05:02:27.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.431+0000 7fbf90000640 1 -- start start 2024-01-31T05:02:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.432+0000 7fbf90000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 0x7fbf880c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.432+0000 7fbf90000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf880cb810 0x7fbf88151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.432+0000 7fbf90000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b400 0x7fbf880c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.432+0000 7fbf90000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88151f80 con 0x7fbf8806db70 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.432+0000 7fbf90000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880ca9d0 con 0x7fbf880cb810 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.432+0000 7fbf90000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880cae60 con 0x7fbf8815b400 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 0x7fbf880c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 0x7fbf880c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36744/0 (socket says 172.21.15.97:36744) 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 -- 172.21.15.97:0/3880060873 learned_addr learned my addr 172.21.15.97:0/3880060873 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf880cb810 0x7fbf88151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8e000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b400 0x7fbf880c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 -- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b400 msgr2=0x7fbf880c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b400 0x7fbf880c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 -- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf880cb810 msgr2=0x7fbf88151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf880cb810 0x7fbf88151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 -- 172.21.15.97:0/3880060873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700094d0 con 0x7fbf8806db70 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf880cb810 0x7fbf88151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.433+0000 7fbf8d600640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 0x7fbf880c8e30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b4a0 tx=0x7fbf7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.434+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78010610 con 0x7fbf8806db70 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.434+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf78010770 con 0x7fbf8806db70 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.434+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78010b20 con 0x7fbf8806db70 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.434+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf880c9dc0 con 0x7fbf8806db70 2024-01-31T05:02:27.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.434+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf880ca380 con 0x7fbf8806db70 2024-01-31T05:02:27.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.435+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf880677c0 con 0x7fbf8806db70 2024-01-31T05:02:27.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.436+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fbf78010c80 con 0x7fbf8806db70 2024-01-31T05:02:27.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.437+0000 7fbf85600640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf6006aed0 0x7fbf6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.437+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbf78089070 con 0x7fbf8806db70 2024-01-31T05:02:27.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.437+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf6006aed0 0x7fbf6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.438+0000 7fbf8cc00640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf6006aed0 0x7fbf6006d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbf70036000 tx=0x7fbf7003c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:27.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.442+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fbf78054150 con 0x7fbf8806db70 2024-01-31T05:02:27.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.710+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fbf880adf00 con 0x7fbf8806db70 2024-01-31T05:02:27.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.714+0000 7fbf85600640 1 -- 172.21.15.97:0/3880060873 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbf78057770 con 0x7fbf8806db70 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf6006aed0 msgr2=0x7fbf6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf6006aed0 0x7fbf6006d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbf70036000 tx=0x7fbf7003c040 comp rx=0 tx=0).stop 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 msgr2=0x7fbf880c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 0x7fbf880c8e30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b4a0 tx=0x7fbf7800b960 comp rx=0 tx=0).stop 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 shutdown_connections 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf8815b400 0x7fbf880c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf880cb810 0x7fbf88151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf6006aed0 0x7fbf6006d390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 --2- 172.21.15.97:0/3880060873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf8806db70 0x7fbf880c8e30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 >> 172.21.15.97:0/3880060873 conn(0x7fbf880b0a10 msgr2=0x7fbf88072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 shutdown_connections 2024-01-31T05:02:27.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.717+0000 7fbf90000640 1 -- 172.21.15.97:0/3880060873 wait complete. 2024-01-31T05:02:27.734 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:27.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.939+0000 7feb86600640 1 Processor -- start 2024-01-31T05:02:27.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.940+0000 7feb86600640 1 -- start start 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb86600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb8015d590 0x7feb8015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb86600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb800cfae0 0x7feb800cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb86600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb800d0460 con 0x7feb80161080 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb86600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb800cded0 con 0x7feb8015d590 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb86600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8015d970 con 0x7feb800cfae0 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb7f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feb8015d590 0x7feb8015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.941+0000 7feb7fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feb80161080 0x7feb80161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60412/0 (socket says 172.21.15.97:60412) 2024-01-31T05:02:27.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.942+0000 7feb7fe00640 1 -- 172.21.15.97:0/3801864452 learned_addr learned my addr 172.21.15.97:0/3801864452 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:27.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.942+0000 7feb7ea00640 1 --2- 172.21.15.97:0/3801864452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb800cfae0 0x7feb800cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.942+0000 7feb7f400640 1 -- 172.21.15.97:0/3801864452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb800cfae0 msgr2=0x7feb800cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:27.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.942+0000 7feb7f400640 1 --2- 172.21.15.97:0/3801864452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb800cfae0 0x7feb800cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.942+0000 7feb7f400640 1 -- 172.21.15.97:0/3801864452 >> v1:172.21.15.97:6789/0 conn(0x7feb80161080 legacy=0x7feb80161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.942+0000 7feb7f400640 1 -- 172.21.15.97:0/3801864452 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb800cf210 con 0x7feb8015d590 2024-01-31T05:02:27.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.943+0000 7feb7f400640 1 --2- 172.21.15.97:0/3801864452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb8015d590 0x7feb8015e540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7feb700097a0 tx=0x7feb7002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cf51def2d420f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:27.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.943+0000 7feb7e000640 1 -- 172.21.15.97:0/3801864452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70030800 con 0x7feb8015d590 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.943+0000 7feb7e000640 1 -- 172.21.15.97:0/3801864452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70030960 con 0x7feb8015d590 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.943+0000 7feb7e000640 1 -- 172.21.15.97:0/3801864452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70030cf0 con 0x7feb8015d590 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 -- 172.21.15.97:0/3801864452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb8015d590 msgr2=0x7feb8015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 --2- 172.21.15.97:0/3801864452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb8015d590 0x7feb8015e540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7feb700097a0 tx=0x7feb7002b940 comp rx=0 tx=0).stop 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 -- 172.21.15.97:0/3801864452 shutdown_connections 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 --2- 172.21.15.97:0/3801864452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb800cfae0 0x7feb800cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 --2- 172.21.15.97:0/3801864452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb8015d590 0x7feb8015e540 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 -- 172.21.15.97:0/3801864452 >> 172.21.15.97:0/3801864452 conn(0x7feb8009c560 msgr2=0x7feb800d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 -- 172.21.15.97:0/3801864452 shutdown_connections 2024-01-31T05:02:27.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.944+0000 7feb86600640 1 -- 172.21.15.97:0/3801864452 wait complete. 2024-01-31T05:02:27.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.946+0000 7feb86600640 1 Processor -- start 2024-01-31T05:02:27.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.946+0000 7feb86600640 1 -- start start 2024-01-31T05:02:27.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb86600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 0x7feb800a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb86600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8015d590 0x7feb8014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb86600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb80161080 0x7feb8014e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb86600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb800c93e0 con 0x7feb800cfae0 2024-01-31T05:02:27.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb86600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb800c96c0 con 0x7feb80161080 2024-01-31T05:02:27.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb86600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8014c250 con 0x7feb8015d590 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb7f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 0x7feb800a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb7f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 0x7feb800a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36764/0 (socket says 172.21.15.97:36764) 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.947+0000 7feb7f400640 1 -- 172.21.15.97:0/1324630047 learned_addr learned my addr 172.21.15.97:0/1324630047 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb7f400640 1 -- 172.21.15.97:0/1324630047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8015d590 msgr2=0x7feb8014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb7f400640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8015d590 0x7feb8014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb7f400640 1 -- 172.21.15.97:0/1324630047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb80161080 msgr2=0x7feb8014e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb7f400640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb80161080 0x7feb8014e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb7f400640 1 -- 172.21.15.97:0/1324630047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009400 con 0x7feb800cfae0 2024-01-31T05:02:27.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb7f400640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 0x7feb800a0a30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7feb700098c0 tx=0x7feb70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:27.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7003e040 con 0x7feb800cfae0 2024-01-31T05:02:27.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb70039cb0 con 0x7feb800cfae0 2024-01-31T05:02:27.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70044630 con 0x7feb800cfae0 2024-01-31T05:02:27.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.948+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8014c530 con 0x7feb800cfae0 2024-01-31T05:02:27.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.949+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8015baf0 con 0x7feb800cfae0 2024-01-31T05:02:27.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.950+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8014ca50 con 0x7feb800cfae0 2024-01-31T05:02:27.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.950+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7feb70043390 con 0x7feb800cfae0 2024-01-31T05:02:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.952+0000 7feb77400640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feb5806aed0 0x7feb5806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.952+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7feb700ab7c0 con 0x7feb800cfae0 2024-01-31T05:02:27.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.952+0000 7feb7ea00640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feb5806aed0 0x7feb5806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:27.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.953+0000 7feb7ea00640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feb5806aed0 0x7feb5806d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb800b4450 tx=0x7feb68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:27.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:27.957+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7feb8014ca50 con 0x7feb800cfae0 2024-01-31T05:02:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.150+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb8014ca50 con 0x7feb800cfae0 2024-01-31T05:02:28.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.152+0000 7feb77400640 1 -- 172.21.15.97:0/1324630047 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13063 (secure 0 0 0) 0x7feb70034090 con 0x7feb800cfae0 2024-01-31T05:02:28.152 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:28.152 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:24.163771+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feb5806aed0 msgr2=0x7feb5806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feb5806aed0 0x7feb5806d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb800b4450 tx=0x7feb68008040 comp rx=0 tx=0).stop 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 msgr2=0x7feb800a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 0x7feb800a0a30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7feb700098c0 tx=0x7feb70004730 comp rx=0 tx=0).stop 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 shutdown_connections 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb80161080 0x7feb8014e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8015d590 0x7feb8014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feb5806aed0 0x7feb5806d390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 --2- 172.21.15.97:0/1324630047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb800cfae0 0x7feb800a0a30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 >> 172.21.15.97:0/1324630047 conn(0x7feb8009c560 msgr2=0x7feb800d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:28.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 shutdown_connections 2024-01-31T05:02:28.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.154+0000 7feb86600640 1 -- 172.21.15.97:0/1324630047 wait complete. 2024-01-31T05:02:28.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:02:28.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.387+0000 7f309f400640 1 Processor -- start 2024-01-31T05:02:28.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.388+0000 7f309f400640 1 -- start start 2024-01-31T05:02:28.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.389+0000 7f309f400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30980cfae0 0x7f30980cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.389+0000 7f309f400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309815ec50 con 0x7f30980d0de0 2024-01-31T05:02:28.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.389+0000 7f309f400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30980d0460 con 0x7f309815d590 2024-01-31T05:02:28.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.389+0000 7f309f400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30980cded0 con 0x7f30980cfae0 2024-01-31T05:02:28.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.389+0000 7f309d400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f30980d0de0 0x7f30980d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60426/0 (socket says 172.21.15.97:60426) 2024-01-31T05:02:28.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.389+0000 7f309d400640 1 -- 172.21.15.97:0/4042039515 learned_addr learned my addr 172.21.15.97:0/4042039515 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f308fc00640 1 --2- 172.21.15.97:0/4042039515 >> v2:172.21.15.188:3300/0 conn(0x7f30980cfae0 0x7f30980cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046182084 0 0) 0x7f309815ec50 con 0x7f30980d0de0 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3078003520 con 0x7f30980d0de0 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67990935 0 0) 0x7f3078003520 con 0x7f30980d0de0 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 >> v2:172.21.15.188:3300/0 conn(0x7f30980cfae0 msgr2=0x7f30980cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 --2- 172.21.15.97:0/4042039515 >> v2:172.21.15.188:3300/0 conn(0x7f30980cfae0 0x7f30980cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 >> v1:172.21.15.188:6789/0 conn(0x7f309815d590 legacy=0x7f309815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309815d970 con 0x7f30980d0de0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f3090001670 con 0x7f30980d0de0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 >> v1:172.21.15.97:6789/0 conn(0x7f30980d0de0 legacy=0x7f30980d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.390+0000 7f3097e00640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30780039c0 0x7f3078003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f3097e00640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30780047c0 0x7f3078004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f3097e00640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f307800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30980cded0 con 0x7f3078005180 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307800c060 con 0x7f30780047c0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307800c220 con 0x7f30780039c0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309d400640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f307800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f308fc00640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30780047c0 0x7f3078004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309ca00640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30780039c0 0x7f3078003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309d400640 1 -- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30780039c0 msgr2=0x7f3078003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309d400640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30780039c0 0x7f3078003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309d400640 1 -- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30780047c0 msgr2=0x7f3078004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309d400640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30780047c0 0x7f3078004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.391+0000 7f309d400640 1 -- 172.21.15.97:0/4042039515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078003520 con 0x7f3078005180 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.392+0000 7f309d400640 1 -- 172.21.15.97:0/4042039515 reap_dead start 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.392+0000 7f309d400640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f307800b980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f309000c0e0 tx=0x7f30780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cd581ec3740d5f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090005ad0 con 0x7f3078005180 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090005c50 con 0x7f3078005180 2024-01-31T05:02:28.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f3097e00640 1 -- 172.21.15.97:0/4042039515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309002c7d0 con 0x7f3078005180 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 -- 172.21.15.97:0/4042039515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 msgr2=0x7f307800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f307800b980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f309000c0e0 tx=0x7f30780057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 -- 172.21.15.97:0/4042039515 shutdown_connections 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 --2- 172.21.15.97:0/4042039515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f307800b980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 -- 172.21.15.97:0/4042039515 >> 172.21.15.97:0/4042039515 conn(0x7f309809c560 msgr2=0x7f30980d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 -- 172.21.15.97:0/4042039515 shutdown_connections 2024-01-31T05:02:28.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.393+0000 7f309f400640 1 -- 172.21.15.97:0/4042039515 wait complete. 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.395+0000 7f309f400640 1 Processor -- start 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.395+0000 7f309f400640 1 -- start start 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f30981506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309809cf40 0x7f309809fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30980a0240 0x7f30980c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3098150c30 con 0x7f3078005180 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309f400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30980a0700 con 0x7f309809cf40 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30980a09e0 con 0x7f30980a0240 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.396+0000 7f309ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f30981506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f30981506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36778/0 (socket says 172.21.15.97:36778) 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 -- 172.21.15.97:0/4272133050 learned_addr learned my addr 172.21.15.97:0/4272133050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:28.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309d400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30980a0240 0x7f30980c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f308fc00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309809cf40 0x7f309809fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 -- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30980a0240 msgr2=0x7f30980c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30980a0240 0x7f30980c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 -- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309809cf40 msgr2=0x7f309809fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309809cf40 0x7f309809fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 -- 172.21.15.97:0/4272133050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980a0cc0 con 0x7f3078005180 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309d400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30980a0240 0x7f30980c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f308fc00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309809cf40 0x7f309809fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f309ca00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f30981506f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f308800d570 tx=0x7f308800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.397+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088005480 con 0x7f3078005180 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.398+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088003610 con 0x7f3078005180 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.398+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088003980 con 0x7f3078005180 2024-01-31T05:02:28.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.398+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30980b24c0 con 0x7f3078005180 2024-01-31T05:02:28.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.398+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30980c8890 con 0x7f3078005180 2024-01-31T05:02:28.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.399+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30980b2320 con 0x7f3078005180 2024-01-31T05:02:28.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.400+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3088003ae0 con 0x7f3078005180 2024-01-31T05:02:28.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.401+0000 7f3095600640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307006aed0 0x7f307006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.401+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3088089410 con 0x7f3078005180 2024-01-31T05:02:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.401+0000 7f308fc00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307006aed0 0x7f307006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.402+0000 7f308fc00640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307006aed0 0x7f307006d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f309815e640 tx=0x7f308000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:28.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.405+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f30980b2320 con 0x7f3078005180 2024-01-31T05:02:28.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.597+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30980b2320 con 0x7f3078005180 2024-01-31T05:02:28.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.597+0000 7f3095600640 1 -- 172.21.15.97:0/4272133050 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30980b2320 con 0x7f3078005180 2024-01-31T05:02:28.597 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307006aed0 msgr2=0x7f307006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307006aed0 0x7f307006d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f309815e640 tx=0x7f308000b040 comp rx=0 tx=0).stop 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 msgr2=0x7f30981506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f30981506f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f308800d570 tx=0x7f308800da30 comp rx=0 tx=0).stop 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 shutdown_connections 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30980a0240 0x7f30980c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f309809cf40 0x7f309809fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f307006aed0 0x7f307006d390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 --2- 172.21.15.97:0/4272133050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3078005180 0x7f30981506f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.600+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 >> 172.21.15.97:0/4272133050 conn(0x7f309809c560 msgr2=0x7f30980cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:28.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.601+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 shutdown_connections 2024-01-31T05:02:28.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.601+0000 7f309f400640 1 -- 172.21.15.97:0/4272133050 wait complete. 2024-01-31T05:02:28.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-31T05:02:28.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.827+0000 7f27cb800640 1 Processor -- start 2024-01-31T05:02:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.827+0000 7f27cb800640 1 -- start start 2024-01-31T05:02:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27cb800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f27c40ccd00 0x7f27c40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27cb800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c406d1a0 con 0x7f27c40cb810 2024-01-31T05:02:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27cb800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c406c600 con 0x7f27c415b400 2024-01-31T05:02:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27cb800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c40cc960 con 0x7f27c40ccd00 2024-01-31T05:02:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27c8e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f27c40cb810 0x7f27c40cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60442/0 (socket says 172.21.15.97:60442) 2024-01-31T05:02:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27c8e00640 1 -- 172.21.15.97:0/1120987367 learned_addr learned my addr 172.21.15.97:0/1120987367 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:28.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27c9800640 1 --2- 172.21.15.97:0/1120987367 >> v2:172.21.15.97:3300/0 conn(0x7f27c40ccd00 0x7f27c40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.829+0000 7f27c3e00640 1 -- 172.21.15.97:0/1120987367 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015283311 0 0) 0x7f27c406d1a0 con 0x7f27c40cb810 2024-01-31T05:02:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.830+0000 7f27c3e00640 1 -- 172.21.15.97:0/1120987367 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a4003520 con 0x7f27c40cb810 2024-01-31T05:02:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.830+0000 7f27c9800640 1 -- 172.21.15.97:0/1120987367 >> v1:172.21.15.188:6789/0 conn(0x7f27c415b400 legacy=0x7f27c406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.830+0000 7f27c9800640 1 -- 172.21.15.97:0/1120987367 >> v1:172.21.15.97:6789/0 conn(0x7f27c40cb810 legacy=0x7f27c40cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.830+0000 7f27c9800640 1 -- 172.21.15.97:0/1120987367 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40cd6d0 con 0x7f27c40ccd00 2024-01-31T05:02:28.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.831+0000 7f27c9800640 1 --2- 172.21.15.97:0/1120987367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40ccd00 0x7f27c40cd190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009d90 tx=0x7f27bc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1112bafd851a18f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:28.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.831+0000 7f27c3e00640 1 -- 172.21.15.97:0/1120987367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc02cd00 con 0x7f27c40ccd00 2024-01-31T05:02:28.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.831+0000 7f27c3e00640 1 -- 172.21.15.97:0/1120987367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc02ce60 con 0x7f27c40ccd00 2024-01-31T05:02:28.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.831+0000 7f27c3e00640 1 -- 172.21.15.97:0/1120987367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc034660 con 0x7f27c40ccd00 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 -- 172.21.15.97:0/1120987367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40ccd00 msgr2=0x7f27c40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 --2- 172.21.15.97:0/1120987367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40ccd00 0x7f27c40cd190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009d90 tx=0x7f27bc02bf50 comp rx=0 tx=0).stop 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 -- 172.21.15.97:0/1120987367 shutdown_connections 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 --2- 172.21.15.97:0/1120987367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40ccd00 0x7f27c40cd190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 -- 172.21.15.97:0/1120987367 >> 172.21.15.97:0/1120987367 conn(0x7f27c40b0a10 msgr2=0x7f27c40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 -- 172.21.15.97:0/1120987367 shutdown_connections 2024-01-31T05:02:28.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.832+0000 7f27cb800640 1 -- 172.21.15.97:0/1120987367 wait complete. 2024-01-31T05:02:28.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.834+0000 7f27cb800640 1 Processor -- start 2024-01-31T05:02:28.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.834+0000 7f27cb800640 1 -- start start 2024-01-31T05:02:28.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.835+0000 7f27cb800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 0x7f27c4154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.835+0000 7f27cb800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27c40ccd00 0x7f27c40b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.835+0000 7f27cb800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27c415b400 0x7f27c41557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27cb800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b2850 con 0x7f27c40cb810 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27cb800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b1eb0 con 0x7f27c415b400 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27cb800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b2190 con 0x7f27c40ccd00 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 0x7f27c4154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 0x7f27c4154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36802/0 (socket says 172.21.15.97:36802) 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 -- 172.21.15.97:0/779233985 learned_addr learned my addr 172.21.15.97:0/779233985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 -- 172.21.15.97:0/779233985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27c40ccd00 msgr2=0x7f27c40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27c40ccd00 0x7f27c40b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 -- 172.21.15.97:0/779233985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27c415b400 msgr2=0x7f27c41557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:28.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27c415b400 0x7f27c41557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:28.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 -- 172.21.15.97:0/779233985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc009a70 con 0x7f27c40cb810 2024-01-31T05:02:28.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.836+0000 7f27c8e00640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 0x7f27c4154950 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f27b400c500 tx=0x7f27b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.837+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4005ce0 con 0x7f27c40cb810 2024-01-31T05:02:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.837+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b4005e40 con 0x7f27c40cb810 2024-01-31T05:02:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.837+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c40b2470 con 0x7f27c40cb810 2024-01-31T05:02:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.837+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4007330 con 0x7f27c40cb810 2024-01-31T05:02:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.837+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c4154e90 con 0x7f27c40cb810 2024-01-31T05:02:28.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.838+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c40677c0 con 0x7f27c40cb810 2024-01-31T05:02:28.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.840+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f27b4007490 con 0x7f27c40cb810 2024-01-31T05:02:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.840+0000 7f27c1600640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f279c06aed0 0x7f279c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.841+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f27b4088a40 con 0x7f27c40cb810 2024-01-31T05:02:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.841+0000 7f27bbe00640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f279c06aed0 0x7f279c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:28.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.841+0000 7f27bbe00640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f279c06aed0 0x7f279c06d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f27ac000f60 tx=0x7f27ac0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:28.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:28.846+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f27b408ae60 con 0x7f27c40cb810 2024-01-31T05:02:29.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.045+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f27c40abf40 con 0x7f27c40cb810 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.046+0000 7f27c1600640 1 -- 172.21.15.97:0/779233985 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7f27b4017020 con 0x7f27c40cb810 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:29.046 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-01-31T05:02:29.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f279c06aed0 msgr2=0x7f279c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:29.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f279c06aed0 0x7f279c06d390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f27ac000f60 tx=0x7f27ac0079b0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 msgr2=0x7f27c4154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:29.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 0x7f27c4154950 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f27b400c500 tx=0x7f27b400c9c0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 shutdown_connections 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27c415b400 0x7f27c41557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27c40ccd00 0x7f27c40b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f279c06aed0 0x7f279c06d390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 --2- 172.21.15.97:0/779233985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c40cb810 0x7f27c4154950 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.049+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 >> 172.21.15.97:0/779233985 conn(0x7f27c40b0a10 msgr2=0x7f27c40ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.050+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 shutdown_connections 2024-01-31T05:02:29.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.050+0000 7f27cb800640 1 -- 172.21.15.97:0/779233985 wait complete. 2024-01-31T05:02:29.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-31T05:02:29.066 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-31T05:02:29.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.281+0000 7f010ee00640 1 Processor -- start 2024-01-31T05:02:29.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.282+0000 7f010ee00640 1 -- start start 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f010ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f010815a370 0x7f010806d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f010ee00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0108071e80 0x7f01080caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f010ee00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01080cdac0 con 0x7f01080722e0 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f010ee00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01080cdd10 con 0x7f0108071e80 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f010ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01080cdff0 con 0x7f010815a370 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f0107400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0108071e80 0x7f01080caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f0107400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0108071e80 0x7f01080caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36814/0 (socket says 172.21.15.97:36814) 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f0107400640 1 -- 172.21.15.97:0/194350857 learned_addr learned my addr 172.21.15.97:0/194350857 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:29.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.283+0000 7f0107e00640 1 --2- 172.21.15.97:0/194350857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f010815a370 0x7f010806d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:29.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.284+0000 7f0107400640 1 -- 172.21.15.97:0/194350857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f010815a370 msgr2=0x7f010806d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:29.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.284+0000 7f0107400640 1 --2- 172.21.15.97:0/194350857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f010815a370 0x7f010806d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.284+0000 7f0107400640 1 -- 172.21.15.97:0/194350857 >> v1:172.21.15.188:6789/0 conn(0x7f01080722e0 legacy=0x7f01080cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:29.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.284+0000 7f0107400640 1 -- 172.21.15.97:0/194350857 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01080ce2d0 con 0x7f0108071e80 2024-01-31T05:02:29.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.285+0000 7f0107400640 1 --2- 172.21.15.97:0/194350857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0108071e80 0x7f01080caf50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009870 tx=0x7f00f002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a3d3b3d4b26416 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:29.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.285+0000 7f0106a00640 1 -- 172.21.15.97:0/194350857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00317c0 con 0x7f0108071e80 2024-01-31T05:02:29.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.285+0000 7f0106a00640 1 -- 172.21.15.97:0/194350857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f0031920 con 0x7f0108071e80 2024-01-31T05:02:29.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.285+0000 7f0106a00640 1 -- 172.21.15.97:0/194350857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0031cb0 con 0x7f0108071e80 2024-01-31T05:02:29.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 -- 172.21.15.97:0/194350857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0108071e80 msgr2=0x7f01080caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 --2- 172.21.15.97:0/194350857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0108071e80 0x7f01080caf50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009870 tx=0x7f00f002b860 comp rx=0 tx=0).stop 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 -- 172.21.15.97:0/194350857 shutdown_connections 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 --2- 172.21.15.97:0/194350857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0108071e80 0x7f01080caf50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 --2- 172.21.15.97:0/194350857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f010815a370 0x7f010806d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 -- 172.21.15.97:0/194350857 >> 172.21.15.97:0/194350857 conn(0x7f01080cfef0 msgr2=0x7f01080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 -- 172.21.15.97:0/194350857 shutdown_connections 2024-01-31T05:02:29.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.286+0000 7f010ee00640 1 -- 172.21.15.97:0/194350857 wait complete. 2024-01-31T05:02:29.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.288+0000 7f010ee00640 1 Processor -- start 2024-01-31T05:02:29.287 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.288+0000 7f010ee00640 1 -- start start 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0108071e80 0x7f0108156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ee00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 0x7f010809e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f010815a370 0x7f010809e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ee00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0108156540 con 0x7f01080722e0 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01080b3d40 con 0x7f010815a370 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01080b3fe0 con 0x7f0108071e80 2024-01-31T05:02:29.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 0x7f010809e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 0x7f010809e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36830/0 (socket says 172.21.15.97:36830) 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 -- 172.21.15.97:0/358115474 learned_addr learned my addr 172.21.15.97:0/358115474 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 -- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0108071e80 msgr2=0x7f0108156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0108071e80 0x7f0108156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 -- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f010815a370 msgr2=0x7f010809e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f010ce00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f010815a370 0x7f010809e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.289+0000 7f0107400640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f010815a370 0x7f010809e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f0107400640 1 -- 172.21.15.97:0/358115474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00094d0 con 0x7f01080722e0 2024-01-31T05:02:29.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f0107400640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 0x7f010809e290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00f002b8a0 tx=0x7f00f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0032860 con 0x7f01080722e0 2024-01-31T05:02:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f00329e0 con 0x7f01080722e0 2024-01-31T05:02:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01080b42c0 con 0x7f01080722e0 2024-01-31T05:02:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0032d90 con 0x7f01080722e0 2024-01-31T05:02:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.290+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01080b4880 con 0x7f01080722e0 2024-01-31T05:02:29.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.291+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0108067830 con 0x7f01080722e0 2024-01-31T05:02:29.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.292+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f00f004e450 con 0x7f01080722e0 2024-01-31T05:02:29.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.294+0000 7f00ffe00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00e006af00 0x7f00e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:29.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.294+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f00f00ab630 con 0x7f01080722e0 2024-01-31T05:02:29.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.294+0000 7f0107e00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00e006af00 0x7f00e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:29.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.294+0000 7f0107e00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00e006af00 0x7f00e006d3c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f00f80070b0 tx=0x7f00f8007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:29.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.298+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f00f00768c0 con 0x7f01080722e0 2024-01-31T05:02:29.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:29.494+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f01080b5e70 con 0x7f01080722e0 2024-01-31T05:02:30.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.219+0000 7f00ffe00640 1 -- 172.21.15.97:0/358115474 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f00f0079ee0 con 0x7f01080722e0 2024-01-31T05:02:30.219 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-01-31T05:02:30.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00e006af00 msgr2=0x7f00e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00e006af00 0x7f00e006d3c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f00f80070b0 tx=0x7f00f8007040 comp rx=0 tx=0).stop 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 msgr2=0x7f010809e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 0x7f010809e290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f00f002b8a0 tx=0x7f00f00046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 shutdown_connections 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f010815a370 0x7f010809e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f00e006af00 0x7f00e006d3c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01080722e0 0x7f010809e290 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 --2- 172.21.15.97:0/358115474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0108071e80 0x7f0108156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.225+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 >> 172.21.15.97:0/358115474 conn(0x7f01080cfef0 msgr2=0x7f010806a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.226+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 shutdown_connections 2024-01-31T05:02:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.226+0000 7f010ee00640 1 -- 172.21.15.97:0/358115474 wait complete. 2024-01-31T05:02:30.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-31T05:02:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.462+0000 7f72bc000640 1 Processor -- start 2024-01-31T05:02:30.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.462+0000 7f72bc000640 1 -- start start 2024-01-31T05:02:30.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.464+0000 7f72bc000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40cfae0 0x7f72b40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.464+0000 7f72bc000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72b40d0de0 0x7f72b40d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.464+0000 7f72bc000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b415ec50 con 0x7f72b415d590 2024-01-31T05:02:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.464+0000 7f72bc000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b40d0460 con 0x7f72b40d0de0 2024-01-31T05:02:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.464+0000 7f72bc000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40cded0 con 0x7f72b40cfae0 2024-01-31T05:02:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.465+0000 7f72b9600640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f72b415d590 0x7f72b415e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:42692/0 (socket says 172.21.15.97:42692) 2024-01-31T05:02:30.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.465+0000 7f72b9600640 1 -- 172.21.15.97:0/4025492435 learned_addr learned my addr 172.21.15.97:0/4025492435 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.465+0000 7f72b8c00640 1 --2- 172.21.15.97:0/4025492435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40cfae0 0x7f72b40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.465+0000 7f72ba000640 1 --2- 172.21.15.97:0/4025492435 >> v2:172.21.15.188:3300/0 conn(0x7f72b40d0de0 0x7f72b40d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.466+0000 7f72b3e00640 1 -- 172.21.15.97:0/4025492435 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499425086 0 0) 0x7f72b415ec50 con 0x7f72b415d590 2024-01-31T05:02:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.466+0000 7f72b3e00640 1 -- 172.21.15.97:0/4025492435 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7294003520 con 0x7f72b415d590 2024-01-31T05:02:30.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.466+0000 7f72b8c00640 1 -- 172.21.15.97:0/4025492435 >> v2:172.21.15.188:3300/0 conn(0x7f72b40d0de0 msgr2=0x7f72b40d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.466+0000 7f72b8c00640 1 --2- 172.21.15.97:0/4025492435 >> v2:172.21.15.188:3300/0 conn(0x7f72b40d0de0 0x7f72b40d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.466+0000 7f72b8c00640 1 -- 172.21.15.97:0/4025492435 >> v1:172.21.15.188:6789/0 conn(0x7f72b415d590 legacy=0x7f72b415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.466+0000 7f72b8c00640 1 -- 172.21.15.97:0/4025492435 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b415d970 con 0x7f72b40cfae0 2024-01-31T05:02:30.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.467+0000 7f72b8c00640 1 --2- 172.21.15.97:0/4025492435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40cfae0 0x7f72b40cff20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009d90 tx=0x7f72a402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d99d36d8602ec server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.468+0000 7f72b3e00640 1 -- 172.21.15.97:0/4025492435 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a402cce0 con 0x7f72b40cfae0 2024-01-31T05:02:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.468+0000 7f72b3e00640 1 -- 172.21.15.97:0/4025492435 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a402ce40 con 0x7f72b40cfae0 2024-01-31T05:02:30.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.468+0000 7f72b3e00640 1 -- 172.21.15.97:0/4025492435 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4034660 con 0x7f72b40cfae0 2024-01-31T05:02:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.468+0000 7f72bc000640 1 -- 172.21.15.97:0/4025492435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40cfae0 msgr2=0x7f72b40cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:30.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.468+0000 7f72bc000640 1 --2- 172.21.15.97:0/4025492435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40cfae0 0x7f72b40cff20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009d90 tx=0x7f72a402bf30 comp rx=0 tx=0).stop 2024-01-31T05:02:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.469+0000 7f72bc000640 1 -- 172.21.15.97:0/4025492435 shutdown_connections 2024-01-31T05:02:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.469+0000 7f72bc000640 1 --2- 172.21.15.97:0/4025492435 >> v2:172.21.15.188:3300/0 conn(0x7f72b40d0de0 0x7f72b40d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.469+0000 7f72bc000640 1 --2- 172.21.15.97:0/4025492435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40cfae0 0x7f72b40cff20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.469+0000 7f72bc000640 1 -- 172.21.15.97:0/4025492435 >> 172.21.15.97:0/4025492435 conn(0x7f72b409c560 msgr2=0x7f72b40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.469+0000 7f72bc000640 1 -- 172.21.15.97:0/4025492435 shutdown_connections 2024-01-31T05:02:30.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.469+0000 7f72bc000640 1 -- 172.21.15.97:0/4025492435 wait complete. 2024-01-31T05:02:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.470+0000 7f72bc000640 1 Processor -- start 2024-01-31T05:02:30.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.470+0000 7f72bc000640 1 -- start start 2024-01-31T05:02:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.471+0000 7f72bc000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 0x7f72b41f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.471+0000 7f72bc000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40d0de0 0x7f72b40b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.471+0000 7f72bc000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b415d590 0x7f72b415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:30.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.471+0000 7f72bc000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40b83c0 con 0x7f72b40cfae0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.471+0000 7f72bc000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b41f63c0 con 0x7f72b415d590 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.471+0000 7f72bc000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40b6b40 con 0x7f72b40d0de0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 0x7f72b41f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 0x7f72b41f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36838/0 (socket says 172.21.15.97:36838) 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 -- 172.21.15.97:0/2393063103 learned_addr learned my addr 172.21.15.97:0/2393063103 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72ba000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b415d590 0x7f72b415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 -- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40d0de0 msgr2=0x7f72b40b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b8c00640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40d0de0 0x7f72b40b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40d0de0 0x7f72b40b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 -- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b415d590 msgr2=0x7f72b415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b415d590 0x7f72b415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.472+0000 7f72b9600640 1 -- 172.21.15.97:0/2393063103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009a70 con 0x7f72b40cfae0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.473+0000 7f72b9600640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 0x7f72b41f6e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c390 tx=0x7f729c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:30.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.473+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c005ce0 con 0x7f72b40cfae0 2024-01-31T05:02:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.473+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c005e40 con 0x7f72b40cfae0 2024-01-31T05:02:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.473+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c0072f0 con 0x7f72b40cfae0 2024-01-31T05:02:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.473+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b40b6e20 con 0x7f72b40cfae0 2024-01-31T05:02:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.473+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b41f6610 con 0x7f72b40cfae0 2024-01-31T05:02:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.474+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b4067830 con 0x7f72b40cfae0 2024-01-31T05:02:30.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.475+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f729c007450 con 0x7f72b40cfae0 2024-01-31T05:02:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.476+0000 7f72b1600640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f728c06aed0 0x7f728c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.476+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f729c088c10 con 0x7f72b40cfae0 2024-01-31T05:02:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.476+0000 7f72b8c00640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f728c06aed0 0x7f728c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:30.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.477+0000 7f72b8c00640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f728c06aed0 0x7f728c06d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f72a4036000 tx=0x7f72a4030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:30.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.481+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f729c053b60 con 0x7f72b40cfae0 2024-01-31T05:02:30.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:30.674+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f72b4073600 con 0x7f72b40cfae0 2024-01-31T05:02:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.226+0000 7f72b1600640 1 -- 172.21.15.97:0/2393063103 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f729c017020 con 0x7f72b40cfae0 2024-01-31T05:02:31.226 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-01-31T05:02:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f728c06aed0 msgr2=0x7f728c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f728c06aed0 0x7f728c06d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f72a4036000 tx=0x7f72a4030000 comp rx=0 tx=0).stop 2024-01-31T05:02:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 msgr2=0x7f72b41f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:31.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 0x7f72b41f6e10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c390 tx=0x7f729c00c850 comp rx=0 tx=0).stop 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 shutdown_connections 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b415d590 0x7f72b415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b40d0de0 0x7f72b40b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f728c06aed0 0x7f728c06d390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 --2- 172.21.15.97:0/2393063103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72b40cfae0 0x7f72b41f6e10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.231+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 >> 172.21.15.97:0/2393063103 conn(0x7f72b409c560 msgr2=0x7f72b40b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.232+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 shutdown_connections 2024-01-31T05:02:31.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.232+0000 7f72bc000640 1 -- 172.21.15.97:0/2393063103 wait complete. 2024-01-31T05:02:31.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-31T05:02:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.452+0000 7fd0efa00640 1 Processor -- start 2024-01-31T05:02:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.452+0000 7fd0efa00640 1 -- start start 2024-01-31T05:02:31.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0efa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e80722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0efa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0e80abcf0 0x7fd0e80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0efa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0e80cb810 0x7fd0e80cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0efa00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80ce970 con 0x7fd0e80cb810 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0efa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80cc1e0 con 0x7fd0e80abcf0 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0efa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e815b850 con 0x7fd0e815b470 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0eda00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0e80cb810 0x7fd0e80cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0eda00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0e80cb810 0x7fd0e80cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36848/0 (socket says 172.21.15.97:36848) 2024-01-31T05:02:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0eda00640 1 -- 172.21.15.97:0/525466127 learned_addr learned my addr 172.21.15.97:0/525466127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0ed000640 1 --2- 172.21.15.97:0/525466127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e80722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.454+0000 7fd0dfc00640 1 --2- 172.21.15.97:0/525466127 >> v2:172.21.15.188:3300/0 conn(0x7fd0e80abcf0 0x7fd0e80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.455+0000 7fd0eda00640 1 -- 172.21.15.97:0/525466127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 msgr2=0x7fd0e80722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.455+0000 7fd0eda00640 1 --2- 172.21.15.97:0/525466127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e80722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.455+0000 7fd0eda00640 1 -- 172.21.15.97:0/525466127 >> v2:172.21.15.188:3300/0 conn(0x7fd0e80abcf0 msgr2=0x7fd0e80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.455+0000 7fd0eda00640 1 --2- 172.21.15.97:0/525466127 >> v2:172.21.15.188:3300/0 conn(0x7fd0e80abcf0 0x7fd0e80ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.455+0000 7fd0eda00640 1 -- 172.21.15.97:0/525466127 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e806cd20 con 0x7fd0e80cb810 2024-01-31T05:02:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.456+0000 7fd0eda00640 1 --2- 172.21.15.97:0/525466127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e80cbca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8072e90 tx=0x7fd0e002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df0b6b2d515180 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.456+0000 7fd0e7e00640 1 -- 172.21.15.97:0/525466127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e002ca50 con 0x7fd0e80cb810 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.456+0000 7fd0e7e00640 1 -- 172.21.15.97:0/525466127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e002cbb0 con 0x7fd0e80cb810 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.456+0000 7fd0e7e00640 1 -- 172.21.15.97:0/525466127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00343f0 con 0x7fd0e80cb810 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 -- 172.21.15.97:0/525466127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 msgr2=0x7fd0e80cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 --2- 172.21.15.97:0/525466127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e80cbca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8072e90 tx=0x7fd0e002beb0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 -- 172.21.15.97:0/525466127 shutdown_connections 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 --2- 172.21.15.97:0/525466127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e80cbca0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 --2- 172.21.15.97:0/525466127 >> v2:172.21.15.188:3300/0 conn(0x7fd0e80abcf0 0x7fd0e80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 --2- 172.21.15.97:0/525466127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e80722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 -- 172.21.15.97:0/525466127 >> 172.21.15.97:0/525466127 conn(0x7fd0e80b0a10 msgr2=0x7fd0e80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 -- 172.21.15.97:0/525466127 shutdown_connections 2024-01-31T05:02:31.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.457+0000 7fd0efa00640 1 -- 172.21.15.97:0/525466127 wait complete. 2024-01-31T05:02:31.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.458+0000 7fd0efa00640 1 Processor -- start 2024-01-31T05:02:31.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.459+0000 7fd0efa00640 1 -- start start 2024-01-31T05:02:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0efa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e80abcf0 0x7fd0e815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0efa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e8099ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0efa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e8152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0efa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e809a0e0 con 0x7fd0e80cb810 2024-01-31T05:02:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0efa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8151460 con 0x7fd0e80abcf0 2024-01-31T05:02:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0efa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8151740 con 0x7fd0e815b470 2024-01-31T05:02:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e8099ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e8099ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36860/0 (socket says 172.21.15.97:36860) 2024-01-31T05:02:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 -- 172.21.15.97:0/1943846362 learned_addr learned my addr 172.21.15.97:0/1943846362 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:31.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 -- 172.21.15.97:0/1943846362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 msgr2=0x7fd0e8152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e8152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 -- 172.21.15.97:0/1943846362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e80abcf0 msgr2=0x7fd0e815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e80abcf0 0x7fd0e815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 -- 172.21.15.97:0/1943846362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0009b30 con 0x7fd0e80cb810 2024-01-31T05:02:31.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.460+0000 7fd0dfc00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e8099ba0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000d7d0 tx=0x7fd0d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.461+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00096a0 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.461+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d0009800 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.461+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0009b90 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.461+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e8151a20 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.461+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e8151fe0 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.462+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e81521a0 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.463+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd0d0009cf0 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.464+0000 7fd0e5600640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0c006aed0 0x7fd0c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.464+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fd0d0089ca0 con 0x7fd0e80cb810 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.465+0000 7fd0ed000640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0c006aed0 0x7fd0c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:31.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.465+0000 7fd0ed000640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0c006aed0 0x7fd0c006d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8006e20 tx=0x7fd0d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:31.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.469+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd0d00549f0 con 0x7fd0e80cb810 2024-01-31T05:02:31.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:31.696+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd0e8067830 con 0x7fd0e80cb810 2024-01-31T05:02:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.266+0000 7fd0e5600640 1 -- 172.21.15.97:0/1943846362 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd0d0058010 con 0x7fd0e80cb810 2024-01-31T05:02:32.266 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 5 and data pool 6 2024-01-31T05:02:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0c006aed0 msgr2=0x7fd0c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0c006aed0 0x7fd0c006d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8006e20 tx=0x7fd0d8008040 comp rx=0 tx=0).stop 2024-01-31T05:02:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 msgr2=0x7fd0e8099ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e8099ba0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000d7d0 tx=0x7fd0d000dc90 comp rx=0 tx=0).stop 2024-01-31T05:02:32.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 shutdown_connections 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0e815b470 0x7fd0e8152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0c006aed0 0x7fd0c006d390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e80cb810 0x7fd0e8099ba0 secure :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000d7d0 tx=0x7fd0d000dc90 comp rx=0 tx=0).stop 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 --2- 172.21.15.97:0/1943846362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0e80abcf0 0x7fd0e815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.271+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 >> 172.21.15.97:0/1943846362 conn(0x7fd0e80b0a10 msgr2=0x7fd0e8158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.272+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 shutdown_connections 2024-01-31T05:02:32.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.272+0000 7fd0efa00640 1 -- 172.21.15.97:0/1943846362 wait complete. 2024-01-31T05:02:32.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.494+0000 7f11e0000640 1 Processor -- start 2024-01-31T05:02:32.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.494+0000 7f11e0000640 1 -- start start 2024-01-31T05:02:32.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11e0000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11d815b400 0x7f11d806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11e0000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11d806db70 0x7f11d80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11e0000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d806d1a0 con 0x7f11d80cb810 2024-01-31T05:02:32.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11e0000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d80cc960 con 0x7f11d815b400 2024-01-31T05:02:32.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11e0000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d806c600 con 0x7f11d806db70 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11dcc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11d815b400 0x7f11d806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11dcc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11d815b400 0x7f11d806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36870/0 (socket says 172.21.15.97:36870) 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11dcc00640 1 -- 172.21.15.97:0/3360433240 learned_addr learned my addr 172.21.15.97:0/3360433240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11dcc00640 1 -- 172.21.15.97:0/3360433240 >> v2:172.21.15.188:3300/0 conn(0x7f11d806db70 msgr2=0x7f11d80ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11de000640 1 --2- 172.21.15.97:0/3360433240 >> v2:172.21.15.188:3300/0 conn(0x7f11d806db70 0x7f11d80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11dcc00640 1 --2- 172.21.15.97:0/3360433240 >> v2:172.21.15.188:3300/0 conn(0x7f11d806db70 0x7f11d80ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.496+0000 7f11dcc00640 1 -- 172.21.15.97:0/3360433240 >> v1:172.21.15.188:6789/0 conn(0x7f11d80cb810 legacy=0x7f11d80cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.497+0000 7f11dcc00640 1 -- 172.21.15.97:0/3360433240 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d806c870 con 0x7f11d815b400 2024-01-31T05:02:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.497+0000 7f11de000640 1 --2- 172.21.15.97:0/3360433240 >> v2:172.21.15.188:3300/0 conn(0x7f11d806db70 0x7f11d80ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:32.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.498+0000 7f11dcc00640 1 --2- 172.21.15.97:0/3360433240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d806d630 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f11d80ae210 tx=0x7f11c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8699a13d86d1a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.498+0000 7f11d7e00640 1 -- 172.21.15.97:0/3360433240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031870 con 0x7f11d815b400 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.498+0000 7f11d7e00640 1 -- 172.21.15.97:0/3360433240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c00319d0 con 0x7f11d815b400 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.498+0000 7f11d7e00640 1 -- 172.21.15.97:0/3360433240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031d60 con 0x7f11d815b400 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 -- 172.21.15.97:0/3360433240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 msgr2=0x7f11d806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 --2- 172.21.15.97:0/3360433240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d806d630 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f11d80ae210 tx=0x7f11c002b910 comp rx=0 tx=0).stop 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 -- 172.21.15.97:0/3360433240 shutdown_connections 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 --2- 172.21.15.97:0/3360433240 >> v2:172.21.15.188:3300/0 conn(0x7f11d806db70 0x7f11d80ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 --2- 172.21.15.97:0/3360433240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d806d630 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 -- 172.21.15.97:0/3360433240 >> 172.21.15.97:0/3360433240 conn(0x7f11d80b0a10 msgr2=0x7f11d80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 -- 172.21.15.97:0/3360433240 shutdown_connections 2024-01-31T05:02:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.499+0000 7f11e0000640 1 -- 172.21.15.97:0/3360433240 wait complete. 2024-01-31T05:02:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.500+0000 7f11e0000640 1 Processor -- start 2024-01-31T05:02:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 -- start start 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d806db70 0x7f11d80cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d80cb810 0x7f11d815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d815d640 con 0x7f11d815b400 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d815de30 con 0x7f11d806db70 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.501+0000 7f11e0000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d815d870 con 0x7f11d80cb810 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d80b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36878/0 (socket says 172.21.15.97:36878) 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 -- 172.21.15.97:0/2843448116 learned_addr learned my addr 172.21.15.97:0/2843448116 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 -- 172.21.15.97:0/2843448116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d80cb810 msgr2=0x7f11d815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d80cb810 0x7f11d815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 -- 172.21.15.97:0/2843448116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d806db70 msgr2=0x7f11d80cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d806db70 0x7f11d80cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 -- 172.21.15.97:0/2843448116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00094d0 con 0x7f11d815b400 2024-01-31T05:02:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.502+0000 7f11de000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d80b3820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f11d000ce80 tx=0x7f11d000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:32.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.503+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d000d840 con 0x7f11d815b400 2024-01-31T05:02:32.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.503+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d000d9e0 con 0x7f11d815b400 2024-01-31T05:02:32.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.503+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d000dd90 con 0x7f11d815b400 2024-01-31T05:02:32.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.503+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d815db50 con 0x7f11d815b400 2024-01-31T05:02:32.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.503+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d81f3420 con 0x7f11d815b400 2024-01-31T05:02:32.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.504+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d81f3610 con 0x7f11d815b400 2024-01-31T05:02:32.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.505+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f11d0015440 con 0x7f11d815b400 2024-01-31T05:02:32.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.506+0000 7f11d5600640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f11b006aed0 0x7f11b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.506+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f11d008a150 con 0x7f11d815b400 2024-01-31T05:02:32.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.506+0000 7f11dd600640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f11b006aed0 0x7f11b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.507+0000 7f11dd600640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f11b006aed0 0x7f11b006d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11c8001560 tx=0x7f11c8007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:32.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.511+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f11d0054ef0 con 0x7f11d815b400 2024-01-31T05:02:32.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.708+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11d8067fa0 con 0x7f11d815b400 2024-01-31T05:02:32.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.711+0000 7f11d5600640 1 -- 172.21.15.97:0/2843448116 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16788 (secure 0 0 0) 0x7f11d0058510 con 0x7f11d815b400 2024-01-31T05:02:32.711 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:32.711 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:31.697720+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f11b006aed0 msgr2=0x7f11b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f11b006aed0 0x7f11b006d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f11c8001560 tx=0x7f11c8007990 comp rx=0 tx=0).stop 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 msgr2=0x7f11d80b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d80b3820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f11d000ce80 tx=0x7f11d000a710 comp rx=0 tx=0).stop 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 shutdown_connections 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f11b006aed0 0x7f11b006d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11d815b400 0x7f11d80b3820 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d80cb810 0x7f11d815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 --2- 172.21.15.97:0/2843448116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d806db70 0x7f11d80cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.713+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 >> 172.21.15.97:0/2843448116 conn(0x7f11d80b0a10 msgr2=0x7f11d80ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.714+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 shutdown_connections 2024-01-31T05:02:32.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.714+0000 7f11e0000640 1 -- 172.21.15.97:0/2843448116 wait complete. 2024-01-31T05:02:32.731 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:32.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.941+0000 7faf3ea00640 1 Processor -- start 2024-01-31T05:02:32.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.941+0000 7faf3ea00640 1 -- start start 2024-01-31T05:02:32.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.942+0000 7faf3ea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf3815a600 0x7faf3815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.942+0000 7faf3ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf3806d090 0x7faf3806d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.942+0000 7faf3ea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3806c8b0 con 0x7faf380b1d10 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.942+0000 7faf3ea00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3815b120 con 0x7faf3815a600 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.942+0000 7faf3ea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3815b400 con 0x7faf3806d090 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf3815a600 0x7faf3815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf3815a600 0x7faf3815aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36886/0 (socket says 172.21.15.97:36886) 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 -- 172.21.15.97:0/2622851888 learned_addr learned my addr 172.21.15.97:0/2622851888 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf3ca00640 1 --2- 172.21.15.97:0/2622851888 >> v2:172.21.15.188:3300/0 conn(0x7faf3806d090 0x7faf3806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 -- 172.21.15.97:0/2622851888 >> v2:172.21.15.188:3300/0 conn(0x7faf3806d090 msgr2=0x7faf3806d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 --2- 172.21.15.97:0/2622851888 >> v2:172.21.15.188:3300/0 conn(0x7faf3806d090 0x7faf3806d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 -- 172.21.15.97:0/2622851888 >> v1:172.21.15.188:6789/0 conn(0x7faf380b1d10 legacy=0x7faf380b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:32.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.943+0000 7faf37400640 1 -- 172.21.15.97:0/2622851888 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3806cb70 con 0x7faf3815a600 2024-01-31T05:02:32.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.944+0000 7faf37400640 1 --2- 172.21.15.97:0/2622851888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf3815a600 0x7faf3815aa60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf380b09a0 tx=0x7faf2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3520d38f27c00e21 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:32.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.944+0000 7faf36a00640 1 -- 172.21.15.97:0/2622851888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20031870 con 0x7faf3815a600 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.944+0000 7faf36a00640 1 -- 172.21.15.97:0/2622851888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf200319d0 con 0x7faf3815a600 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.944+0000 7faf36a00640 1 -- 172.21.15.97:0/2622851888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20031d60 con 0x7faf3815a600 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 -- 172.21.15.97:0/2622851888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf3815a600 msgr2=0x7faf3815aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 --2- 172.21.15.97:0/2622851888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf3815a600 0x7faf3815aa60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf380b09a0 tx=0x7faf2002b910 comp rx=0 tx=0).stop 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 -- 172.21.15.97:0/2622851888 shutdown_connections 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 --2- 172.21.15.97:0/2622851888 >> v2:172.21.15.188:3300/0 conn(0x7faf3806d090 0x7faf3806d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 --2- 172.21.15.97:0/2622851888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf3815a600 0x7faf3815aa60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 -- 172.21.15.97:0/2622851888 >> 172.21.15.97:0/2622851888 conn(0x7faf380b0a10 msgr2=0x7faf380b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 -- 172.21.15.97:0/2622851888 shutdown_connections 2024-01-31T05:02:32.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.945+0000 7faf3ea00640 1 -- 172.21.15.97:0/2622851888 wait complete. 2024-01-31T05:02:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.947+0000 7faf3ea00640 1 Processor -- start 2024-01-31T05:02:32.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.947+0000 7faf3ea00640 1 -- start start 2024-01-31T05:02:32.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf3ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3806d090 0x7faf38149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf3ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 0x7faf38159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf3ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3815a600 0x7faf3815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf3ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815a260 con 0x7faf380b1d10 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf3ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38159ca0 con 0x7faf3806d090 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf3ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf38159f80 con 0x7faf3815a600 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 0x7faf38159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 0x7faf38159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36888/0 (socket says 172.21.15.97:36888) 2024-01-31T05:02:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.948+0000 7faf37400640 1 -- 172.21.15.97:0/3104225842 learned_addr learned my addr 172.21.15.97:0/3104225842 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37e00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3806d090 0x7faf38149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf3ca00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3815a600 0x7faf3815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37400640 1 -- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3815a600 msgr2=0x7faf3815bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37400640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3815a600 0x7faf3815bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37400640 1 -- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3806d090 msgr2=0x7faf38149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37400640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3806d090 0x7faf38149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37400640 1 -- 172.21.15.97:0/3104225842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200094d0 con 0x7faf380b1d10 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37e00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3806d090 0x7faf38149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf37400640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 0x7faf38159450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faf2002b950 tx=0x7faf200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20031820 con 0x7faf380b1d10 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf200443f0 con 0x7faf380b1d10 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3815b7e0 con 0x7faf380b1d10 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.949+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf200447a0 con 0x7faf380b1d10 2024-01-31T05:02:32.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.950+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf38159990 con 0x7faf380b1d10 2024-01-31T05:02:32.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.951+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3815b640 con 0x7faf380b1d10 2024-01-31T05:02:32.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.952+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7faf2004d450 con 0x7faf380b1d10 2024-01-31T05:02:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.954+0000 7faf2fe00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf1006aed0 0x7faf1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:32.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.954+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faf2002d040 con 0x7faf380b1d10 2024-01-31T05:02:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.954+0000 7faf37e00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf1006aed0 0x7faf1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.955+0000 7faf37e00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf1006aed0 0x7faf1006d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:32.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:32.959+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7faf3815b640 con 0x7faf380b1d10 2024-01-31T05:02:33.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.152+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf3815b640 con 0x7faf380b1d10 2024-01-31T05:02:33.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.155+0000 7faf2fe00640 1 -- 172.21.15.97:0/3104225842 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16775 (secure 0 0 0) 0x7faf3815b640 con 0x7faf380b1d10 2024-01-31T05:02:33.156 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:33.156 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":40,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:32.839601+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:33.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.157+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf1006aed0 msgr2=0x7faf1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.157+0000 7faf3ea00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf1006aed0 0x7faf1006d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28009040 comp rx=0 tx=0).stop 2024-01-31T05:02:33.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.157+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 msgr2=0x7faf38159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.157+0000 7faf3ea00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 0x7faf38159450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faf2002b950 tx=0x7faf200046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 shutdown_connections 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf3815a600 0x7faf3815bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faf1006aed0 0x7faf1006d390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf380b1d10 0x7faf38159450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 --2- 172.21.15.97:0/3104225842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf3806d090 0x7faf38149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 >> 172.21.15.97:0/3104225842 conn(0x7faf380b0a10 msgr2=0x7faf3806de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 shutdown_connections 2024-01-31T05:02:33.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.158+0000 7faf3ea00640 1 -- 172.21.15.97:0/3104225842 wait complete. 2024-01-31T05:02:33.176 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-31T05:02:33.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.379+0000 7fc3f5c00640 1 Processor -- start 2024-01-31T05:02:33.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.379+0000 7fc3f5c00640 1 -- start start 2024-01-31T05:02:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.380+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 0x7fc3f00722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.380+0000 7fc3f5c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f006cd20 con 0x7fc3f015b470 2024-01-31T05:02:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.380+0000 7fc3f5c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f00ceb60 con 0x7fc3f00abcf0 2024-01-31T05:02:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.380+0000 7fc3f5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015b850 con 0x7fc3f00cbf20 2024-01-31T05:02:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.380+0000 7fc3eea00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc3f015b470 0x7fc3f00cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60450/0 (socket says 172.21.15.97:60450) 2024-01-31T05:02:33.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3eea00640 1 -- 172.21.15.97:0/3968513093 learned_addr learned my addr 172.21.15.97:0/3968513093 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:33.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3e5e00640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 0x7fc3f00722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411064035 0 0) 0x7fc3f006cd20 con 0x7fc3f015b470 2024-01-31T05:02:33.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d0003520 con 0x7fc3f015b470 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44121721 0 0) 0x7fc3d0003520 con 0x7fc3f015b470 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 msgr2=0x7fc3f00722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 0x7fc3f00722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 >> v1:172.21.15.188:6789/0 conn(0x7fc3f00abcf0 legacy=0x7fc3f006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f006c3f0 con 0x7fc3f015b470 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.381+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fc3e00017f0 con 0x7fc3f015b470 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 >> v1:172.21.15.97:6789/0 conn(0x7fc3f015b470 legacy=0x7fc3f00cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3d00039a0 0x7fc3d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 0x7fc3d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d0004d00 0x7fc3d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015b850 con 0x7fc3d0004d00 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000ba30 con 0x7fc3d00039a0 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000bbf0 con 0x7fc3f00cbf20 2024-01-31T05:02:33.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ef400640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d0004d00 0x7fc3d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3eea00640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3d00039a0 0x7fc3d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3e5e00640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 0x7fc3d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ef400640 1 -- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 msgr2=0x7fc3d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ef400640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f00cbf20 0x7fc3d00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ef400640 1 -- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3d00039a0 msgr2=0x7fc3d0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ef400640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3d00039a0 0x7fc3d0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3ef400640 1 -- 172.21.15.97:0/3968513093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0003520 con 0x7fc3d0004d00 2024-01-31T05:02:33.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.382+0000 7fc3eea00640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3d00039a0 0x7fc3d0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3ef400640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d0004d00 0x7fc3d0005180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00cb140 tx=0x7fc3d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a91d0c20c045dc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8003b70 con 0x7fc3d0004d00 2024-01-31T05:02:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8003cd0 con 0x7fc3d0004d00 2024-01-31T05:02:33.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3ee000640 1 -- 172.21.15.97:0/3968513093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005310 con 0x7fc3d0004d00 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3f5c00640 1 -- 172.21.15.97:0/3968513093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d0004d00 msgr2=0x7fc3d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/3968513093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d0004d00 0x7fc3d0005180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00cb140 tx=0x7fc3d00057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3ef400640 1 -- 172.21.15.97:0/3968513093 reap_dead start 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3f5c00640 1 -- 172.21.15.97:0/3968513093 shutdown_connections 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.384+0000 7fc3f5c00640 1 -- 172.21.15.97:0/3968513093 >> 172.21.15.97:0/3968513093 conn(0x7fc3f00b0a10 msgr2=0x7fc3f00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.385+0000 7fc3f5c00640 1 -- 172.21.15.97:0/3968513093 shutdown_connections 2024-01-31T05:02:33.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.385+0000 7fc3f5c00640 1 -- 172.21.15.97:0/3968513093 wait complete. 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.386+0000 7fc3f5c00640 1 Processor -- start 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.386+0000 7fc3f5c00640 1 -- start start 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.387+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3d0004d00 0x7fc3f015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.387+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f0152280 0x7fc3f009ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.387+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 0x7fc3f01533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.387+0000 7fc3f5c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015e170 con 0x7fc3f009c1b0 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.387+0000 7fc3f5c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f009b2d0 con 0x7fc3f0152280 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.387+0000 7fc3f5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015ce70 con 0x7fc3d0004d00 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 0x7fc3f01533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 0x7fc3f01533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36914/0 (socket says 172.21.15.97:36914) 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 -- 172.21.15.97:0/2013036202 learned_addr learned my addr 172.21.15.97:0/2013036202 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:33.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3e5e00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f0152280 0x7fc3f009ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3eea00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3d0004d00 0x7fc3f015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 -- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3d0004d00 msgr2=0x7fc3f015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3d0004d00 0x7fc3f015dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 -- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f0152280 msgr2=0x7fc3f009ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f0152280 0x7fc3f009ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 -- 172.21.15.97:0/2013036202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f015d150 con 0x7fc3f009c1b0 2024-01-31T05:02:33.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.388+0000 7fc3ef400640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 0x7fc3f01533d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc3e802c020 tx=0x7fc3e80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.389+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8006d50 con 0x7fc3f009c1b0 2024-01-31T05:02:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.389+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8007db0 con 0x7fc3f009c1b0 2024-01-31T05:02:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.389+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8007230 con 0x7fc3f009c1b0 2024-01-31T05:02:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.389+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f015d430 con 0x7fc3f009c1b0 2024-01-31T05:02:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.389+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f009c630 con 0x7fc3f009c1b0 2024-01-31T05:02:33.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.390+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f0067830 con 0x7fc3f009c1b0 2024-01-31T05:02:33.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.392+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc3e8007390 con 0x7fc3f009c1b0 2024-01-31T05:02:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.392+0000 7fc3e7400640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3c806af00 0x7fc3c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.393+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc3e80aafa0 con 0x7fc3f009c1b0 2024-01-31T05:02:33.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.393+0000 7fc3eea00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3c806af00 0x7fc3c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:33.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.394+0000 7fc3eea00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3c806af00 0x7fc3c806d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0001920 tx=0x7fc3e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:33.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.397+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc3e8075d50 con 0x7fc3f009c1b0 2024-01-31T05:02:33.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:33.634+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc3f015e170 con 0x7fc3f009c1b0 2024-01-31T05:02:34.150 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-01-31T05:02:34.150+0000 7fc5b9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:02:34.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.280+0000 7fc3e7400640 1 -- 172.21.15.97:0/2013036202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc3e8079370 con 0x7fc3f009c1b0 2024-01-31T05:02:34.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3c806af00 msgr2=0x7fc3c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3c806af00 0x7fc3c806d3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0001920 tx=0x7fc3e0009040 comp rx=0 tx=0).stop 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 msgr2=0x7fc3f01533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 0x7fc3f01533d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc3e802c020 tx=0x7fc3e80014a0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 shutdown_connections 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3c806af00 0x7fc3c806d3c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f009c1b0 0x7fc3f01533d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f0152280 0x7fc3f009ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 --2- 172.21.15.97:0/2013036202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3d0004d00 0x7fc3f015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 >> 172.21.15.97:0/2013036202 conn(0x7fc3f00b0a10 msgr2=0x7fc3f00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 shutdown_connections 2024-01-31T05:02:34.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.285+0000 7fc3f5c00640 1 -- 172.21.15.97:0/2013036202 wait complete. 2024-01-31T05:02:34.308 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-31T05:02:34.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-31T05:02:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.521+0000 7fa299a00640 1 Processor -- start 2024-01-31T05:02:34.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.521+0000 7fa299a00640 1 -- start start 2024-01-31T05:02:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa299a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa29415b470 0x7fa2940722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa299a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2940abcf0 0x7fa2940ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa299a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2940cb810 0x7fa2940cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa299a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940ce970 con 0x7fa2940cb810 2024-01-31T05:02:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa299a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2940cc1e0 con 0x7fa2940abcf0 2024-01-31T05:02:34.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa299a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29415b850 con 0x7fa29415b470 2024-01-31T05:02:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa293400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2940cb810 0x7fa2940cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa289e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2940abcf0 0x7fa2940ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa293400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa2940cb810 0x7fa2940cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36930/0 (socket says 172.21.15.97:36930) 2024-01-31T05:02:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa293400640 1 -- 172.21.15.97:0/1785313746 learned_addr learned my addr 172.21.15.97:0/1785313746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:34.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa292a00640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa29415b470 0x7fa2940722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa293400640 1 -- 172.21.15.97:0/1785313746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa29415b470 msgr2=0x7fa2940722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.523+0000 7fa293400640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa29415b470 0x7fa2940722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.524+0000 7fa293400640 1 -- 172.21.15.97:0/1785313746 >> v2:172.21.15.188:3300/0 conn(0x7fa2940abcf0 msgr2=0x7fa2940ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.524+0000 7fa293400640 1 --2- 172.21.15.97:0/1785313746 >> v2:172.21.15.188:3300/0 conn(0x7fa2940abcf0 0x7fa2940ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.524+0000 7fa293400640 1 -- 172.21.15.97:0/1785313746 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29406cd20 con 0x7fa2940cb810 2024-01-31T05:02:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.524+0000 7fa292a00640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa29415b470 0x7fa2940722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:02:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa293400640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940cb810 0x7fa2940cbca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa294072e90 tx=0x7fa28c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c856c5a850797b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa292000640 1 -- 172.21.15.97:0/1785313746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c02ca50 con 0x7fa2940cb810 2024-01-31T05:02:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa292000640 1 -- 172.21.15.97:0/1785313746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c02cbb0 con 0x7fa2940cb810 2024-01-31T05:02:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa292000640 1 -- 172.21.15.97:0/1785313746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0343f0 con 0x7fa2940cb810 2024-01-31T05:02:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 -- 172.21.15.97:0/1785313746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940cb810 msgr2=0x7fa2940cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:34.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940cb810 0x7fa2940cbca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa294072e90 tx=0x7fa28c02beb0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 -- 172.21.15.97:0/1785313746 shutdown_connections 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940cb810 0x7fa2940cbca0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 --2- 172.21.15.97:0/1785313746 >> v2:172.21.15.188:3300/0 conn(0x7fa2940abcf0 0x7fa2940ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 --2- 172.21.15.97:0/1785313746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa29415b470 0x7fa2940722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 -- 172.21.15.97:0/1785313746 >> 172.21.15.97:0/1785313746 conn(0x7fa2940b0a10 msgr2=0x7fa2940b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 -- 172.21.15.97:0/1785313746 shutdown_connections 2024-01-31T05:02:34.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.526+0000 7fa299a00640 1 -- 172.21.15.97:0/1785313746 wait complete. 2024-01-31T05:02:34.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.527+0000 7fa299a00640 1 Processor -- start 2024-01-31T05:02:34.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.528+0000 7fa299a00640 1 -- start start 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa299a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2940abcf0 0x7fa2940ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa299a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2940cb810 0x7fa2940ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa299a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 0x7fa294148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa299a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940c8880 con 0x7fa29415b470 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa299a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940c9130 con 0x7fa2940cb810 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa299a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940c8e50 con 0x7fa2940abcf0 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 0x7fa294148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 0x7fa294148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36938/0 (socket says 172.21.15.97:36938) 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 -- 172.21.15.97:0/1523636410 learned_addr learned my addr 172.21.15.97:0/1523636410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa289e00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2940cb810 0x7fa2940ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa292a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2940abcf0 0x7fa2940ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 -- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2940abcf0 msgr2=0x7fa2940ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2940abcf0 0x7fa2940ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 -- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2940cb810 msgr2=0x7fa2940ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2940cb810 0x7fa2940ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.529+0000 7fa293400640 1 -- 172.21.15.97:0/1523636410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c009b30 con 0x7fa29415b470 2024-01-31T05:02:34.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.530+0000 7fa293400640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 0x7fa294148480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009fa0 tx=0x7fa28c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.530+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c009510 con 0x7fa29415b470 2024-01-31T05:02:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.530+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c03e450 con 0x7fa29415b470 2024-01-31T05:02:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.530+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2940c8230 con 0x7fa29415b470 2024-01-31T05:02:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.530+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c03e800 con 0x7fa29415b470 2024-01-31T05:02:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.530+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29414f330 con 0x7fa29415b470 2024-01-31T05:02:34.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.531+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2940c8090 con 0x7fa29415b470 2024-01-31T05:02:34.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.532+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa28c03e960 con 0x7fa29415b470 2024-01-31T05:02:34.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.533+0000 7fa28b400640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa26c06af00 0x7fa26c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:34.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.533+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa28c0ac290 con 0x7fa29415b470 2024-01-31T05:02:34.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.534+0000 7fa292a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa26c06af00 0x7fa26c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:34.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.534+0000 7fa292a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa26c06af00 0x7fa26c06d3c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa28400b9f0 tx=0x7fa284008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:34.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.538+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa2940c8090 con 0x7fa29415b470 2024-01-31T05:02:34.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:34.779+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa2940b2890 con 0x7fa29415b470 2024-01-31T05:02:35.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.292+0000 7fa28b400640 1 -- 172.21.15.97:0/1523636410 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2940b2890 con 0x7fa29415b470 2024-01-31T05:02:35.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa26c06af00 msgr2=0x7fa26c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa26c06af00 0x7fa26c06d3c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa28400b9f0 tx=0x7fa284008040 comp rx=0 tx=0).stop 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 msgr2=0x7fa294148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 0x7fa294148480 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009fa0 tx=0x7fa28c004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 shutdown_connections 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa26c06af00 0x7fa26c06d3c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa29415b470 0x7fa294148480 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2940cb810 0x7fa2940ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 --2- 172.21.15.97:0/1523636410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2940abcf0 0x7fa2940ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 >> 172.21.15.97:0/1523636410 conn(0x7fa2940b0a10 msgr2=0x7fa294159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 shutdown_connections 2024-01-31T05:02:35.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.296+0000 7fa299a00640 1 -- 172.21.15.97:0/1523636410 wait complete. 2024-01-31T05:02:35.316 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-31T05:02:35.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:35.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.524+0000 7f3e91200640 1 Processor -- start 2024-01-31T05:02:35.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.524+0000 7f3e91200640 1 -- start start 2024-01-31T05:02:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e91200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 0x7f3e8c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e91200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c06d1a0 con 0x7f3e8c0ccd00 2024-01-31T05:02:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e91200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c06c600 con 0x7f3e8c15b400 2024-01-31T05:02:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e91200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0cc960 con 0x7f3e8c0cb810 2024-01-31T05:02:35.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e8a800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3e8c0ccd00 0x7f3e8c071cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58454/0 (socket says 172.21.15.97:58454) 2024-01-31T05:02:35.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e8a800640 1 -- 172.21.15.97:0/421954309 learned_addr learned my addr 172.21.15.97:0/421954309 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e89e00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 0x7f3e8c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.526+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972946234 0 0) 0x7f3e8c06d1a0 con 0x7f3e8c0ccd00 2024-01-31T05:02:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e6c003520 con 0x7f3e8c0ccd00 2024-01-31T05:02:35.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414671314 0 0) 0x7f3e6c003520 con 0x7f3e8c0ccd00 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 msgr2=0x7f3e8c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 0x7f3e8c0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 >> v1:172.21.15.188:6789/0 conn(0x7f3e8c15b400 legacy=0x7f3e8c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0cc130 con 0x7f3e8c0ccd00 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f3e840019f0 con 0x7f3e8c0ccd00 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 >> v1:172.21.15.97:6789/0 conn(0x7f3e8c0ccd00 legacy=0x7f3e8c071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0039a0 0x7f3e6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c0047c0 0x7f3e6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 0x7f3e6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0cc960 con 0x7f3e6c0039a0 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c00ba60 con 0x7f3e6c0047c0 2024-01-31T05:02:35.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c00bc20 con 0x7f3e8c0cb810 2024-01-31T05:02:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.527+0000 7f3e89e00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0039a0 0x7f3e6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.528+0000 7f3e89e00640 1 -- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 msgr2=0x7f3e6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.528+0000 7f3e89e00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e8c0cb810 0x7f3e6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.528+0000 7f3e89e00640 1 -- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c0047c0 msgr2=0x7f3e6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.528+0000 7f3e89e00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e6c0047c0 0x7f3e6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.528+0000 7f3e89e00640 1 -- 172.21.15.97:0/421954309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c003520 con 0x7f3e6c0039a0 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.529+0000 7f3e89e00640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0039a0 0x7f3e6c003dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009a90 tx=0x7f3e6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b76f962f2026f80 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.529+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02c690 con 0x7f3e6c0039a0 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.529+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c02c7f0 con 0x7f3e6c0039a0 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.529+0000 7f3e88a00640 1 -- 172.21.15.97:0/421954309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02cb80 con 0x7f3e6c0039a0 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e91200640 1 -- 172.21.15.97:0/421954309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0039a0 msgr2=0x7f3e6c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e91200640 1 --2- 172.21.15.97:0/421954309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0039a0 0x7f3e6c003dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009a90 tx=0x7f3e6c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e8a800640 1 -- 172.21.15.97:0/421954309 reap_dead start 2024-01-31T05:02:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e91200640 1 -- 172.21.15.97:0/421954309 shutdown_connections 2024-01-31T05:02:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e91200640 1 -- 172.21.15.97:0/421954309 >> 172.21.15.97:0/421954309 conn(0x7f3e8c0b0a10 msgr2=0x7f3e8c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e91200640 1 -- 172.21.15.97:0/421954309 shutdown_connections 2024-01-31T05:02:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.530+0000 7f3e91200640 1 -- 172.21.15.97:0/421954309 wait complete. 2024-01-31T05:02:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.531+0000 7f3e91200640 1 Processor -- start 2024-01-31T05:02:35.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 -- start start 2024-01-31T05:02:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0039a0 0x7f3e8c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 0x7f3e8c0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e8c0c85d0 0x7f3e8c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0b3ea0 con 0x7f3e8c15df10 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c09ba20 con 0x7f3e8c0c85d0 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e91200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c1f3b50 con 0x7f3e6c0039a0 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 0x7f3e8c0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 0x7f3e8c0c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54348/0 (socket says 172.21.15.97:54348) 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 -- 172.21.15.97:0/101950475 learned_addr learned my addr 172.21.15.97:0/101950475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 -- 172.21.15.97:0/101950475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0039a0 msgr2=0x7f3e8c1f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0039a0 0x7f3e8c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 -- 172.21.15.97:0/101950475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e8c0c85d0 msgr2=0x7f3e8c14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e8c0c85d0 0x7f3e8c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 -- 172.21.15.97:0/101950475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c15e470 con 0x7f3e8c15df10 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.532+0000 7f3e89400640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 0x7f3e8c0c8090 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c460 tx=0x7f3e7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.534+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740052d0 con 0x7f3e8c15df10 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.534+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74005450 con 0x7f3e8c15df10 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.534+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74007260 con 0x7f3e8c15df10 2024-01-31T05:02:35.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.534+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c09b3d0 con 0x7f3e8c15df10 2024-01-31T05:02:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.534+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c09b080 con 0x7f3e8c15df10 2024-01-31T05:02:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.535+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c0677c0 con 0x7f3e8c15df10 2024-01-31T05:02:35.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.536+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3e740073c0 con 0x7f3e8c15df10 2024-01-31T05:02:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.537+0000 7f3e82000640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3e6406aed0 0x7f3e6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.537+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3e74088b40 con 0x7f3e8c15df10 2024-01-31T05:02:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.537+0000 7f3e89e00640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3e6406aed0 0x7f3e6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.538+0000 7f3e89e00640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3e6406aed0 0x7f3e6406d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c036000 tx=0x7f3e7c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:35.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.541+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3e74053830 con 0x7f3e8c15df10 2024-01-31T05:02:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.770+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e8c0736f0 con 0x7f3e8c15df10 2024-01-31T05:02:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.771+0000 7f3e82000640 1 -- 172.21.15.97:0/101950475 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f3e74056e50 con 0x7f3e8c15df10 2024-01-31T05:02:35.770 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:35.771 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:34.782181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:02:35.771 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3e6406aed0 msgr2=0x7f3e6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3e6406aed0 0x7f3e6406d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c036000 tx=0x7f3e7c02e000 comp rx=0 tx=0).stop 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 msgr2=0x7f3e8c0c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 0x7f3e8c0c8090 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c460 tx=0x7f3e7400c920 comp rx=0 tx=0).stop 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 shutdown_connections 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e8c0c85d0 0x7f3e8c14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3e6406aed0 0x7f3e6406d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e8c15df10 0x7f3e8c0c8090 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 --2- 172.21.15.97:0/101950475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3e6c0039a0 0x7f3e8c1f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.773+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 >> 172.21.15.97:0/101950475 conn(0x7f3e8c0b0a10 msgr2=0x7f3e8c0ad6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.774+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 shutdown_connections 2024-01-31T05:02:35.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:35.774+0000 7f3e91200640 1 -- 172.21.15.97:0/101950475 wait complete. 2024-01-31T05:02:35.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:36.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.004+0000 7ff9b2e00640 1 Processor -- start 2024-01-31T05:02:36.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.004+0000 7ff9b2e00640 1 -- start start 2024-01-31T05:02:36.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.005+0000 7ff9b2e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9ac0b1d10 0x7ff9ac0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9b2e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 0x7ff9ac15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9b2e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0b2630 con 0x7ff9ac15dc50 2024-01-31T05:02:36.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9b2e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac06c8b0 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9b2e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac15b120 con 0x7ff9ac15a600 2024-01-31T05:02:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9ab400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 0x7ff9ac15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9abe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9ac0b1d10 0x7ff9ac0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9abe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9ac0b1d10 0x7ff9ac0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:56534/0 (socket says 172.21.15.97:56534) 2024-01-31T05:02:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9abe00640 1 -- 172.21.15.97:0/1408865491 learned_addr learned my addr 172.21.15.97:0/1408865491 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:36.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.006+0000 7ff9ab400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 0x7ff9ac15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:54974/0 (socket says 172.21.15.97:54974) 2024-01-31T05:02:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.007+0000 7ff9ab400640 1 -- 172.21.15.97:0/1408865491 >> v2:172.21.15.188:3300/0 conn(0x7ff9ac0b1d10 msgr2=0x7ff9ac0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.007+0000 7ff9ab400640 1 --2- 172.21.15.97:0/1408865491 >> v2:172.21.15.188:3300/0 conn(0x7ff9ac0b1d10 0x7ff9ac0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.007+0000 7ff9ab400640 1 -- 172.21.15.97:0/1408865491 >> v1:172.21.15.188:6789/0 conn(0x7ff9ac15dc50 legacy=0x7ff9ac15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.007+0000 7ff9ab400640 1 -- 172.21.15.97:0/1408865491 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac15b400 con 0x7ff9ac15a600 2024-01-31T05:02:36.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.008+0000 7ff9ab400640 1 --2- 172.21.15.97:0/1408865491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 0x7ff9ac15aa60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff9940097a0 tx=0x7ff99402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0147ed602e0397 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.009+0000 7ff9aaa00640 1 -- 172.21.15.97:0/1408865491 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99402d040 con 0x7ff9ac15a600 2024-01-31T05:02:36.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.009+0000 7ff9aaa00640 1 -- 172.21.15.97:0/1408865491 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994031810 con 0x7ff9ac15a600 2024-01-31T05:02:36.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.009+0000 7ff9aaa00640 1 -- 172.21.15.97:0/1408865491 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994031ba0 con 0x7ff9ac15a600 2024-01-31T05:02:36.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.009+0000 7ff9b2e00640 1 -- 172.21.15.97:0/1408865491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 msgr2=0x7ff9ac15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.009+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/1408865491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 0x7ff9ac15aa60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff9940097a0 tx=0x7ff99402b8b0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.010+0000 7ff9b2e00640 1 -- 172.21.15.97:0/1408865491 shutdown_connections 2024-01-31T05:02:36.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.010+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/1408865491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15a600 0x7ff9ac15aa60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.010+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/1408865491 >> v2:172.21.15.188:3300/0 conn(0x7ff9ac0b1d10 0x7ff9ac0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.010+0000 7ff9b2e00640 1 -- 172.21.15.97:0/1408865491 >> 172.21.15.97:0/1408865491 conn(0x7ff9ac0b0a10 msgr2=0x7ff9ac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:36.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.010+0000 7ff9b2e00640 1 -- 172.21.15.97:0/1408865491 shutdown_connections 2024-01-31T05:02:36.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.010+0000 7ff9b2e00640 1 -- 172.21.15.97:0/1408865491 wait complete. 2024-01-31T05:02:36.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.011+0000 7ff9b2e00640 1 Processor -- start 2024-01-31T05:02:36.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 -- start start 2024-01-31T05:02:36.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 0x7ff9ac0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac15a600 0x7ff9ac0718a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15dc50 0x7ff9ac142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac099f00 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac070800 con 0x7ff9ac15a600 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.012+0000 7ff9b2e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac070ae0 con 0x7ff9ac15dc50 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 0x7ff9ac0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 0x7ff9ac0999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54356/0 (socket says 172.21.15.97:54356) 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 -- 172.21.15.97:0/2131682733 learned_addr learned my addr 172.21.15.97:0/2131682733 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 -- 172.21.15.97:0/2131682733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15dc50 msgr2=0x7ff9ac142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15dc50 0x7ff9ac142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 -- 172.21.15.97:0/2131682733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac15a600 msgr2=0x7ff9ac0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac15a600 0x7ff9ac0718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 -- 172.21.15.97:0/2131682733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994009400 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.013+0000 7ff9abe00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 0x7ff9ac0999c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00e840 tx=0x7ff99c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.014+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c00c8e0 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.014+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c00ca40 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.014+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c00cdf0 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.014+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac0adb80 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.014+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac071380 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.015+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac0677c0 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.016+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff99c013470 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.017+0000 7ff9a3e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff98406aed0 0x7ff98406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.017+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff99c089820 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.017+0000 7ff9ab400640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff98406aed0 0x7ff98406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.018+0000 7ff9ab400640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff98406aed0 0x7ff98406d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff994036000 tx=0x7ff994030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.022+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff99c0545b0 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.203+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9ac06e740 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.206+0000 7ff9a3e00640 1 -- 172.21.15.97:0/2131682733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16775 (secure 0 0 0) 0x7ff99c017090 con 0x7ff9ac0b1d10 2024-01-31T05:02:36.206 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:36.206 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":40,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:32.839601+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff98406aed0 msgr2=0x7ff98406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff98406aed0 0x7ff98406d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff994036000 tx=0x7ff994030000 comp rx=0 tx=0).stop 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 msgr2=0x7ff9ac0999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 0x7ff9ac0999c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00e840 tx=0x7ff99c00ed00 comp rx=0 tx=0).stop 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 shutdown_connections 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15dc50 0x7ff9ac142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac15a600 0x7ff9ac0718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff98406aed0 0x7ff98406d390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 --2- 172.21.15.97:0/2131682733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0b1d10 0x7ff9ac0999c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 >> 172.21.15.97:0/2131682733 conn(0x7ff9ac0b0a10 msgr2=0x7ff9ac0abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 shutdown_connections 2024-01-31T05:02:36.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.209+0000 7ff9b2e00640 1 -- 172.21.15.97:0/2131682733 wait complete. 2024-01-31T05:02:36.227 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-31T05:02:36.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:02:36.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.427+0000 7f2983e00640 1 Processor -- start 2024-01-31T05:02:36.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.427+0000 7f2983e00640 1 -- start start 2024-01-31T05:02:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2983e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f298415d2d0 0x7f29840d3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2983e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2983e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29840d1b50 0x7f29840cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2983e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29840d1fc0 con 0x7f298415d2d0 2024-01-31T05:02:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2983e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29840cef50 con 0x7f29840d1b50 2024-01-31T05:02:36.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2983e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29840d10f0 con 0x7f29840b1f50 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2982a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f298415d2d0 0x7f29840d3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2982a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f298415d2d0 0x7f29840d3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54368/0 (socket says 172.21.15.97:54368) 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.429+0000 7f2982a00640 1 -- 172.21.15.97:0/3984053009 learned_addr learned my addr 172.21.15.97:0/3984053009 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f2982a00640 1 -- 172.21.15.97:0/3984053009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 msgr2=0x7f29840d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f297be00640 1 --2- 172.21.15.97:0/3984053009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f2983400640 1 --2- 172.21.15.97:0/3984053009 >> v2:172.21.15.188:3300/0 conn(0x7f29840d1b50 0x7f29840cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f2982a00640 1 --2- 172.21.15.97:0/3984053009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f2982a00640 1 -- 172.21.15.97:0/3984053009 >> v2:172.21.15.188:3300/0 conn(0x7f29840d1b50 msgr2=0x7f29840cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f2982a00640 1 --2- 172.21.15.97:0/3984053009 >> v2:172.21.15.188:3300/0 conn(0x7f29840d1b50 0x7f29840cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.430+0000 7f2982a00640 1 -- 172.21.15.97:0/3984053009 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29840ce6f0 con 0x7f298415d2d0 2024-01-31T05:02:36.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.431+0000 7f2982a00640 1 --2- 172.21.15.97:0/3984053009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298415d2d0 0x7f29840d3b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2984160ec0 tx=0x7f297402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c6ac1c8981b65f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.431+0000 7f2982000640 1 -- 172.21.15.97:0/3984053009 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740317e0 con 0x7f298415d2d0 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.431+0000 7f2982000640 1 -- 172.21.15.97:0/3984053009 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974031940 con 0x7f298415d2d0 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.431+0000 7f2982000640 1 -- 172.21.15.97:0/3984053009 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974031cd0 con 0x7f298415d2d0 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 -- 172.21.15.97:0/3984053009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298415d2d0 msgr2=0x7f29840d3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 --2- 172.21.15.97:0/3984053009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298415d2d0 0x7f29840d3b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2984160ec0 tx=0x7f297402b880 comp rx=0 tx=0).stop 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 -- 172.21.15.97:0/3984053009 shutdown_connections 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 --2- 172.21.15.97:0/3984053009 >> v2:172.21.15.188:3300/0 conn(0x7f29840d1b50 0x7f29840cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 --2- 172.21.15.97:0/3984053009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 --2- 172.21.15.97:0/3984053009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298415d2d0 0x7f29840d3b50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 -- 172.21.15.97:0/3984053009 >> 172.21.15.97:0/3984053009 conn(0x7f29840d3030 msgr2=0x7f29840d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 -- 172.21.15.97:0/3984053009 shutdown_connections 2024-01-31T05:02:36.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.432+0000 7f2983e00640 1 -- 172.21.15.97:0/3984053009 wait complete. 2024-01-31T05:02:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.434+0000 7f2983e00640 1 Processor -- start 2024-01-31T05:02:36.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.434+0000 7f2983e00640 1 -- start start 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.435+0000 7f2983e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.435+0000 7f2983e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 0x7f2984156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.435+0000 7f2983e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f298415d2d0 0x7f298414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.435+0000 7f2983e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984158480 con 0x7f29840d1b50 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.435+0000 7f2983e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29841588a0 con 0x7f298415d2d0 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.435+0000 7f2983e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2984157800 con 0x7f29840b1f50 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 0x7f2984156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 0x7f2984156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54376/0 (socket says 172.21.15.97:54376) 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 -- 172.21.15.97:0/2091444386 learned_addr learned my addr 172.21.15.97:0/2091444386 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 -- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 msgr2=0x7f29840cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f2982a00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f2983400640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f298415d2d0 0x7f298414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 -- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f298415d2d0 msgr2=0x7f298414cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f298415d2d0 0x7f298414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 -- 172.21.15.97:0/2091444386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740094d0 con 0x7f29840d1b50 2024-01-31T05:02:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.436+0000 7f297be00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 0x7f2984156d40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f296c00c5a0 tx=0x7f296c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.437+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c004220 con 0x7f29840d1b50 2024-01-31T05:02:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.437+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c004380 con 0x7f29840d1b50 2024-01-31T05:02:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.437+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c003850 con 0x7f29840d1b50 2024-01-31T05:02:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.437+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2984157ae0 con 0x7f29840d1b50 2024-01-31T05:02:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.437+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29841580a0 con 0x7f29840d1b50 2024-01-31T05:02:36.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.438+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29841585c0 con 0x7f29840d1b50 2024-01-31T05:02:36.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.439+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f296c0039b0 con 0x7f29840d1b50 2024-01-31T05:02:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.440+0000 7f297aa00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f295c06aed0 0x7f295c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.440+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f296c0893e0 con 0x7f29840d1b50 2024-01-31T05:02:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.441+0000 7f2982a00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f295c06aed0 0x7f295c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.441+0000 7f2982a00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f295c06aed0 0x7f295c06d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2974036000 tx=0x7f297403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.445+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f29841585c0 con 0x7f29840d1b50 2024-01-31T05:02:36.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.634+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29840cec70 con 0x7f295c06aed0 2024-01-31T05:02:36.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.650+0000 7f297aa00640 1 -- 172.21.15.97:0/2091444386 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287234 (secure 0 0 0) 0x7f2974031c30 con 0x7f295c06aed0 2024-01-31T05:02:36.654 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:02:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.655+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f295c06aed0 msgr2=0x7f295c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.655+0000 7f2983e00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f295c06aed0 0x7f295c06d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2974036000 tx=0x7f297403d040 comp rx=0 tx=0).stop 2024-01-31T05:02:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.655+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 msgr2=0x7f2984156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.655+0000 7f2983e00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 0x7f2984156d40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f296c00c5a0 tx=0x7f296c00ca60 comp rx=0 tx=0).stop 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 shutdown_connections 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f298415d2d0 0x7f298414cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f295c06aed0 0x7f295c06d390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29840d1b50 0x7f2984156d40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 --2- 172.21.15.97:0/2091444386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29840b1f50 0x7f29840cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 >> 172.21.15.97:0/2091444386 conn(0x7f29840d3030 msgr2=0x7f29840b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 shutdown_connections 2024-01-31T05:02:36.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.656+0000 7f2983e00640 1 -- 172.21.15.97:0/2091444386 wait complete. 2024-01-31T05:02:36.673 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-01-31T05:02:35.587557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220936,"kb_used_data":4764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753784,"statfs":{"total":773094113280,"available":772867874816,"internally_reserved":0,"allocated":4878336,"data_stored":2731552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.365490"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852607+0000","last_change":"2024-01-31T05:02:31.862463+0000","last_active":"2024-01-31T05:02:32.852607+0000","last_peered":"2024-01-31T05:02:32.852607+0000","last_clean":"2024-01-31T05:02:32.852607+0000","last_became_active":"2024-01-31T05:02:31.862126+0000","last_became_peered":"2024-01-31T05:02:31.862126+0000","last_unstale":"2024-01-31T05:02:32.852607+0000","last_undegraded":"2024-01-31T05:02:32.852607+0000","last_fullsized":"2024-01-31T05:02:32.852607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:03.888729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850900+0000","last_change":"2024-01-31T05:02:31.875565+0000","last_active":"2024-01-31T05:02:32.850900+0000","last_peered":"2024-01-31T05:02:32.850900+0000","last_clean":"2024-01-31T05:02:32.850900+0000","last_became_active":"2024-01-31T05:02:31.874063+0000","last_became_peered":"2024-01-31T05:02:31.874063+0000","last_unstale":"2024-01-31T05:02:32.850900+0000","last_undegraded":"2024-01-31T05:02:32.850900+0000","last_fullsized":"2024-01-31T05:02:32.850900+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:57.318680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852013+0000","last_change":"2024-01-31T05:02:31.877123+0000","last_active":"2024-01-31T05:02:32.852013+0000","last_peered":"2024-01-31T05:02:32.852013+0000","last_clean":"2024-01-31T05:02:32.852013+0000","last_became_active":"2024-01-31T05:02:31.876844+0000","last_became_peered":"2024-01-31T05:02:31.876844+0000","last_unstale":"2024-01-31T05:02:32.852013+0000","last_undegraded":"2024-01-31T05:02:32.852013+0000","last_fullsized":"2024-01-31T05:02:32.852013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:03.071117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851857+0000","last_change":"2024-01-31T05:02:31.878009+0000","last_active":"2024-01-31T05:02:32.851857+0000","last_peered":"2024-01-31T05:02:32.851857+0000","last_clean":"2024-01-31T05:02:32.851857+0000","last_became_active":"2024-01-31T05:02:31.877742+0000","last_became_peered":"2024-01-31T05:02:31.877742+0000","last_unstale":"2024-01-31T05:02:32.851857+0000","last_undegraded":"2024-01-31T05:02:32.851857+0000","last_fullsized":"2024-01-31T05:02:32.851857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:06:07.665433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389637+0000","last_change":"2024-01-31T05:02:31.876537+0000","last_active":"2024-01-31T05:02:33.389637+0000","last_peered":"2024-01-31T05:02:33.389637+0000","last_clean":"2024-01-31T05:02:33.389637+0000","last_became_active":"2024-01-31T05:02:31.876245+0000","last_became_peered":"2024-01-31T05:02:31.876245+0000","last_unstale":"2024-01-31T05:02:33.389637+0000","last_undegraded":"2024-01-31T05:02:33.389637+0000","last_fullsized":"2024-01-31T05:02:33.389637+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:42.190280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850610+0000","last_change":"2024-01-31T05:02:31.869363+0000","last_active":"2024-01-31T05:02:32.850610+0000","last_peered":"2024-01-31T05:02:32.850610+0000","last_clean":"2024-01-31T05:02:32.850610+0000","last_became_active":"2024-01-31T05:02:31.869077+0000","last_became_peered":"2024-01-31T05:02:31.869077+0000","last_unstale":"2024-01-31T05:02:32.850610+0000","last_undegraded":"2024-01-31T05:02:32.850610+0000","last_fullsized":"2024-01-31T05:02:32.850610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:27:51.934492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636751+0000","last_change":"2024-01-31T05:02:31.866927+0000","last_active":"2024-01-31T05:02:33.636751+0000","last_peered":"2024-01-31T05:02:33.636751+0000","last_clean":"2024-01-31T05:02:33.636751+0000","last_became_active":"2024-01-31T05:02:31.866385+0000","last_became_peered":"2024-01-31T05:02:31.866385+0000","last_unstale":"2024-01-31T05:02:33.636751+0000","last_undegraded":"2024-01-31T05:02:33.636751+0000","last_fullsized":"2024-01-31T05:02:33.636751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:51:11.026998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849844+0000","last_change":"2024-01-31T05:02:31.871523+0000","last_active":"2024-01-31T05:02:32.849844+0000","last_peered":"2024-01-31T05:02:32.849844+0000","last_clean":"2024-01-31T05:02:32.849844+0000","last_became_active":"2024-01-31T05:02:31.870534+0000","last_became_peered":"2024-01-31T05:02:31.870534+0000","last_unstale":"2024-01-31T05:02:32.849844+0000","last_undegraded":"2024-01-31T05:02:32.849844+0000","last_fullsized":"2024-01-31T05:02:32.849844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:18:30.141878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389977+0000","last_change":"2024-01-31T05:02:31.877818+0000","last_active":"2024-01-31T05:02:33.389977+0000","last_peered":"2024-01-31T05:02:33.389977+0000","last_clean":"2024-01-31T05:02:33.389977+0000","last_became_active":"2024-01-31T05:02:31.877562+0000","last_became_peered":"2024-01-31T05:02:31.877562+0000","last_unstale":"2024-01-31T05:02:33.389977+0000","last_undegraded":"2024-01-31T05:02:33.389977+0000","last_fullsized":"2024-01-31T05:02:33.389977+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:51:11.026998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637417+0000","last_change":"2024-01-31T05:02:31.881948+0000","last_active":"2024-01-31T05:02:33.637417+0000","last_peered":"2024-01-31T05:02:33.637417+0000","last_clean":"2024-01-31T05:02:33.637417+0000","last_became_active":"2024-01-31T05:02:31.881230+0000","last_became_peered":"2024-01-31T05:02:31.881230+0000","last_unstale":"2024-01-31T05:02:33.637417+0000","last_undegraded":"2024-01-31T05:02:33.637417+0000","last_fullsized":"2024-01-31T05:02:33.637417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:35:16.668974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393746+0000","last_change":"2024-01-31T05:02:31.875423+0000","last_active":"2024-01-31T05:02:33.393746+0000","last_peered":"2024-01-31T05:02:33.393746+0000","last_clean":"2024-01-31T05:02:33.393746+0000","last_became_active":"2024-01-31T05:02:31.875161+0000","last_became_peered":"2024-01-31T05:02:31.875161+0000","last_unstale":"2024-01-31T05:02:33.393746+0000","last_undegraded":"2024-01-31T05:02:33.393746+0000","last_fullsized":"2024-01-31T05:02:33.393746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:43:54.231166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389707+0000","last_change":"2024-01-31T05:02:31.880363+0000","last_active":"2024-01-31T05:02:33.389707+0000","last_peered":"2024-01-31T05:02:33.389707+0000","last_clean":"2024-01-31T05:02:33.389707+0000","last_became_active":"2024-01-31T05:02:31.879182+0000","last_became_peered":"2024-01-31T05:02:31.879182+0000","last_unstale":"2024-01-31T05:02:33.389707+0000","last_undegraded":"2024-01-31T05:02:33.389707+0000","last_fullsized":"2024-01-31T05:02:33.389707+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:40:59.923350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637123+0000","last_change":"2024-01-31T05:02:31.880164+0000","last_active":"2024-01-31T05:02:33.637123+0000","last_peered":"2024-01-31T05:02:33.637123+0000","last_clean":"2024-01-31T05:02:33.637123+0000","last_became_active":"2024-01-31T05:02:31.879634+0000","last_became_peered":"2024-01-31T05:02:31.879634+0000","last_unstale":"2024-01-31T05:02:33.637123+0000","last_undegraded":"2024-01-31T05:02:33.637123+0000","last_fullsized":"2024-01-31T05:02:33.637123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:59:46.315012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633079+0000","last_change":"2024-01-31T05:02:31.868717+0000","last_active":"2024-01-31T05:02:33.633079+0000","last_peered":"2024-01-31T05:02:33.633079+0000","last_clean":"2024-01-31T05:02:33.633079+0000","last_became_active":"2024-01-31T05:02:31.868453+0000","last_became_peered":"2024-01-31T05:02:31.868453+0000","last_unstale":"2024-01-31T05:02:33.633079+0000","last_undegraded":"2024-01-31T05:02:33.633079+0000","last_fullsized":"2024-01-31T05:02:33.633079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:41:32.038067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847730+0000","last_change":"2024-01-31T05:02:31.880181+0000","last_active":"2024-01-31T05:02:32.847730+0000","last_peered":"2024-01-31T05:02:32.847730+0000","last_clean":"2024-01-31T05:02:32.847730+0000","last_became_active":"2024-01-31T05:02:31.879926+0000","last_became_peered":"2024-01-31T05:02:31.879926+0000","last_unstale":"2024-01-31T05:02:32.847730+0000","last_undegraded":"2024-01-31T05:02:32.847730+0000","last_fullsized":"2024-01-31T05:02:32.847730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:46:26.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636711+0000","last_change":"2024-01-31T05:02:31.880158+0000","last_active":"2024-01-31T05:02:33.636711+0000","last_peered":"2024-01-31T05:02:33.636711+0000","last_clean":"2024-01-31T05:02:33.636711+0000","last_became_active":"2024-01-31T05:02:31.879895+0000","last_became_peered":"2024-01-31T05:02:31.879895+0000","last_unstale":"2024-01-31T05:02:33.636711+0000","last_undegraded":"2024-01-31T05:02:33.636711+0000","last_fullsized":"2024-01-31T05:02:33.636711+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:40:59.923350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848567+0000","last_change":"2024-01-31T05:02:31.882801+0000","last_active":"2024-01-31T05:02:32.848567+0000","last_peered":"2024-01-31T05:02:32.848567+0000","last_clean":"2024-01-31T05:02:32.848567+0000","last_became_active":"2024-01-31T05:02:31.881247+0000","last_became_peered":"2024-01-31T05:02:31.881247+0000","last_unstale":"2024-01-31T05:02:32.848567+0000","last_undegraded":"2024-01-31T05:02:32.848567+0000","last_fullsized":"2024-01-31T05:02:32.848567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:02:26.441143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848175+0000","last_change":"2024-01-31T05:02:31.879308+0000","last_active":"2024-01-31T05:02:32.848175+0000","last_peered":"2024-01-31T05:02:32.848175+0000","last_clean":"2024-01-31T05:02:32.848175+0000","last_became_active":"2024-01-31T05:02:31.876184+0000","last_became_peered":"2024-01-31T05:02:31.876184+0000","last_unstale":"2024-01-31T05:02:32.848175+0000","last_undegraded":"2024-01-31T05:02:32.848175+0000","last_fullsized":"2024-01-31T05:02:32.848175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:34:42.139273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638959+0000","last_change":"2024-01-31T05:02:31.877503+0000","last_active":"2024-01-31T05:02:33.638959+0000","last_peered":"2024-01-31T05:02:33.638959+0000","last_clean":"2024-01-31T05:02:33.638959+0000","last_became_active":"2024-01-31T05:02:31.877230+0000","last_became_peered":"2024-01-31T05:02:31.877230+0000","last_unstale":"2024-01-31T05:02:33.638959+0000","last_undegraded":"2024-01-31T05:02:33.638959+0000","last_fullsized":"2024-01-31T05:02:33.638959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:22:17.142232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393423+0000","last_change":"2024-01-31T05:02:31.878823+0000","last_active":"2024-01-31T05:02:33.393423+0000","last_peered":"2024-01-31T05:02:33.393423+0000","last_clean":"2024-01-31T05:02:33.393423+0000","last_became_active":"2024-01-31T05:02:31.878556+0000","last_became_peered":"2024-01-31T05:02:31.878556+0000","last_unstale":"2024-01-31T05:02:33.393423+0000","last_undegraded":"2024-01-31T05:02:33.393423+0000","last_fullsized":"2024-01-31T05:02:33.393423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:43:21.077786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393366+0000","last_change":"2024-01-31T05:02:31.879043+0000","last_active":"2024-01-31T05:02:33.393366+0000","last_peered":"2024-01-31T05:02:33.393366+0000","last_clean":"2024-01-31T05:02:33.393366+0000","last_became_active":"2024-01-31T05:02:31.878786+0000","last_became_peered":"2024-01-31T05:02:31.878786+0000","last_unstale":"2024-01-31T05:02:33.393366+0000","last_undegraded":"2024-01-31T05:02:33.393366+0000","last_fullsized":"2024-01-31T05:02:33.393366+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:50:09.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393489+0000","last_change":"2024-01-31T05:02:31.875472+0000","last_active":"2024-01-31T05:02:33.393489+0000","last_peered":"2024-01-31T05:02:33.393489+0000","last_clean":"2024-01-31T05:02:33.393489+0000","last_became_active":"2024-01-31T05:02:31.874699+0000","last_became_peered":"2024-01-31T05:02:31.874699+0000","last_unstale":"2024-01-31T05:02:33.393489+0000","last_undegraded":"2024-01-31T05:02:33.393489+0000","last_fullsized":"2024-01-31T05:02:33.393489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:02:27.914422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847560+0000","last_change":"2024-01-31T05:02:31.883795+0000","last_active":"2024-01-31T05:02:32.847560+0000","last_peered":"2024-01-31T05:02:32.847560+0000","last_clean":"2024-01-31T05:02:32.847560+0000","last_became_active":"2024-01-31T05:02:31.882949+0000","last_became_peered":"2024-01-31T05:02:31.882949+0000","last_unstale":"2024-01-31T05:02:32.847560+0000","last_undegraded":"2024-01-31T05:02:32.847560+0000","last_fullsized":"2024-01-31T05:02:32.847560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:09:53.884970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849887+0000","last_change":"2024-01-31T05:02:31.867632+0000","last_active":"2024-01-31T05:02:32.849887+0000","last_peered":"2024-01-31T05:02:32.849887+0000","last_clean":"2024-01-31T05:02:32.849887+0000","last_became_active":"2024-01-31T05:02:31.867025+0000","last_became_peered":"2024-01-31T05:02:31.867025+0000","last_unstale":"2024-01-31T05:02:32.849887+0000","last_undegraded":"2024-01-31T05:02:32.849887+0000","last_fullsized":"2024-01-31T05:02:32.849887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:47:19.273646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.634090+0000","last_change":"2024-01-31T05:02:31.877765+0000","last_active":"2024-01-31T05:02:33.634090+0000","last_peered":"2024-01-31T05:02:33.634090+0000","last_clean":"2024-01-31T05:02:33.634090+0000","last_became_active":"2024-01-31T05:02:31.877523+0000","last_became_peered":"2024-01-31T05:02:31.877523+0000","last_unstale":"2024-01-31T05:02:33.634090+0000","last_undegraded":"2024-01-31T05:02:33.634090+0000","last_fullsized":"2024-01-31T05:02:33.634090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:49.577996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851526+0000","last_change":"2024-01-31T05:02:31.872149+0000","last_active":"2024-01-31T05:02:32.851526+0000","last_peered":"2024-01-31T05:02:32.851526+0000","last_clean":"2024-01-31T05:02:32.851526+0000","last_became_active":"2024-01-31T05:02:31.870394+0000","last_became_peered":"2024-01-31T05:02:31.870394+0000","last_unstale":"2024-01-31T05:02:32.851526+0000","last_undegraded":"2024-01-31T05:02:32.851526+0000","last_fullsized":"2024-01-31T05:02:32.851526+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:40:23.763126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393876+0000","last_change":"2024-01-31T05:02:31.876551+0000","last_active":"2024-01-31T05:02:33.393876+0000","last_peered":"2024-01-31T05:02:33.393876+0000","last_clean":"2024-01-31T05:02:33.393876+0000","last_became_active":"2024-01-31T05:02:31.876291+0000","last_became_peered":"2024-01-31T05:02:31.876291+0000","last_unstale":"2024-01-31T05:02:33.393876+0000","last_undegraded":"2024-01-31T05:02:33.393876+0000","last_fullsized":"2024-01-31T05:02:33.393876+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:21:03.928004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637300+0000","last_change":"2024-01-31T05:02:31.864617+0000","last_active":"2024-01-31T05:02:33.637300+0000","last_peered":"2024-01-31T05:02:33.637300+0000","last_clean":"2024-01-31T05:02:33.637300+0000","last_became_active":"2024-01-31T05:02:31.863915+0000","last_became_peered":"2024-01-31T05:02:31.863915+0000","last_unstale":"2024-01-31T05:02:33.637300+0000","last_undegraded":"2024-01-31T05:02:33.637300+0000","last_fullsized":"2024-01-31T05:02:33.637300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:41:32.038067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847860+0000","last_change":"2024-01-31T05:02:31.877672+0000","last_active":"2024-01-31T05:02:32.847860+0000","last_peered":"2024-01-31T05:02:32.847860+0000","last_clean":"2024-01-31T05:02:32.847860+0000","last_became_active":"2024-01-31T05:02:31.877435+0000","last_became_peered":"2024-01-31T05:02:31.877435+0000","last_unstale":"2024-01-31T05:02:32.847860+0000","last_undegraded":"2024-01-31T05:02:32.847860+0000","last_fullsized":"2024-01-31T05:02:32.847860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:12:31.764335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847801+0000","last_change":"2024-01-31T05:02:31.884527+0000","last_active":"2024-01-31T05:02:32.847801+0000","last_peered":"2024-01-31T05:02:32.847801+0000","last_clean":"2024-01-31T05:02:32.847801+0000","last_became_active":"2024-01-31T05:02:31.884279+0000","last_became_peered":"2024-01-31T05:02:31.884279+0000","last_unstale":"2024-01-31T05:02:32.847801+0000","last_undegraded":"2024-01-31T05:02:32.847801+0000","last_fullsized":"2024-01-31T05:02:32.847801+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:02:30.931525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636894+0000","last_change":"2024-01-31T05:02:31.869818+0000","last_active":"2024-01-31T05:02:33.636894+0000","last_peered":"2024-01-31T05:02:33.636894+0000","last_clean":"2024-01-31T05:02:33.636894+0000","last_became_active":"2024-01-31T05:02:31.869547+0000","last_became_peered":"2024-01-31T05:02:31.869547+0000","last_unstale":"2024-01-31T05:02:33.636894+0000","last_undegraded":"2024-01-31T05:02:33.636894+0000","last_fullsized":"2024-01-31T05:02:33.636894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:34:23.316642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850578+0000","last_change":"2024-01-31T05:02:31.871216+0000","last_active":"2024-01-31T05:02:32.850578+0000","last_peered":"2024-01-31T05:02:32.850578+0000","last_clean":"2024-01-31T05:02:32.850578+0000","last_became_active":"2024-01-31T05:02:31.870159+0000","last_became_peered":"2024-01-31T05:02:31.870159+0000","last_unstale":"2024-01-31T05:02:32.850578+0000","last_undegraded":"2024-01-31T05:02:32.850578+0000","last_fullsized":"2024-01-31T05:02:32.850578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:15.569227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848805+0000","last_change":"2024-01-31T05:02:31.875682+0000","last_active":"2024-01-31T05:02:32.848805+0000","last_peered":"2024-01-31T05:02:32.848805+0000","last_clean":"2024-01-31T05:02:32.848805+0000","last_became_active":"2024-01-31T05:02:31.874983+0000","last_became_peered":"2024-01-31T05:02:31.874983+0000","last_unstale":"2024-01-31T05:02:32.848805+0000","last_undegraded":"2024-01-31T05:02:32.848805+0000","last_fullsized":"2024-01-31T05:02:32.848805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:54:43.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.634357+0000","last_change":"2024-01-31T05:02:31.244919+0000","last_active":"2024-01-31T05:02:33.634357+0000","last_peered":"2024-01-31T05:02:33.634357+0000","last_clean":"2024-01-31T05:02:33.634357+0000","last_became_active":"2024-01-31T05:02:31.244423+0000","last_became_peered":"2024-01-31T05:02:31.244423+0000","last_unstale":"2024-01-31T05:02:33.634357+0000","last_undegraded":"2024-01-31T05:02:33.634357+0000","last_fullsized":"2024-01-31T05:02:33.634357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:51:10.015785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852780+0000","last_change":"2024-01-31T05:02:31.875103+0000","last_active":"2024-01-31T05:02:32.852780+0000","last_peered":"2024-01-31T05:02:32.852780+0000","last_clean":"2024-01-31T05:02:32.852780+0000","last_became_active":"2024-01-31T05:02:31.874793+0000","last_became_peered":"2024-01-31T05:02:31.874793+0000","last_unstale":"2024-01-31T05:02:32.852780+0000","last_undegraded":"2024-01-31T05:02:32.852780+0000","last_fullsized":"2024-01-31T05:02:32.852780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:50:09.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850846+0000","last_change":"2024-01-31T05:02:31.270188+0000","last_active":"2024-01-31T05:02:32.850846+0000","last_peered":"2024-01-31T05:02:32.850846+0000","last_clean":"2024-01-31T05:02:32.850846+0000","last_became_active":"2024-01-31T05:02:31.268820+0000","last_became_peered":"2024-01-31T05:02:31.268820+0000","last_unstale":"2024-01-31T05:02:32.850846+0000","last_undegraded":"2024-01-31T05:02:32.850846+0000","last_fullsized":"2024-01-31T05:02:32.850846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:48:10.110170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848484+0000","last_change":"2024-01-31T05:02:31.872070+0000","last_active":"2024-01-31T05:02:32.848484+0000","last_peered":"2024-01-31T05:02:32.848484+0000","last_clean":"2024-01-31T05:02:32.848484+0000","last_became_active":"2024-01-31T05:02:31.871758+0000","last_became_peered":"2024-01-31T05:02:31.871758+0000","last_unstale":"2024-01-31T05:02:32.848484+0000","last_undegraded":"2024-01-31T05:02:32.848484+0000","last_fullsized":"2024-01-31T05:02:32.848484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:38:45.016439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852310+0000","last_change":"2024-01-31T05:02:31.271167+0000","last_active":"2024-01-31T05:02:32.852310+0000","last_peered":"2024-01-31T05:02:32.852310+0000","last_clean":"2024-01-31T05:02:32.852310+0000","last_became_active":"2024-01-31T05:02:31.270868+0000","last_became_peered":"2024-01-31T05:02:31.270868+0000","last_unstale":"2024-01-31T05:02:32.852310+0000","last_undegraded":"2024-01-31T05:02:32.852310+0000","last_fullsized":"2024-01-31T05:02:32.852310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:40.024536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851391+0000","last_change":"2024-01-31T05:02:31.873903+0000","last_active":"2024-01-31T05:02:32.851391+0000","last_peered":"2024-01-31T05:02:32.851391+0000","last_clean":"2024-01-31T05:02:32.851391+0000","last_became_active":"2024-01-31T05:02:31.873442+0000","last_became_peered":"2024-01-31T05:02:31.873442+0000","last_unstale":"2024-01-31T05:02:32.851391+0000","last_undegraded":"2024-01-31T05:02:32.851391+0000","last_fullsized":"2024-01-31T05:02:32.851391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:32:04.118546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633544+0000","last_change":"2024-01-31T05:02:31.267286+0000","last_active":"2024-01-31T05:02:33.633544+0000","last_peered":"2024-01-31T05:02:33.633544+0000","last_clean":"2024-01-31T05:02:33.633544+0000","last_became_active":"2024-01-31T05:02:31.266815+0000","last_became_peered":"2024-01-31T05:02:31.266815+0000","last_unstale":"2024-01-31T05:02:33.633544+0000","last_undegraded":"2024-01-31T05:02:33.633544+0000","last_fullsized":"2024-01-31T05:02:33.633544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:35:44.023929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850284+0000","last_change":"2024-01-31T05:02:31.870847+0000","last_active":"2024-01-31T05:02:32.850284+0000","last_peered":"2024-01-31T05:02:32.850284+0000","last_clean":"2024-01-31T05:02:32.850284+0000","last_became_active":"2024-01-31T05:02:31.869744+0000","last_became_peered":"2024-01-31T05:02:31.869744+0000","last_unstale":"2024-01-31T05:02:32.850284+0000","last_undegraded":"2024-01-31T05:02:32.850284+0000","last_fullsized":"2024-01-31T05:02:32.850284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:51:43.956563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633556+0000","last_change":"2024-01-31T05:02:31.258203+0000","last_active":"2024-01-31T05:02:33.633556+0000","last_peered":"2024-01-31T05:02:33.633556+0000","last_clean":"2024-01-31T05:02:33.633556+0000","last_became_active":"2024-01-31T05:02:31.257613+0000","last_became_peered":"2024-01-31T05:02:31.257613+0000","last_unstale":"2024-01-31T05:02:33.633556+0000","last_undegraded":"2024-01-31T05:02:33.633556+0000","last_fullsized":"2024-01-31T05:02:33.633556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:41.179067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393285+0000","last_change":"2024-01-31T05:02:31.870332+0000","last_active":"2024-01-31T05:02:33.393285+0000","last_peered":"2024-01-31T05:02:33.393285+0000","last_clean":"2024-01-31T05:02:33.393285+0000","last_became_active":"2024-01-31T05:02:31.870067+0000","last_became_peered":"2024-01-31T05:02:31.870067+0000","last_unstale":"2024-01-31T05:02:33.393285+0000","last_undegraded":"2024-01-31T05:02:33.393285+0000","last_fullsized":"2024-01-31T05:02:33.393285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:13:22.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637017+0000","last_change":"2024-01-31T05:02:31.267811+0000","last_active":"2024-01-31T05:02:33.637017+0000","last_peered":"2024-01-31T05:02:33.637017+0000","last_clean":"2024-01-31T05:02:33.637017+0000","last_became_active":"2024-01-31T05:02:31.267372+0000","last_became_peered":"2024-01-31T05:02:31.267372+0000","last_unstale":"2024-01-31T05:02:33.637017+0000","last_undegraded":"2024-01-31T05:02:33.637017+0000","last_fullsized":"2024-01-31T05:02:33.637017+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:15:52.267028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848225+0000","last_change":"2024-01-31T05:02:31.881170+0000","last_active":"2024-01-31T05:02:32.848225+0000","last_peered":"2024-01-31T05:02:32.848225+0000","last_clean":"2024-01-31T05:02:32.848225+0000","last_became_active":"2024-01-31T05:02:31.880911+0000","last_became_peered":"2024-01-31T05:02:31.880911+0000","last_unstale":"2024-01-31T05:02:32.848225+0000","last_undegraded":"2024-01-31T05:02:32.848225+0000","last_fullsized":"2024-01-31T05:02:32.848225+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:53:22.934871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850161+0000","last_change":"2024-01-31T05:02:31.271499+0000","last_active":"2024-01-31T05:02:32.850161+0000","last_peered":"2024-01-31T05:02:32.850161+0000","last_clean":"2024-01-31T05:02:32.850161+0000","last_became_active":"2024-01-31T05:02:31.271079+0000","last_became_peered":"2024-01-31T05:02:31.271079+0000","last_unstale":"2024-01-31T05:02:32.850161+0000","last_undegraded":"2024-01-31T05:02:32.850161+0000","last_fullsized":"2024-01-31T05:02:32.850161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:09.596995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851657+0000","last_change":"2024-01-31T05:02:31.871548+0000","last_active":"2024-01-31T05:02:32.851657+0000","last_peered":"2024-01-31T05:02:32.851657+0000","last_clean":"2024-01-31T05:02:32.851657+0000","last_became_active":"2024-01-31T05:02:31.871251+0000","last_became_peered":"2024-01-31T05:02:31.871251+0000","last_unstale":"2024-01-31T05:02:32.851657+0000","last_undegraded":"2024-01-31T05:02:32.851657+0000","last_fullsized":"2024-01-31T05:02:32.851657+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:13:22.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636940+0000","last_change":"2024-01-31T05:02:31.266460+0000","last_active":"2024-01-31T05:02:33.636940+0000","last_peered":"2024-01-31T05:02:33.636940+0000","last_clean":"2024-01-31T05:02:33.636940+0000","last_became_active":"2024-01-31T05:02:31.266223+0000","last_became_peered":"2024-01-31T05:02:31.266223+0000","last_unstale":"2024-01-31T05:02:33.636940+0000","last_undegraded":"2024-01-31T05:02:33.636940+0000","last_fullsized":"2024-01-31T05:02:33.636940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:23:15.917063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852321+0000","last_change":"2024-01-31T05:02:31.877404+0000","last_active":"2024-01-31T05:02:32.852321+0000","last_peered":"2024-01-31T05:02:32.852321+0000","last_clean":"2024-01-31T05:02:32.852321+0000","last_became_active":"2024-01-31T05:02:31.877014+0000","last_became_peered":"2024-01-31T05:02:31.877014+0000","last_unstale":"2024-01-31T05:02:32.852321+0000","last_undegraded":"2024-01-31T05:02:32.852321+0000","last_fullsized":"2024-01-31T05:02:32.852321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:52:53.957175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852709+0000","last_change":"2024-01-31T05:02:31.266140+0000","last_active":"2024-01-31T05:02:32.852709+0000","last_peered":"2024-01-31T05:02:32.852709+0000","last_clean":"2024-01-31T05:02:32.852709+0000","last_became_active":"2024-01-31T05:02:31.265668+0000","last_became_peered":"2024-01-31T05:02:31.265668+0000","last_unstale":"2024-01-31T05:02:32.852709+0000","last_undegraded":"2024-01-31T05:02:32.852709+0000","last_fullsized":"2024-01-31T05:02:32.852709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:33.507917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633140+0000","last_change":"2024-01-31T05:02:31.873285+0000","last_active":"2024-01-31T05:02:33.633140+0000","last_peered":"2024-01-31T05:02:33.633140+0000","last_clean":"2024-01-31T05:02:33.633140+0000","last_became_active":"2024-01-31T05:02:31.872844+0000","last_became_peered":"2024-01-31T05:02:31.872844+0000","last_unstale":"2024-01-31T05:02:33.633140+0000","last_undegraded":"2024-01-31T05:02:33.633140+0000","last_fullsized":"2024-01-31T05:02:33.633140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:35:16.668974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848182+0000","last_change":"2024-01-31T05:02:31.261021+0000","last_active":"2024-01-31T05:02:32.848182+0000","last_peered":"2024-01-31T05:02:32.848182+0000","last_clean":"2024-01-31T05:02:32.848182+0000","last_became_active":"2024-01-31T05:02:31.260743+0000","last_became_peered":"2024-01-31T05:02:31.260743+0000","last_unstale":"2024-01-31T05:02:32.848182+0000","last_undegraded":"2024-01-31T05:02:32.848182+0000","last_fullsized":"2024-01-31T05:02:32.848182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:25:46.965349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852053+0000","last_change":"2024-01-31T05:02:31.872454+0000","last_active":"2024-01-31T05:02:32.852053+0000","last_peered":"2024-01-31T05:02:32.852053+0000","last_clean":"2024-01-31T05:02:32.852053+0000","last_became_active":"2024-01-31T05:02:31.871846+0000","last_became_peered":"2024-01-31T05:02:31.871846+0000","last_unstale":"2024-01-31T05:02:32.852053+0000","last_undegraded":"2024-01-31T05:02:32.852053+0000","last_fullsized":"2024-01-31T05:02:32.852053+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:03.150918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848419+0000","last_change":"2024-01-31T05:02:31.268384+0000","last_active":"2024-01-31T05:02:32.848419+0000","last_peered":"2024-01-31T05:02:32.848419+0000","last_clean":"2024-01-31T05:02:32.848419+0000","last_became_active":"2024-01-31T05:02:31.267973+0000","last_became_peered":"2024-01-31T05:02:31.267973+0000","last_unstale":"2024-01-31T05:02:32.848419+0000","last_undegraded":"2024-01-31T05:02:32.848419+0000","last_fullsized":"2024-01-31T05:02:32.848419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:14.558014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637143+0000","last_change":"2024-01-31T05:02:31.880907+0000","last_active":"2024-01-31T05:02:33.637143+0000","last_peered":"2024-01-31T05:02:33.637143+0000","last_clean":"2024-01-31T05:02:33.637143+0000","last_became_active":"2024-01-31T05:02:31.880640+0000","last_became_peered":"2024-01-31T05:02:31.880640+0000","last_unstale":"2024-01-31T05:02:33.637143+0000","last_undegraded":"2024-01-31T05:02:33.637143+0000","last_fullsized":"2024-01-31T05:02:33.637143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:10.608208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852354+0000","last_change":"2024-01-31T05:02:31.269065+0000","last_active":"2024-01-31T05:02:32.852354+0000","last_peered":"2024-01-31T05:02:32.852354+0000","last_clean":"2024-01-31T05:02:32.852354+0000","last_became_active":"2024-01-31T05:02:31.268574+0000","last_became_peered":"2024-01-31T05:02:31.268574+0000","last_unstale":"2024-01-31T05:02:32.852354+0000","last_undegraded":"2024-01-31T05:02:32.852354+0000","last_fullsized":"2024-01-31T05:02:32.852354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:35:15.657762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850707+0000","last_change":"2024-01-31T05:02:31.867347+0000","last_active":"2024-01-31T05:02:32.850707+0000","last_peered":"2024-01-31T05:02:32.850707+0000","last_clean":"2024-01-31T05:02:32.850707+0000","last_became_active":"2024-01-31T05:02:31.866500+0000","last_became_peered":"2024-01-31T05:02:31.866500+0000","last_unstale":"2024-01-31T05:02:32.850707+0000","last_undegraded":"2024-01-31T05:02:32.850707+0000","last_fullsized":"2024-01-31T05:02:32.850707+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:25:47.976561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850090+0000","last_change":"2024-01-31T05:02:31.267874+0000","last_active":"2024-01-31T05:02:32.850090+0000","last_peered":"2024-01-31T05:02:32.850090+0000","last_clean":"2024-01-31T05:02:32.850090+0000","last_became_active":"2024-01-31T05:02:31.267483+0000","last_became_peered":"2024-01-31T05:02:31.267483+0000","last_unstale":"2024-01-31T05:02:32.850090+0000","last_undegraded":"2024-01-31T05:02:32.850090+0000","last_fullsized":"2024-01-31T05:02:32.850090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:16.500985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850342+0000","last_change":"2024-01-31T05:02:31.875891+0000","last_active":"2024-01-31T05:02:32.850342+0000","last_peered":"2024-01-31T05:02:32.850342+0000","last_clean":"2024-01-31T05:02:32.850342+0000","last_became_active":"2024-01-31T05:02:31.875599+0000","last_became_peered":"2024-01-31T05:02:31.875599+0000","last_unstale":"2024-01-31T05:02:32.850342+0000","last_undegraded":"2024-01-31T05:02:32.850342+0000","last_fullsized":"2024-01-31T05:02:32.850342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:19:34.125226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850865+0000","last_change":"2024-01-31T05:02:31.269485+0000","last_active":"2024-01-31T05:02:32.850865+0000","last_peered":"2024-01-31T05:02:32.850865+0000","last_clean":"2024-01-31T05:02:32.850865+0000","last_became_active":"2024-01-31T05:02:31.268894+0000","last_became_peered":"2024-01-31T05:02:31.268894+0000","last_unstale":"2024-01-31T05:02:32.850865+0000","last_undegraded":"2024-01-31T05:02:32.850865+0000","last_fullsized":"2024-01-31T05:02:32.850865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:33:30.699042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393641+0000","last_change":"2024-01-31T05:02:31.873351+0000","last_active":"2024-01-31T05:02:33.393641+0000","last_peered":"2024-01-31T05:02:33.393641+0000","last_clean":"2024-01-31T05:02:33.393641+0000","last_became_active":"2024-01-31T05:02:31.873054+0000","last_became_peered":"2024-01-31T05:02:31.873054+0000","last_unstale":"2024-01-31T05:02:33.393641+0000","last_undegraded":"2024-01-31T05:02:33.393641+0000","last_fullsized":"2024-01-31T05:02:33.393641+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:25:21.456002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847678+0000","last_change":"2024-01-31T05:02:31.260162+0000","last_active":"2024-01-31T05:02:32.847678+0000","last_peered":"2024-01-31T05:02:32.847678+0000","last_clean":"2024-01-31T05:02:32.847678+0000","last_became_active":"2024-01-31T05:02:31.259179+0000","last_became_peered":"2024-01-31T05:02:31.259179+0000","last_unstale":"2024-01-31T05:02:32.847678+0000","last_undegraded":"2024-01-31T05:02:32.847678+0000","last_fullsized":"2024-01-31T05:02:32.847678+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:47:18.262434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389495+0000","last_change":"2024-01-31T05:02:31.883109+0000","last_active":"2024-01-31T05:02:33.389495+0000","last_peered":"2024-01-31T05:02:33.389495+0000","last_clean":"2024-01-31T05:02:33.389495+0000","last_became_active":"2024-01-31T05:02:31.882315+0000","last_became_peered":"2024-01-31T05:02:31.882315+0000","last_unstale":"2024-01-31T05:02:33.389495+0000","last_undegraded":"2024-01-31T05:02:33.389495+0000","last_fullsized":"2024-01-31T05:02:33.389495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:35:45.035142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.632836+0000","last_change":"2024-01-31T05:02:31.264411+0000","last_active":"2024-01-31T05:02:33.632836+0000","last_peered":"2024-01-31T05:02:33.632836+0000","last_clean":"2024-01-31T05:02:33.632836+0000","last_became_active":"2024-01-31T05:02:31.264119+0000","last_became_peered":"2024-01-31T05:02:31.264119+0000","last_unstale":"2024-01-31T05:02:33.632836+0000","last_undegraded":"2024-01-31T05:02:33.632836+0000","last_fullsized":"2024-01-31T05:02:33.632836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:40:58.912138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851351+0000","last_change":"2024-01-31T05:02:31.870998+0000","last_active":"2024-01-31T05:02:32.851351+0000","last_peered":"2024-01-31T05:02:32.851351+0000","last_clean":"2024-01-31T05:02:32.851351+0000","last_became_active":"2024-01-31T05:02:31.870686+0000","last_became_peered":"2024-01-31T05:02:31.870686+0000","last_unstale":"2024-01-31T05:02:32.851351+0000","last_undegraded":"2024-01-31T05:02:32.851351+0000","last_fullsized":"2024-01-31T05:02:32.851351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:03.633979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637242+0000","last_change":"2024-01-31T05:02:31.271009+0000","last_active":"2024-01-31T05:02:33.637242+0000","last_peered":"2024-01-31T05:02:33.637242+0000","last_clean":"2024-01-31T05:02:33.637242+0000","last_became_active":"2024-01-31T05:02:31.270689+0000","last_became_peered":"2024-01-31T05:02:31.270689+0000","last_unstale":"2024-01-31T05:02:33.637242+0000","last_undegraded":"2024-01-31T05:02:33.637242+0000","last_fullsized":"2024-01-31T05:02:33.637242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:50:54.221182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637228+0000","last_change":"2024-01-31T05:02:31.864362+0000","last_active":"2024-01-31T05:02:33.637228+0000","last_peered":"2024-01-31T05:02:33.637228+0000","last_clean":"2024-01-31T05:02:33.637228+0000","last_became_active":"2024-01-31T05:02:31.863638+0000","last_became_peered":"2024-01-31T05:02:31.863638+0000","last_unstale":"2024-01-31T05:02:33.637228+0000","last_undegraded":"2024-01-31T05:02:33.637228+0000","last_fullsized":"2024-01-31T05:02:33.637228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:17.512197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848362+0000","last_change":"2024-01-31T05:02:31.266270+0000","last_active":"2024-01-31T05:02:32.848362+0000","last_peered":"2024-01-31T05:02:32.848362+0000","last_clean":"2024-01-31T05:02:32.848362+0000","last_became_active":"2024-01-31T05:02:31.265985+0000","last_became_peered":"2024-01-31T05:02:31.265985+0000","last_unstale":"2024-01-31T05:02:32.848362+0000","last_undegraded":"2024-01-31T05:02:32.848362+0000","last_fullsized":"2024-01-31T05:02:32.848362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:40:22.751913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637708+0000","last_change":"2024-01-31T05:02:31.876731+0000","last_active":"2024-01-31T05:02:33.637708+0000","last_peered":"2024-01-31T05:02:33.637708+0000","last_clean":"2024-01-31T05:02:33.637708+0000","last_became_active":"2024-01-31T05:02:31.875897+0000","last_became_peered":"2024-01-31T05:02:31.875897+0000","last_unstale":"2024-01-31T05:02:33.637708+0000","last_undegraded":"2024-01-31T05:02:33.637708+0000","last_fullsized":"2024-01-31T05:02:33.637708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:49.577996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389562+0000","last_change":"2024-01-31T05:02:31.267514+0000","last_active":"2024-01-31T05:02:33.389562+0000","last_peered":"2024-01-31T05:02:33.389562+0000","last_clean":"2024-01-31T05:02:33.389562+0000","last_became_active":"2024-01-31T05:02:31.267102+0000","last_became_peered":"2024-01-31T05:02:31.267102+0000","last_unstale":"2024-01-31T05:02:33.389562+0000","last_undegraded":"2024-01-31T05:02:33.389562+0000","last_fullsized":"2024-01-31T05:02:33.389562+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:15:52.267028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.639136+0000","last_change":"2024-01-31T05:02:31.879023+0000","last_active":"2024-01-31T05:02:33.639136+0000","last_peered":"2024-01-31T05:02:33.639136+0000","last_clean":"2024-01-31T05:02:33.639136+0000","last_became_active":"2024-01-31T05:02:31.878756+0000","last_became_peered":"2024-01-31T05:02:31.878756+0000","last_unstale":"2024-01-31T05:02:33.639136+0000","last_undegraded":"2024-01-31T05:02:33.639136+0000","last_fullsized":"2024-01-31T05:02:33.639136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:57.318680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638781+0000","last_change":"2024-01-31T05:02:31.270602+0000","last_active":"2024-01-31T05:02:33.638781+0000","last_peered":"2024-01-31T05:02:33.638781+0000","last_clean":"2024-01-31T05:02:33.638781+0000","last_became_active":"2024-01-31T05:02:31.269556+0000","last_became_peered":"2024-01-31T05:02:31.269556+0000","last_unstale":"2024-01-31T05:02:33.638781+0000","last_undegraded":"2024-01-31T05:02:33.638781+0000","last_fullsized":"2024-01-31T05:02:33.638781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:51:47.047325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847470+0000","last_change":"2024-01-31T05:02:31.874593+0000","last_active":"2024-01-31T05:02:32.847470+0000","last_peered":"2024-01-31T05:02:32.847470+0000","last_clean":"2024-01-31T05:02:32.847470+0000","last_became_active":"2024-01-31T05:02:31.874300+0000","last_became_peered":"2024-01-31T05:02:31.874300+0000","last_unstale":"2024-01-31T05:02:32.847470+0000","last_undegraded":"2024-01-31T05:02:32.847470+0000","last_fullsized":"2024-01-31T05:02:32.847470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:18:43.735715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638696+0000","last_change":"2024-01-31T05:02:31.261011+0000","last_active":"2024-01-31T05:02:33.638696+0000","last_peered":"2024-01-31T05:02:33.638696+0000","last_clean":"2024-01-31T05:02:33.638696+0000","last_became_active":"2024-01-31T05:02:31.259901+0000","last_became_peered":"2024-01-31T05:02:31.259901+0000","last_unstale":"2024-01-31T05:02:33.638696+0000","last_undegraded":"2024-01-31T05:02:33.638696+0000","last_fullsized":"2024-01-31T05:02:33.638696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:26:17.957679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849872+0000","last_change":"2024-01-31T05:02:31.864488+0000","last_active":"2024-01-31T05:02:32.849872+0000","last_peered":"2024-01-31T05:02:32.849872+0000","last_clean":"2024-01-31T05:02:32.849872+0000","last_became_active":"2024-01-31T05:02:31.863826+0000","last_became_peered":"2024-01-31T05:02:31.863826+0000","last_unstale":"2024-01-31T05:02:32.849872+0000","last_undegraded":"2024-01-31T05:02:32.849872+0000","last_fullsized":"2024-01-31T05:02:32.849872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:26:04.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851111+0000","last_change":"2024-01-31T05:02:31.256596+0000","last_active":"2024-01-31T05:02:32.851111+0000","last_peered":"2024-01-31T05:02:32.851111+0000","last_clean":"2024-01-31T05:02:32.851111+0000","last_became_active":"2024-01-31T05:02:31.256269+0000","last_became_peered":"2024-01-31T05:02:31.256269+0000","last_unstale":"2024-01-31T05:02:32.851111+0000","last_undegraded":"2024-01-31T05:02:32.851111+0000","last_fullsized":"2024-01-31T05:02:32.851111+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:44:09.657004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393504+0000","last_change":"2024-01-31T05:02:31.875706+0000","last_active":"2024-01-31T05:02:33.393504+0000","last_peered":"2024-01-31T05:02:33.393504+0000","last_clean":"2024-01-31T05:02:33.393504+0000","last_became_active":"2024-01-31T05:02:31.875030+0000","last_became_peered":"2024-01-31T05:02:31.875030+0000","last_unstale":"2024-01-31T05:02:33.393504+0000","last_undegraded":"2024-01-31T05:02:33.393504+0000","last_fullsized":"2024-01-31T05:02:33.393504+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:43:28.000528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637075+0000","last_change":"2024-01-31T05:02:31.272108+0000","last_active":"2024-01-31T05:02:33.637075+0000","last_peered":"2024-01-31T05:02:33.637075+0000","last_clean":"2024-01-31T05:02:33.637075+0000","last_became_active":"2024-01-31T05:02:31.271812+0000","last_became_peered":"2024-01-31T05:02:31.271812+0000","last_unstale":"2024-01-31T05:02:33.637075+0000","last_undegraded":"2024-01-31T05:02:33.637075+0000","last_fullsized":"2024-01-31T05:02:33.637075+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:00:56.863174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849797+0000","last_change":"2024-01-31T05:02:31.867868+0000","last_active":"2024-01-31T05:02:32.849797+0000","last_peered":"2024-01-31T05:02:32.849797+0000","last_clean":"2024-01-31T05:02:32.849797+0000","last_became_active":"2024-01-31T05:02:31.867474+0000","last_became_peered":"2024-01-31T05:02:31.867474+0000","last_unstale":"2024-01-31T05:02:32.849797+0000","last_undegraded":"2024-01-31T05:02:32.849797+0000","last_fullsized":"2024-01-31T05:02:32.849797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:43.970952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849515+0000","last_change":"2024-01-31T05:02:31.263062+0000","last_active":"2024-01-31T05:02:32.849515+0000","last_peered":"2024-01-31T05:02:32.849515+0000","last_clean":"2024-01-31T05:02:32.849515+0000","last_became_active":"2024-01-31T05:02:31.262782+0000","last_became_peered":"2024-01-31T05:02:31.262782+0000","last_unstale":"2024-01-31T05:02:32.849515+0000","last_undegraded":"2024-01-31T05:02:32.849515+0000","last_fullsized":"2024-01-31T05:02:32.849515+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.037570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.390102+0000","last_change":"2024-01-31T05:02:31.870719+0000","last_active":"2024-01-31T05:02:33.390102+0000","last_peered":"2024-01-31T05:02:33.390102+0000","last_clean":"2024-01-31T05:02:33.390102+0000","last_became_active":"2024-01-31T05:02:31.870400+0000","last_became_peered":"2024-01-31T05:02:31.870400+0000","last_unstale":"2024-01-31T05:02:33.390102+0000","last_undegraded":"2024-01-31T05:02:33.390102+0000","last_fullsized":"2024-01-31T05:02:33.390102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:34:23.316642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848657+0000","last_change":"2024-01-31T05:02:31.260406+0000","last_active":"2024-01-31T05:02:32.848657+0000","last_peered":"2024-01-31T05:02:32.848657+0000","last_clean":"2024-01-31T05:02:32.848657+0000","last_became_active":"2024-01-31T05:02:31.258453+0000","last_became_peered":"2024-01-31T05:02:31.258453+0000","last_unstale":"2024-01-31T05:02:32.848657+0000","last_undegraded":"2024-01-31T05:02:32.848657+0000","last_fullsized":"2024-01-31T05:02:32.848657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:06:06.654220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393928+0000","last_change":"2024-01-31T05:02:31.873336+0000","last_active":"2024-01-31T05:02:33.393928+0000","last_peered":"2024-01-31T05:02:33.393928+0000","last_clean":"2024-01-31T05:02:33.393928+0000","last_became_active":"2024-01-31T05:02:31.873037+0000","last_became_peered":"2024-01-31T05:02:31.873037+0000","last_unstale":"2024-01-31T05:02:33.393928+0000","last_undegraded":"2024-01-31T05:02:33.393928+0000","last_fullsized":"2024-01-31T05:02:33.393928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:49:45.758767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.392862+0000","last_change":"2024-01-31T05:02:31.264450+0000","last_active":"2024-01-31T05:02:33.392862+0000","last_peered":"2024-01-31T05:02:33.392862+0000","last_clean":"2024-01-31T05:02:33.392862+0000","last_became_active":"2024-01-31T05:02:31.264030+0000","last_became_peered":"2024-01-31T05:02:31.264030+0000","last_unstale":"2024-01-31T05:02:33.392862+0000","last_undegraded":"2024-01-31T05:02:33.392862+0000","last_fullsized":"2024-01-31T05:02:33.392862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:23:15.917063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638835+0000","last_change":"2024-01-31T05:02:31.874669+0000","last_active":"2024-01-31T05:02:33.638835+0000","last_peered":"2024-01-31T05:02:33.638835+0000","last_clean":"2024-01-31T05:02:33.638835+0000","last_became_active":"2024-01-31T05:02:31.874380+0000","last_became_peered":"2024-01-31T05:02:31.874380+0000","last_unstale":"2024-01-31T05:02:33.638835+0000","last_undegraded":"2024-01-31T05:02:33.638835+0000","last_fullsized":"2024-01-31T05:02:33.638835+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:27:51.934492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389576+0000","last_change":"2024-01-31T05:02:31.267507+0000","last_active":"2024-01-31T05:02:33.389576+0000","last_peered":"2024-01-31T05:02:33.389576+0000","last_clean":"2024-01-31T05:02:33.389576+0000","last_became_active":"2024-01-31T05:02:31.267138+0000","last_became_peered":"2024-01-31T05:02:31.267138+0000","last_unstale":"2024-01-31T05:02:33.389576+0000","last_undegraded":"2024-01-31T05:02:33.389576+0000","last_fullsized":"2024-01-31T05:02:33.389576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:50:54.221182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389372+0000","last_change":"2024-01-31T05:02:31.869112+0000","last_active":"2024-01-31T05:02:33.389372+0000","last_peered":"2024-01-31T05:02:33.389372+0000","last_clean":"2024-01-31T05:02:33.389372+0000","last_became_active":"2024-01-31T05:02:31.868821+0000","last_became_peered":"2024-01-31T05:02:31.868821+0000","last_unstale":"2024-01-31T05:02:33.389372+0000","last_undegraded":"2024-01-31T05:02:33.389372+0000","last_fullsized":"2024-01-31T05:02:33.389372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:25:06.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850571+0000","last_change":"2024-01-31T05:02:31.267746+0000","last_active":"2024-01-31T05:02:32.850571+0000","last_peered":"2024-01-31T05:02:32.850571+0000","last_clean":"2024-01-31T05:02:32.850571+0000","last_became_active":"2024-01-31T05:02:31.267092+0000","last_became_peered":"2024-01-31T05:02:31.267092+0000","last_unstale":"2024-01-31T05:02:32.850571+0000","last_undegraded":"2024-01-31T05:02:32.850571+0000","last_fullsized":"2024-01-31T05:02:32.850571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:41:31.026855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850731+0000","last_change":"2024-01-31T05:02:31.868631+0000","last_active":"2024-01-31T05:02:32.850731+0000","last_peered":"2024-01-31T05:02:32.850731+0000","last_clean":"2024-01-31T05:02:32.850731+0000","last_became_active":"2024-01-31T05:02:31.868327+0000","last_became_peered":"2024-01-31T05:02:31.868327+0000","last_unstale":"2024-01-31T05:02:32.850731+0000","last_undegraded":"2024-01-31T05:02:32.850731+0000","last_fullsized":"2024-01-31T05:02:32.850731+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:22:17.142232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393597+0000","last_change":"2024-01-31T05:02:31.257680+0000","last_active":"2024-01-31T05:02:33.393597+0000","last_peered":"2024-01-31T05:02:33.393597+0000","last_clean":"2024-01-31T05:02:33.393597+0000","last_became_active":"2024-01-31T05:02:31.257410+0000","last_became_peered":"2024-01-31T05:02:31.257410+0000","last_unstale":"2024-01-31T05:02:33.393597+0000","last_undegraded":"2024-01-31T05:02:33.393597+0000","last_fullsized":"2024-01-31T05:02:33.393597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:46.819224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848040+0000","last_change":"2024-01-31T05:02:31.873278+0000","last_active":"2024-01-31T05:02:32.848040+0000","last_peered":"2024-01-31T05:02:32.848040+0000","last_clean":"2024-01-31T05:02:32.848040+0000","last_became_active":"2024-01-31T05:02:31.873035+0000","last_became_peered":"2024-01-31T05:02:31.873035+0000","last_unstale":"2024-01-31T05:02:32.848040+0000","last_undegraded":"2024-01-31T05:02:32.848040+0000","last_fullsized":"2024-01-31T05:02:32.848040+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:32:37.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847786+0000","last_change":"2024-01-31T05:02:31.263208+0000","last_active":"2024-01-31T05:02:32.847786+0000","last_peered":"2024-01-31T05:02:32.847786+0000","last_clean":"2024-01-31T05:02:32.847786+0000","last_became_active":"2024-01-31T05:02:31.262273+0000","last_became_peered":"2024-01-31T05:02:31.262273+0000","last_unstale":"2024-01-31T05:02:32.847786+0000","last_undegraded":"2024-01-31T05:02:32.847786+0000","last_fullsized":"2024-01-31T05:02:32.847786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:51:42.945351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393451+0000","last_change":"2024-01-31T05:02:31.873839+0000","last_active":"2024-01-31T05:02:33.393451+0000","last_peered":"2024-01-31T05:02:33.393451+0000","last_clean":"2024-01-31T05:02:33.393451+0000","last_became_active":"2024-01-31T05:02:31.873651+0000","last_became_peered":"2024-01-31T05:02:31.873651+0000","last_unstale":"2024-01-31T05:02:33.393451+0000","last_undegraded":"2024-01-31T05:02:33.393451+0000","last_fullsized":"2024-01-31T05:02:33.393451+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:09:25.550905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850968+0000","last_change":"2024-01-31T05:02:31.264960+0000","last_active":"2024-01-31T05:02:32.850968+0000","last_peered":"2024-01-31T05:02:32.850968+0000","last_clean":"2024-01-31T05:02:32.850968+0000","last_became_active":"2024-01-31T05:02:31.264475+0000","last_became_peered":"2024-01-31T05:02:31.264475+0000","last_unstale":"2024-01-31T05:02:32.850968+0000","last_undegraded":"2024-01-31T05:02:32.850968+0000","last_fullsized":"2024-01-31T05:02:32.850968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:48.566783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848634+0000","last_change":"2024-01-31T05:02:31.884529+0000","last_active":"2024-01-31T05:02:32.848634+0000","last_peered":"2024-01-31T05:02:32.848634+0000","last_clean":"2024-01-31T05:02:32.848634+0000","last_became_active":"2024-01-31T05:02:31.884245+0000","last_became_peered":"2024-01-31T05:02:31.884245+0000","last_unstale":"2024-01-31T05:02:32.848634+0000","last_undegraded":"2024-01-31T05:02:32.848634+0000","last_fullsized":"2024-01-31T05:02:32.848634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:15:35.685367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393740+0000","last_change":"2024-01-31T05:02:31.258710+0000","last_active":"2024-01-31T05:02:33.393740+0000","last_peered":"2024-01-31T05:02:33.393740+0000","last_clean":"2024-01-31T05:02:33.393740+0000","last_became_active":"2024-01-31T05:02:31.258379+0000","last_became_peered":"2024-01-31T05:02:31.258379+0000","last_unstale":"2024-01-31T05:02:33.393740+0000","last_undegraded":"2024-01-31T05:02:33.393740+0000","last_fullsized":"2024-01-31T05:02:33.393740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:13:36.775475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":203,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637018+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:02:33.637018+0000","last_peered":"2024-01-31T05:02:33.637018+0000","last_clean":"2024-01-31T05:02:33.637018+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:02:33.637018+0000","last_undegraded":"2024-01-31T05:02:33.637018+0000","last_fullsized":"2024-01-31T05:02:33.637018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.538},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.397},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.494}]},{"osd":3,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.304}]},{"osd":4,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.609},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.5669999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.1859999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.5750000000000002,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":1.3560000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":195813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.147},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.1919999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.2270000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.403},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.496}]},{"osd":5,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.878},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.5790000000000002,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.0780000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.552},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.369}]},{"osd":7,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.6000000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.8529999999999998,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.5350000000000001,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.73099999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":1.486},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.84599999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":5.6760000000000002,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.1120000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.774,"15min":1.774},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":8.2850000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.9990000000000001},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":6.2949999999999999,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.696}]},{"osd":5,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":6.048,"5min":6.048,"15min":6.048},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.875,"5min":1.875,"15min":1.875},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":7.3789999999999996,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.341}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":5.0490000000000004,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.5449999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":1.784,"15min":1.784},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":4.9050000000000002,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":1.802},{"interface":"front","average":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.6080000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":196271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.361},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.6200000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.472},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.671}]},{"osd":3,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.502}]},{"osd":4,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":2.0059999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":1.8220000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":2.1320000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.5629999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.869}]}]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1052672,"data_stored":784467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.115}]},{"osd":2,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.046}]},{"osd":3,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.294}]},{"osd":4,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.248}]},{"osd":5,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.387},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.204}]},{"osd":6,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.5029999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.361},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.413}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.425}]},{"osd":2,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.216}]},{"osd":3,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.339}]},{"osd":4,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.135},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.583}]},{"osd":7,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.546},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6819999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1052672,"data_stored":785027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.462},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.395}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.272}]},{"osd":3,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.216},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.429}]},{"osd":4,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.0780000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.3560000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.3120000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":187549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.282,"5min":3.282,"15min":3.282},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.101},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.2050000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:01:38 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.4730000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.423},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.639},{"interface":"front","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":1.456}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:02:36.679 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:02:36.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.926+0000 7f7781a00640 1 Processor -- start 2024-01-31T05:02:36.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.926+0000 7f7781a00640 1 -- start start 2024-01-31T05:02:36.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f7781a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 0x7f777c0d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f7781a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c06b570 con 0x7f777c160e00 2024-01-31T05:02:36.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f7781a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c0cb6c0 con 0x7f777c0ce000 2024-01-31T05:02:36.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f7781a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c1611e0 con 0x7f777c0b2330 2024-01-31T05:02:36.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f777aa00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f777c160e00 0x7f777c0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58456/0 (socket says 172.21.15.97:58456) 2024-01-31T05:02:36.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f777aa00640 1 -- 172.21.15.97:0/14927368 learned_addr learned my addr 172.21.15.97:0/14927368 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.928+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101196352 0 0) 0x7f777c06b570 con 0x7f777c160e00 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777b400640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 0x7f777c0d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f775c003520 con 0x7f777c160e00 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410378804 0 0) 0x7f775c003520 con 0x7f777c160e00 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 msgr2=0x7f777c0d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 0x7f777c0d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 >> v1:172.21.15.188:6789/0 conn(0x7f777c0ce000 legacy=0x7f777c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0cfff0 con 0x7f777c160e00 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f776c001670 con 0x7f777c160e00 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 >> v1:172.21.15.97:6789/0 conn(0x7f777c160e00 legacy=0x7f777c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 0x7f775c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c003f70 0x7f775c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c004d00 0x7f775c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c1611e0 con 0x7f775c003f70 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c00bac0 con 0x7f775c004d00 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c00bc80 con 0x7f777c0b2330 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.929+0000 7f7773e00640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c003f70 0x7f775c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.930+0000 7f7773e00640 1 -- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 msgr2=0x7f775c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.930+0000 7f7773e00640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f777c0b2330 0x7f775c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.930+0000 7f7773e00640 1 -- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c004d00 msgr2=0x7f775c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.930+0000 7f7773e00640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c004d00 0x7f775c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.930+0000 7f7773e00640 1 -- 172.21.15.97:0/14927368 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c003520 con 0x7f775c003f70 2024-01-31T05:02:36.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.931+0000 7f7773e00640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c003f70 0x7f775c0047c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7764001750 tx=0x7f775c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e437a96c73cd3b18 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.931+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77640052d0 con 0x7f775c003f70 2024-01-31T05:02:36.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.931+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776402e050 con 0x7f775c003f70 2024-01-31T05:02:36.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.931+0000 7f777a000640 1 -- 172.21.15.97:0/14927368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764003830 con 0x7f775c003f70 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f7781a00640 1 -- 172.21.15.97:0/14927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c003f70 msgr2=0x7f775c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f7781a00640 1 --2- 172.21.15.97:0/14927368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f775c003f70 0x7f775c0047c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7764001750 tx=0x7f775c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f777b400640 1 -- 172.21.15.97:0/14927368 reap_dead start 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f7781a00640 1 -- 172.21.15.97:0/14927368 shutdown_connections 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f7781a00640 1 -- 172.21.15.97:0/14927368 >> 172.21.15.97:0/14927368 conn(0x7f777c0d1070 msgr2=0x7f777c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f7781a00640 1 -- 172.21.15.97:0/14927368 shutdown_connections 2024-01-31T05:02:36.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.932+0000 7f7781a00640 1 -- 172.21.15.97:0/14927368 wait complete. 2024-01-31T05:02:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.933+0000 7f7781a00640 1 Processor -- start 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 -- start start 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f775c003f70 0x7f777c1f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f777c15c890 0x7f777c0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 0x7f777c0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c06fb30 con 0x7f777c15cd10 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c1f2e00 con 0x7f777c15c890 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.934+0000 7f7781a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0c5d40 con 0x7f775c003f70 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 0x7f777c0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 0x7f777c0c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54396/0 (socket says 172.21.15.97:54396) 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 -- 172.21.15.97:0/4181920830 learned_addr learned my addr 172.21.15.97:0/4181920830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 -- 172.21.15.97:0/4181920830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f775c003f70 msgr2=0x7f777c1f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f775c003f70 0x7f777c1f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 -- 172.21.15.97:0/4181920830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f777c15c890 msgr2=0x7f777c0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f777c15c890 0x7f777c0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 -- 172.21.15.97:0/4181920830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c0c5130 con 0x7f777c15cd10 2024-01-31T05:02:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.935+0000 7f777b400640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 0x7f777c0c7ae0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f777c071c40 tx=0x7f777400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.936+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774010040 con 0x7f777c15cd10 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.936+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777400bd10 con 0x7f777c15cd10 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.936+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774014640 con 0x7f777c15cd10 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.936+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c0c5410 con 0x7f777c15cd10 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.936+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c0c59d0 con 0x7f777c15cd10 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.937+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c0677c0 con 0x7f777c15cd10 2024-01-31T05:02:36.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.938+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f77740147a0 con 0x7f777c15cd10 2024-01-31T05:02:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.939+0000 7f7772a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f775406af00 0x7f775406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.939+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7774089e00 con 0x7f777c15cd10 2024-01-31T05:02:36.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.939+0000 7f777aa00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f775406af00 0x7f775406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:36.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.940+0000 7f777aa00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f775406af00 0x7f775406d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f776c0078d0 tx=0x7f776c001590 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:36.944+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7774054c20 con 0x7f777c15cd10 2024-01-31T05:02:37.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.132+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f777c0c5eb0 con 0x7f775406af00 2024-01-31T05:02:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.154+0000 7f7772a00640 1 -- 172.21.15.97:0/4181920830 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287234 (secure 0 0 0) 0x7f777c0c5eb0 con 0x7f775406af00 2024-01-31T05:02:37.157 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:37.159 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f775406af00 msgr2=0x7f775406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f775406af00 0x7f775406d3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f776c0078d0 tx=0x7f776c001590 comp rx=0 tx=0).stop 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 msgr2=0x7f777c0c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 0x7f777c0c7ae0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f777c071c40 tx=0x7f777400b480 comp rx=0 tx=0).stop 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 shutdown_connections 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f775406af00 0x7f775406d3c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f777c15cd10 0x7f777c0c7ae0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f777c15c890 0x7f777c0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 --2- 172.21.15.97:0/4181920830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f775c003f70 0x7f777c1f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 >> 172.21.15.97:0/4181920830 conn(0x7f777c0d1070 msgr2=0x7f777c0ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.161+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 shutdown_connections 2024-01-31T05:02:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.162+0000 7f7781a00640 1 -- 172.21.15.97:0/4181920830 wait complete. 2024-01-31T05:02:37.180 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-01-31T05:02:35.587557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220936,"kb_used_data":4764,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753784,"statfs":{"total":773094113280,"available":772867874816,"internally_reserved":0,"allocated":4878336,"data_stored":2731552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.365490"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852607+0000","last_change":"2024-01-31T05:02:31.862463+0000","last_active":"2024-01-31T05:02:32.852607+0000","last_peered":"2024-01-31T05:02:32.852607+0000","last_clean":"2024-01-31T05:02:32.852607+0000","last_became_active":"2024-01-31T05:02:31.862126+0000","last_became_peered":"2024-01-31T05:02:31.862126+0000","last_unstale":"2024-01-31T05:02:32.852607+0000","last_undegraded":"2024-01-31T05:02:32.852607+0000","last_fullsized":"2024-01-31T05:02:32.852607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:02:03.888729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850900+0000","last_change":"2024-01-31T05:02:31.875565+0000","last_active":"2024-01-31T05:02:32.850900+0000","last_peered":"2024-01-31T05:02:32.850900+0000","last_clean":"2024-01-31T05:02:32.850900+0000","last_became_active":"2024-01-31T05:02:31.874063+0000","last_became_peered":"2024-01-31T05:02:31.874063+0000","last_unstale":"2024-01-31T05:02:32.850900+0000","last_undegraded":"2024-01-31T05:02:32.850900+0000","last_fullsized":"2024-01-31T05:02:32.850900+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:57.318680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852013+0000","last_change":"2024-01-31T05:02:31.877123+0000","last_active":"2024-01-31T05:02:32.852013+0000","last_peered":"2024-01-31T05:02:32.852013+0000","last_clean":"2024-01-31T05:02:32.852013+0000","last_became_active":"2024-01-31T05:02:31.876844+0000","last_became_peered":"2024-01-31T05:02:31.876844+0000","last_unstale":"2024-01-31T05:02:32.852013+0000","last_undegraded":"2024-01-31T05:02:32.852013+0000","last_fullsized":"2024-01-31T05:02:32.852013+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:03.071117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851857+0000","last_change":"2024-01-31T05:02:31.878009+0000","last_active":"2024-01-31T05:02:32.851857+0000","last_peered":"2024-01-31T05:02:32.851857+0000","last_clean":"2024-01-31T05:02:32.851857+0000","last_became_active":"2024-01-31T05:02:31.877742+0000","last_became_peered":"2024-01-31T05:02:31.877742+0000","last_unstale":"2024-01-31T05:02:32.851857+0000","last_undegraded":"2024-01-31T05:02:32.851857+0000","last_fullsized":"2024-01-31T05:02:32.851857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:06:07.665433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389637+0000","last_change":"2024-01-31T05:02:31.876537+0000","last_active":"2024-01-31T05:02:33.389637+0000","last_peered":"2024-01-31T05:02:33.389637+0000","last_clean":"2024-01-31T05:02:33.389637+0000","last_became_active":"2024-01-31T05:02:31.876245+0000","last_became_peered":"2024-01-31T05:02:31.876245+0000","last_unstale":"2024-01-31T05:02:33.389637+0000","last_undegraded":"2024-01-31T05:02:33.389637+0000","last_fullsized":"2024-01-31T05:02:33.389637+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:42.190280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850610+0000","last_change":"2024-01-31T05:02:31.869363+0000","last_active":"2024-01-31T05:02:32.850610+0000","last_peered":"2024-01-31T05:02:32.850610+0000","last_clean":"2024-01-31T05:02:32.850610+0000","last_became_active":"2024-01-31T05:02:31.869077+0000","last_became_peered":"2024-01-31T05:02:31.869077+0000","last_unstale":"2024-01-31T05:02:32.850610+0000","last_undegraded":"2024-01-31T05:02:32.850610+0000","last_fullsized":"2024-01-31T05:02:32.850610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:27:51.934492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636751+0000","last_change":"2024-01-31T05:02:31.866927+0000","last_active":"2024-01-31T05:02:33.636751+0000","last_peered":"2024-01-31T05:02:33.636751+0000","last_clean":"2024-01-31T05:02:33.636751+0000","last_became_active":"2024-01-31T05:02:31.866385+0000","last_became_peered":"2024-01-31T05:02:31.866385+0000","last_unstale":"2024-01-31T05:02:33.636751+0000","last_undegraded":"2024-01-31T05:02:33.636751+0000","last_fullsized":"2024-01-31T05:02:33.636751+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:51:11.026998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849844+0000","last_change":"2024-01-31T05:02:31.871523+0000","last_active":"2024-01-31T05:02:32.849844+0000","last_peered":"2024-01-31T05:02:32.849844+0000","last_clean":"2024-01-31T05:02:32.849844+0000","last_became_active":"2024-01-31T05:02:31.870534+0000","last_became_peered":"2024-01-31T05:02:31.870534+0000","last_unstale":"2024-01-31T05:02:32.849844+0000","last_undegraded":"2024-01-31T05:02:32.849844+0000","last_fullsized":"2024-01-31T05:02:32.849844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:18:30.141878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389977+0000","last_change":"2024-01-31T05:02:31.877818+0000","last_active":"2024-01-31T05:02:33.389977+0000","last_peered":"2024-01-31T05:02:33.389977+0000","last_clean":"2024-01-31T05:02:33.389977+0000","last_became_active":"2024-01-31T05:02:31.877562+0000","last_became_peered":"2024-01-31T05:02:31.877562+0000","last_unstale":"2024-01-31T05:02:33.389977+0000","last_undegraded":"2024-01-31T05:02:33.389977+0000","last_fullsized":"2024-01-31T05:02:33.389977+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:51:11.026998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637417+0000","last_change":"2024-01-31T05:02:31.881948+0000","last_active":"2024-01-31T05:02:33.637417+0000","last_peered":"2024-01-31T05:02:33.637417+0000","last_clean":"2024-01-31T05:02:33.637417+0000","last_became_active":"2024-01-31T05:02:31.881230+0000","last_became_peered":"2024-01-31T05:02:31.881230+0000","last_unstale":"2024-01-31T05:02:33.637417+0000","last_undegraded":"2024-01-31T05:02:33.637417+0000","last_fullsized":"2024-01-31T05:02:33.637417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:35:16.668974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393746+0000","last_change":"2024-01-31T05:02:31.875423+0000","last_active":"2024-01-31T05:02:33.393746+0000","last_peered":"2024-01-31T05:02:33.393746+0000","last_clean":"2024-01-31T05:02:33.393746+0000","last_became_active":"2024-01-31T05:02:31.875161+0000","last_became_peered":"2024-01-31T05:02:31.875161+0000","last_unstale":"2024-01-31T05:02:33.393746+0000","last_undegraded":"2024-01-31T05:02:33.393746+0000","last_fullsized":"2024-01-31T05:02:33.393746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:43:54.231166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389707+0000","last_change":"2024-01-31T05:02:31.880363+0000","last_active":"2024-01-31T05:02:33.389707+0000","last_peered":"2024-01-31T05:02:33.389707+0000","last_clean":"2024-01-31T05:02:33.389707+0000","last_became_active":"2024-01-31T05:02:31.879182+0000","last_became_peered":"2024-01-31T05:02:31.879182+0000","last_unstale":"2024-01-31T05:02:33.389707+0000","last_undegraded":"2024-01-31T05:02:33.389707+0000","last_fullsized":"2024-01-31T05:02:33.389707+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:40:59.923350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637123+0000","last_change":"2024-01-31T05:02:31.880164+0000","last_active":"2024-01-31T05:02:33.637123+0000","last_peered":"2024-01-31T05:02:33.637123+0000","last_clean":"2024-01-31T05:02:33.637123+0000","last_became_active":"2024-01-31T05:02:31.879634+0000","last_became_peered":"2024-01-31T05:02:31.879634+0000","last_unstale":"2024-01-31T05:02:33.637123+0000","last_undegraded":"2024-01-31T05:02:33.637123+0000","last_fullsized":"2024-01-31T05:02:33.637123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:59:46.315012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633079+0000","last_change":"2024-01-31T05:02:31.868717+0000","last_active":"2024-01-31T05:02:33.633079+0000","last_peered":"2024-01-31T05:02:33.633079+0000","last_clean":"2024-01-31T05:02:33.633079+0000","last_became_active":"2024-01-31T05:02:31.868453+0000","last_became_peered":"2024-01-31T05:02:31.868453+0000","last_unstale":"2024-01-31T05:02:33.633079+0000","last_undegraded":"2024-01-31T05:02:33.633079+0000","last_fullsized":"2024-01-31T05:02:33.633079+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:41:32.038067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847730+0000","last_change":"2024-01-31T05:02:31.880181+0000","last_active":"2024-01-31T05:02:32.847730+0000","last_peered":"2024-01-31T05:02:32.847730+0000","last_clean":"2024-01-31T05:02:32.847730+0000","last_became_active":"2024-01-31T05:02:31.879926+0000","last_became_peered":"2024-01-31T05:02:31.879926+0000","last_unstale":"2024-01-31T05:02:32.847730+0000","last_undegraded":"2024-01-31T05:02:32.847730+0000","last_fullsized":"2024-01-31T05:02:32.847730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:46:26.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636711+0000","last_change":"2024-01-31T05:02:31.880158+0000","last_active":"2024-01-31T05:02:33.636711+0000","last_peered":"2024-01-31T05:02:33.636711+0000","last_clean":"2024-01-31T05:02:33.636711+0000","last_became_active":"2024-01-31T05:02:31.879895+0000","last_became_peered":"2024-01-31T05:02:31.879895+0000","last_unstale":"2024-01-31T05:02:33.636711+0000","last_undegraded":"2024-01-31T05:02:33.636711+0000","last_fullsized":"2024-01-31T05:02:33.636711+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:40:59.923350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848567+0000","last_change":"2024-01-31T05:02:31.882801+0000","last_active":"2024-01-31T05:02:32.848567+0000","last_peered":"2024-01-31T05:02:32.848567+0000","last_clean":"2024-01-31T05:02:32.848567+0000","last_became_active":"2024-01-31T05:02:31.881247+0000","last_became_peered":"2024-01-31T05:02:31.881247+0000","last_unstale":"2024-01-31T05:02:32.848567+0000","last_undegraded":"2024-01-31T05:02:32.848567+0000","last_fullsized":"2024-01-31T05:02:32.848567+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:02:26.441143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848175+0000","last_change":"2024-01-31T05:02:31.879308+0000","last_active":"2024-01-31T05:02:32.848175+0000","last_peered":"2024-01-31T05:02:32.848175+0000","last_clean":"2024-01-31T05:02:32.848175+0000","last_became_active":"2024-01-31T05:02:31.876184+0000","last_became_peered":"2024-01-31T05:02:31.876184+0000","last_unstale":"2024-01-31T05:02:32.848175+0000","last_undegraded":"2024-01-31T05:02:32.848175+0000","last_fullsized":"2024-01-31T05:02:32.848175+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:34:42.139273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638959+0000","last_change":"2024-01-31T05:02:31.877503+0000","last_active":"2024-01-31T05:02:33.638959+0000","last_peered":"2024-01-31T05:02:33.638959+0000","last_clean":"2024-01-31T05:02:33.638959+0000","last_became_active":"2024-01-31T05:02:31.877230+0000","last_became_peered":"2024-01-31T05:02:31.877230+0000","last_unstale":"2024-01-31T05:02:33.638959+0000","last_undegraded":"2024-01-31T05:02:33.638959+0000","last_fullsized":"2024-01-31T05:02:33.638959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:22:17.142232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393423+0000","last_change":"2024-01-31T05:02:31.878823+0000","last_active":"2024-01-31T05:02:33.393423+0000","last_peered":"2024-01-31T05:02:33.393423+0000","last_clean":"2024-01-31T05:02:33.393423+0000","last_became_active":"2024-01-31T05:02:31.878556+0000","last_became_peered":"2024-01-31T05:02:31.878556+0000","last_unstale":"2024-01-31T05:02:33.393423+0000","last_undegraded":"2024-01-31T05:02:33.393423+0000","last_fullsized":"2024-01-31T05:02:33.393423+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:43:21.077786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393366+0000","last_change":"2024-01-31T05:02:31.879043+0000","last_active":"2024-01-31T05:02:33.393366+0000","last_peered":"2024-01-31T05:02:33.393366+0000","last_clean":"2024-01-31T05:02:33.393366+0000","last_became_active":"2024-01-31T05:02:31.878786+0000","last_became_peered":"2024-01-31T05:02:31.878786+0000","last_unstale":"2024-01-31T05:02:33.393366+0000","last_undegraded":"2024-01-31T05:02:33.393366+0000","last_fullsized":"2024-01-31T05:02:33.393366+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:50:09.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393489+0000","last_change":"2024-01-31T05:02:31.875472+0000","last_active":"2024-01-31T05:02:33.393489+0000","last_peered":"2024-01-31T05:02:33.393489+0000","last_clean":"2024-01-31T05:02:33.393489+0000","last_became_active":"2024-01-31T05:02:31.874699+0000","last_became_peered":"2024-01-31T05:02:31.874699+0000","last_unstale":"2024-01-31T05:02:33.393489+0000","last_undegraded":"2024-01-31T05:02:33.393489+0000","last_fullsized":"2024-01-31T05:02:33.393489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:02:27.914422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847560+0000","last_change":"2024-01-31T05:02:31.883795+0000","last_active":"2024-01-31T05:02:32.847560+0000","last_peered":"2024-01-31T05:02:32.847560+0000","last_clean":"2024-01-31T05:02:32.847560+0000","last_became_active":"2024-01-31T05:02:31.882949+0000","last_became_peered":"2024-01-31T05:02:31.882949+0000","last_unstale":"2024-01-31T05:02:32.847560+0000","last_undegraded":"2024-01-31T05:02:32.847560+0000","last_fullsized":"2024-01-31T05:02:32.847560+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:09:53.884970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849887+0000","last_change":"2024-01-31T05:02:31.867632+0000","last_active":"2024-01-31T05:02:32.849887+0000","last_peered":"2024-01-31T05:02:32.849887+0000","last_clean":"2024-01-31T05:02:32.849887+0000","last_became_active":"2024-01-31T05:02:31.867025+0000","last_became_peered":"2024-01-31T05:02:31.867025+0000","last_unstale":"2024-01-31T05:02:32.849887+0000","last_undegraded":"2024-01-31T05:02:32.849887+0000","last_fullsized":"2024-01-31T05:02:32.849887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:47:19.273646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.634090+0000","last_change":"2024-01-31T05:02:31.877765+0000","last_active":"2024-01-31T05:02:33.634090+0000","last_peered":"2024-01-31T05:02:33.634090+0000","last_clean":"2024-01-31T05:02:33.634090+0000","last_became_active":"2024-01-31T05:02:31.877523+0000","last_became_peered":"2024-01-31T05:02:31.877523+0000","last_unstale":"2024-01-31T05:02:33.634090+0000","last_undegraded":"2024-01-31T05:02:33.634090+0000","last_fullsized":"2024-01-31T05:02:33.634090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:49.577996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851526+0000","last_change":"2024-01-31T05:02:31.872149+0000","last_active":"2024-01-31T05:02:32.851526+0000","last_peered":"2024-01-31T05:02:32.851526+0000","last_clean":"2024-01-31T05:02:32.851526+0000","last_became_active":"2024-01-31T05:02:31.870394+0000","last_became_peered":"2024-01-31T05:02:31.870394+0000","last_unstale":"2024-01-31T05:02:32.851526+0000","last_undegraded":"2024-01-31T05:02:32.851526+0000","last_fullsized":"2024-01-31T05:02:32.851526+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:40:23.763126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393876+0000","last_change":"2024-01-31T05:02:31.876551+0000","last_active":"2024-01-31T05:02:33.393876+0000","last_peered":"2024-01-31T05:02:33.393876+0000","last_clean":"2024-01-31T05:02:33.393876+0000","last_became_active":"2024-01-31T05:02:31.876291+0000","last_became_peered":"2024-01-31T05:02:31.876291+0000","last_unstale":"2024-01-31T05:02:33.393876+0000","last_undegraded":"2024-01-31T05:02:33.393876+0000","last_fullsized":"2024-01-31T05:02:33.393876+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:21:03.928004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637300+0000","last_change":"2024-01-31T05:02:31.864617+0000","last_active":"2024-01-31T05:02:33.637300+0000","last_peered":"2024-01-31T05:02:33.637300+0000","last_clean":"2024-01-31T05:02:33.637300+0000","last_became_active":"2024-01-31T05:02:31.863915+0000","last_became_peered":"2024-01-31T05:02:31.863915+0000","last_unstale":"2024-01-31T05:02:33.637300+0000","last_undegraded":"2024-01-31T05:02:33.637300+0000","last_fullsized":"2024-01-31T05:02:33.637300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:41:32.038067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847860+0000","last_change":"2024-01-31T05:02:31.877672+0000","last_active":"2024-01-31T05:02:32.847860+0000","last_peered":"2024-01-31T05:02:32.847860+0000","last_clean":"2024-01-31T05:02:32.847860+0000","last_became_active":"2024-01-31T05:02:31.877435+0000","last_became_peered":"2024-01-31T05:02:31.877435+0000","last_unstale":"2024-01-31T05:02:32.847860+0000","last_undegraded":"2024-01-31T05:02:32.847860+0000","last_fullsized":"2024-01-31T05:02:32.847860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:12:31.764335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847801+0000","last_change":"2024-01-31T05:02:31.884527+0000","last_active":"2024-01-31T05:02:32.847801+0000","last_peered":"2024-01-31T05:02:32.847801+0000","last_clean":"2024-01-31T05:02:32.847801+0000","last_became_active":"2024-01-31T05:02:31.884279+0000","last_became_peered":"2024-01-31T05:02:31.884279+0000","last_unstale":"2024-01-31T05:02:32.847801+0000","last_undegraded":"2024-01-31T05:02:32.847801+0000","last_fullsized":"2024-01-31T05:02:32.847801+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:02:30.931525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636894+0000","last_change":"2024-01-31T05:02:31.869818+0000","last_active":"2024-01-31T05:02:33.636894+0000","last_peered":"2024-01-31T05:02:33.636894+0000","last_clean":"2024-01-31T05:02:33.636894+0000","last_became_active":"2024-01-31T05:02:31.869547+0000","last_became_peered":"2024-01-31T05:02:31.869547+0000","last_unstale":"2024-01-31T05:02:33.636894+0000","last_undegraded":"2024-01-31T05:02:33.636894+0000","last_fullsized":"2024-01-31T05:02:33.636894+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:34:23.316642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850578+0000","last_change":"2024-01-31T05:02:31.871216+0000","last_active":"2024-01-31T05:02:32.850578+0000","last_peered":"2024-01-31T05:02:32.850578+0000","last_clean":"2024-01-31T05:02:32.850578+0000","last_became_active":"2024-01-31T05:02:31.870159+0000","last_became_peered":"2024-01-31T05:02:31.870159+0000","last_unstale":"2024-01-31T05:02:32.850578+0000","last_undegraded":"2024-01-31T05:02:32.850578+0000","last_fullsized":"2024-01-31T05:02:32.850578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:15.569227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848805+0000","last_change":"2024-01-31T05:02:31.875682+0000","last_active":"2024-01-31T05:02:32.848805+0000","last_peered":"2024-01-31T05:02:32.848805+0000","last_clean":"2024-01-31T05:02:32.848805+0000","last_became_active":"2024-01-31T05:02:31.874983+0000","last_became_peered":"2024-01-31T05:02:31.874983+0000","last_unstale":"2024-01-31T05:02:32.848805+0000","last_undegraded":"2024-01-31T05:02:32.848805+0000","last_fullsized":"2024-01-31T05:02:32.848805+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:54:43.937881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.634357+0000","last_change":"2024-01-31T05:02:31.244919+0000","last_active":"2024-01-31T05:02:33.634357+0000","last_peered":"2024-01-31T05:02:33.634357+0000","last_clean":"2024-01-31T05:02:33.634357+0000","last_became_active":"2024-01-31T05:02:31.244423+0000","last_became_peered":"2024-01-31T05:02:31.244423+0000","last_unstale":"2024-01-31T05:02:33.634357+0000","last_undegraded":"2024-01-31T05:02:33.634357+0000","last_fullsized":"2024-01-31T05:02:33.634357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:51:10.015785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852780+0000","last_change":"2024-01-31T05:02:31.875103+0000","last_active":"2024-01-31T05:02:32.852780+0000","last_peered":"2024-01-31T05:02:32.852780+0000","last_clean":"2024-01-31T05:02:32.852780+0000","last_became_active":"2024-01-31T05:02:31.874793+0000","last_became_peered":"2024-01-31T05:02:31.874793+0000","last_unstale":"2024-01-31T05:02:32.852780+0000","last_undegraded":"2024-01-31T05:02:32.852780+0000","last_fullsized":"2024-01-31T05:02:32.852780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:50:09.846695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850846+0000","last_change":"2024-01-31T05:02:31.270188+0000","last_active":"2024-01-31T05:02:32.850846+0000","last_peered":"2024-01-31T05:02:32.850846+0000","last_clean":"2024-01-31T05:02:32.850846+0000","last_became_active":"2024-01-31T05:02:31.268820+0000","last_became_peered":"2024-01-31T05:02:31.268820+0000","last_unstale":"2024-01-31T05:02:32.850846+0000","last_undegraded":"2024-01-31T05:02:32.850846+0000","last_fullsized":"2024-01-31T05:02:32.850846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:48:10.110170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848484+0000","last_change":"2024-01-31T05:02:31.872070+0000","last_active":"2024-01-31T05:02:32.848484+0000","last_peered":"2024-01-31T05:02:32.848484+0000","last_clean":"2024-01-31T05:02:32.848484+0000","last_became_active":"2024-01-31T05:02:31.871758+0000","last_became_peered":"2024-01-31T05:02:31.871758+0000","last_unstale":"2024-01-31T05:02:32.848484+0000","last_undegraded":"2024-01-31T05:02:32.848484+0000","last_fullsized":"2024-01-31T05:02:32.848484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:38:45.016439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852310+0000","last_change":"2024-01-31T05:02:31.271167+0000","last_active":"2024-01-31T05:02:32.852310+0000","last_peered":"2024-01-31T05:02:32.852310+0000","last_clean":"2024-01-31T05:02:32.852310+0000","last_became_active":"2024-01-31T05:02:31.270868+0000","last_became_peered":"2024-01-31T05:02:31.270868+0000","last_unstale":"2024-01-31T05:02:32.852310+0000","last_undegraded":"2024-01-31T05:02:32.852310+0000","last_fullsized":"2024-01-31T05:02:32.852310+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:36:40.024536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851391+0000","last_change":"2024-01-31T05:02:31.873903+0000","last_active":"2024-01-31T05:02:32.851391+0000","last_peered":"2024-01-31T05:02:32.851391+0000","last_clean":"2024-01-31T05:02:32.851391+0000","last_became_active":"2024-01-31T05:02:31.873442+0000","last_became_peered":"2024-01-31T05:02:31.873442+0000","last_unstale":"2024-01-31T05:02:32.851391+0000","last_undegraded":"2024-01-31T05:02:32.851391+0000","last_fullsized":"2024-01-31T05:02:32.851391+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:32:04.118546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633544+0000","last_change":"2024-01-31T05:02:31.267286+0000","last_active":"2024-01-31T05:02:33.633544+0000","last_peered":"2024-01-31T05:02:33.633544+0000","last_clean":"2024-01-31T05:02:33.633544+0000","last_became_active":"2024-01-31T05:02:31.266815+0000","last_became_peered":"2024-01-31T05:02:31.266815+0000","last_unstale":"2024-01-31T05:02:33.633544+0000","last_undegraded":"2024-01-31T05:02:33.633544+0000","last_fullsized":"2024-01-31T05:02:33.633544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:35:44.023929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850284+0000","last_change":"2024-01-31T05:02:31.870847+0000","last_active":"2024-01-31T05:02:32.850284+0000","last_peered":"2024-01-31T05:02:32.850284+0000","last_clean":"2024-01-31T05:02:32.850284+0000","last_became_active":"2024-01-31T05:02:31.869744+0000","last_became_peered":"2024-01-31T05:02:31.869744+0000","last_unstale":"2024-01-31T05:02:32.850284+0000","last_undegraded":"2024-01-31T05:02:32.850284+0000","last_fullsized":"2024-01-31T05:02:32.850284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:51:43.956563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633556+0000","last_change":"2024-01-31T05:02:31.258203+0000","last_active":"2024-01-31T05:02:33.633556+0000","last_peered":"2024-01-31T05:02:33.633556+0000","last_clean":"2024-01-31T05:02:33.633556+0000","last_became_active":"2024-01-31T05:02:31.257613+0000","last_became_peered":"2024-01-31T05:02:31.257613+0000","last_unstale":"2024-01-31T05:02:33.633556+0000","last_undegraded":"2024-01-31T05:02:33.633556+0000","last_fullsized":"2024-01-31T05:02:33.633556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:41.179067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393285+0000","last_change":"2024-01-31T05:02:31.870332+0000","last_active":"2024-01-31T05:02:33.393285+0000","last_peered":"2024-01-31T05:02:33.393285+0000","last_clean":"2024-01-31T05:02:33.393285+0000","last_became_active":"2024-01-31T05:02:31.870067+0000","last_became_peered":"2024-01-31T05:02:31.870067+0000","last_unstale":"2024-01-31T05:02:33.393285+0000","last_undegraded":"2024-01-31T05:02:33.393285+0000","last_fullsized":"2024-01-31T05:02:33.393285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:13:22.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637017+0000","last_change":"2024-01-31T05:02:31.267811+0000","last_active":"2024-01-31T05:02:33.637017+0000","last_peered":"2024-01-31T05:02:33.637017+0000","last_clean":"2024-01-31T05:02:33.637017+0000","last_became_active":"2024-01-31T05:02:31.267372+0000","last_became_peered":"2024-01-31T05:02:31.267372+0000","last_unstale":"2024-01-31T05:02:33.637017+0000","last_undegraded":"2024-01-31T05:02:33.637017+0000","last_fullsized":"2024-01-31T05:02:33.637017+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:15:52.267028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848225+0000","last_change":"2024-01-31T05:02:31.881170+0000","last_active":"2024-01-31T05:02:32.848225+0000","last_peered":"2024-01-31T05:02:32.848225+0000","last_clean":"2024-01-31T05:02:32.848225+0000","last_became_active":"2024-01-31T05:02:31.880911+0000","last_became_peered":"2024-01-31T05:02:31.880911+0000","last_unstale":"2024-01-31T05:02:32.848225+0000","last_undegraded":"2024-01-31T05:02:32.848225+0000","last_fullsized":"2024-01-31T05:02:32.848225+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:53:22.934871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850161+0000","last_change":"2024-01-31T05:02:31.271499+0000","last_active":"2024-01-31T05:02:32.850161+0000","last_peered":"2024-01-31T05:02:32.850161+0000","last_clean":"2024-01-31T05:02:32.850161+0000","last_became_active":"2024-01-31T05:02:31.271079+0000","last_became_peered":"2024-01-31T05:02:31.271079+0000","last_unstale":"2024-01-31T05:02:32.850161+0000","last_undegraded":"2024-01-31T05:02:32.850161+0000","last_fullsized":"2024-01-31T05:02:32.850161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:09.596995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851657+0000","last_change":"2024-01-31T05:02:31.871548+0000","last_active":"2024-01-31T05:02:32.851657+0000","last_peered":"2024-01-31T05:02:32.851657+0000","last_clean":"2024-01-31T05:02:32.851657+0000","last_became_active":"2024-01-31T05:02:31.871251+0000","last_became_peered":"2024-01-31T05:02:31.871251+0000","last_unstale":"2024-01-31T05:02:32.851657+0000","last_undegraded":"2024-01-31T05:02:32.851657+0000","last_fullsized":"2024-01-31T05:02:32.851657+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:13:22.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.636940+0000","last_change":"2024-01-31T05:02:31.266460+0000","last_active":"2024-01-31T05:02:33.636940+0000","last_peered":"2024-01-31T05:02:33.636940+0000","last_clean":"2024-01-31T05:02:33.636940+0000","last_became_active":"2024-01-31T05:02:31.266223+0000","last_became_peered":"2024-01-31T05:02:31.266223+0000","last_unstale":"2024-01-31T05:02:33.636940+0000","last_undegraded":"2024-01-31T05:02:33.636940+0000","last_fullsized":"2024-01-31T05:02:33.636940+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:23:15.917063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852321+0000","last_change":"2024-01-31T05:02:31.877404+0000","last_active":"2024-01-31T05:02:32.852321+0000","last_peered":"2024-01-31T05:02:32.852321+0000","last_clean":"2024-01-31T05:02:32.852321+0000","last_became_active":"2024-01-31T05:02:31.877014+0000","last_became_peered":"2024-01-31T05:02:31.877014+0000","last_unstale":"2024-01-31T05:02:32.852321+0000","last_undegraded":"2024-01-31T05:02:32.852321+0000","last_fullsized":"2024-01-31T05:02:32.852321+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:52:53.957175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852709+0000","last_change":"2024-01-31T05:02:31.266140+0000","last_active":"2024-01-31T05:02:32.852709+0000","last_peered":"2024-01-31T05:02:32.852709+0000","last_clean":"2024-01-31T05:02:32.852709+0000","last_became_active":"2024-01-31T05:02:31.265668+0000","last_became_peered":"2024-01-31T05:02:31.265668+0000","last_unstale":"2024-01-31T05:02:32.852709+0000","last_undegraded":"2024-01-31T05:02:32.852709+0000","last_fullsized":"2024-01-31T05:02:32.852709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:33.507917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.633140+0000","last_change":"2024-01-31T05:02:31.873285+0000","last_active":"2024-01-31T05:02:33.633140+0000","last_peered":"2024-01-31T05:02:33.633140+0000","last_clean":"2024-01-31T05:02:33.633140+0000","last_became_active":"2024-01-31T05:02:31.872844+0000","last_became_peered":"2024-01-31T05:02:31.872844+0000","last_unstale":"2024-01-31T05:02:33.633140+0000","last_undegraded":"2024-01-31T05:02:33.633140+0000","last_fullsized":"2024-01-31T05:02:33.633140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:35:16.668974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848182+0000","last_change":"2024-01-31T05:02:31.261021+0000","last_active":"2024-01-31T05:02:32.848182+0000","last_peered":"2024-01-31T05:02:32.848182+0000","last_clean":"2024-01-31T05:02:32.848182+0000","last_became_active":"2024-01-31T05:02:31.260743+0000","last_became_peered":"2024-01-31T05:02:31.260743+0000","last_unstale":"2024-01-31T05:02:32.848182+0000","last_undegraded":"2024-01-31T05:02:32.848182+0000","last_fullsized":"2024-01-31T05:02:32.848182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:25:46.965349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852053+0000","last_change":"2024-01-31T05:02:31.872454+0000","last_active":"2024-01-31T05:02:32.852053+0000","last_peered":"2024-01-31T05:02:32.852053+0000","last_clean":"2024-01-31T05:02:32.852053+0000","last_became_active":"2024-01-31T05:02:31.871846+0000","last_became_peered":"2024-01-31T05:02:31.871846+0000","last_unstale":"2024-01-31T05:02:32.852053+0000","last_undegraded":"2024-01-31T05:02:32.852053+0000","last_fullsized":"2024-01-31T05:02:32.852053+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:03.150918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848419+0000","last_change":"2024-01-31T05:02:31.268384+0000","last_active":"2024-01-31T05:02:32.848419+0000","last_peered":"2024-01-31T05:02:32.848419+0000","last_clean":"2024-01-31T05:02:32.848419+0000","last_became_active":"2024-01-31T05:02:31.267973+0000","last_became_peered":"2024-01-31T05:02:31.267973+0000","last_unstale":"2024-01-31T05:02:32.848419+0000","last_undegraded":"2024-01-31T05:02:32.848419+0000","last_fullsized":"2024-01-31T05:02:32.848419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:14.558014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637143+0000","last_change":"2024-01-31T05:02:31.880907+0000","last_active":"2024-01-31T05:02:33.637143+0000","last_peered":"2024-01-31T05:02:33.637143+0000","last_clean":"2024-01-31T05:02:33.637143+0000","last_became_active":"2024-01-31T05:02:31.880640+0000","last_became_peered":"2024-01-31T05:02:31.880640+0000","last_unstale":"2024-01-31T05:02:33.637143+0000","last_undegraded":"2024-01-31T05:02:33.637143+0000","last_fullsized":"2024-01-31T05:02:33.637143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:10.608208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.852354+0000","last_change":"2024-01-31T05:02:31.269065+0000","last_active":"2024-01-31T05:02:32.852354+0000","last_peered":"2024-01-31T05:02:32.852354+0000","last_clean":"2024-01-31T05:02:32.852354+0000","last_became_active":"2024-01-31T05:02:31.268574+0000","last_became_peered":"2024-01-31T05:02:31.268574+0000","last_unstale":"2024-01-31T05:02:32.852354+0000","last_undegraded":"2024-01-31T05:02:32.852354+0000","last_fullsized":"2024-01-31T05:02:32.852354+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:35:15.657762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850707+0000","last_change":"2024-01-31T05:02:31.867347+0000","last_active":"2024-01-31T05:02:32.850707+0000","last_peered":"2024-01-31T05:02:32.850707+0000","last_clean":"2024-01-31T05:02:32.850707+0000","last_became_active":"2024-01-31T05:02:31.866500+0000","last_became_peered":"2024-01-31T05:02:31.866500+0000","last_unstale":"2024-01-31T05:02:32.850707+0000","last_undegraded":"2024-01-31T05:02:32.850707+0000","last_fullsized":"2024-01-31T05:02:32.850707+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:25:47.976561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850090+0000","last_change":"2024-01-31T05:02:31.267874+0000","last_active":"2024-01-31T05:02:32.850090+0000","last_peered":"2024-01-31T05:02:32.850090+0000","last_clean":"2024-01-31T05:02:32.850090+0000","last_became_active":"2024-01-31T05:02:31.267483+0000","last_became_peered":"2024-01-31T05:02:31.267483+0000","last_unstale":"2024-01-31T05:02:32.850090+0000","last_undegraded":"2024-01-31T05:02:32.850090+0000","last_fullsized":"2024-01-31T05:02:32.850090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:16.500985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850342+0000","last_change":"2024-01-31T05:02:31.875891+0000","last_active":"2024-01-31T05:02:32.850342+0000","last_peered":"2024-01-31T05:02:32.850342+0000","last_clean":"2024-01-31T05:02:32.850342+0000","last_became_active":"2024-01-31T05:02:31.875599+0000","last_became_peered":"2024-01-31T05:02:31.875599+0000","last_unstale":"2024-01-31T05:02:32.850342+0000","last_undegraded":"2024-01-31T05:02:32.850342+0000","last_fullsized":"2024-01-31T05:02:32.850342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:19:34.125226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850865+0000","last_change":"2024-01-31T05:02:31.269485+0000","last_active":"2024-01-31T05:02:32.850865+0000","last_peered":"2024-01-31T05:02:32.850865+0000","last_clean":"2024-01-31T05:02:32.850865+0000","last_became_active":"2024-01-31T05:02:31.268894+0000","last_became_peered":"2024-01-31T05:02:31.268894+0000","last_unstale":"2024-01-31T05:02:32.850865+0000","last_undegraded":"2024-01-31T05:02:32.850865+0000","last_fullsized":"2024-01-31T05:02:32.850865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:33:30.699042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393641+0000","last_change":"2024-01-31T05:02:31.873351+0000","last_active":"2024-01-31T05:02:33.393641+0000","last_peered":"2024-01-31T05:02:33.393641+0000","last_clean":"2024-01-31T05:02:33.393641+0000","last_became_active":"2024-01-31T05:02:31.873054+0000","last_became_peered":"2024-01-31T05:02:31.873054+0000","last_unstale":"2024-01-31T05:02:33.393641+0000","last_undegraded":"2024-01-31T05:02:33.393641+0000","last_fullsized":"2024-01-31T05:02:33.393641+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:25:21.456002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847678+0000","last_change":"2024-01-31T05:02:31.260162+0000","last_active":"2024-01-31T05:02:32.847678+0000","last_peered":"2024-01-31T05:02:32.847678+0000","last_clean":"2024-01-31T05:02:32.847678+0000","last_became_active":"2024-01-31T05:02:31.259179+0000","last_became_peered":"2024-01-31T05:02:31.259179+0000","last_unstale":"2024-01-31T05:02:32.847678+0000","last_undegraded":"2024-01-31T05:02:32.847678+0000","last_fullsized":"2024-01-31T05:02:32.847678+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:47:18.262434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389495+0000","last_change":"2024-01-31T05:02:31.883109+0000","last_active":"2024-01-31T05:02:33.389495+0000","last_peered":"2024-01-31T05:02:33.389495+0000","last_clean":"2024-01-31T05:02:33.389495+0000","last_became_active":"2024-01-31T05:02:31.882315+0000","last_became_peered":"2024-01-31T05:02:31.882315+0000","last_unstale":"2024-01-31T05:02:33.389495+0000","last_undegraded":"2024-01-31T05:02:33.389495+0000","last_fullsized":"2024-01-31T05:02:33.389495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:35:45.035142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.632836+0000","last_change":"2024-01-31T05:02:31.264411+0000","last_active":"2024-01-31T05:02:33.632836+0000","last_peered":"2024-01-31T05:02:33.632836+0000","last_clean":"2024-01-31T05:02:33.632836+0000","last_became_active":"2024-01-31T05:02:31.264119+0000","last_became_peered":"2024-01-31T05:02:31.264119+0000","last_unstale":"2024-01-31T05:02:33.632836+0000","last_undegraded":"2024-01-31T05:02:33.632836+0000","last_fullsized":"2024-01-31T05:02:33.632836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:40:58.912138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851351+0000","last_change":"2024-01-31T05:02:31.870998+0000","last_active":"2024-01-31T05:02:32.851351+0000","last_peered":"2024-01-31T05:02:32.851351+0000","last_clean":"2024-01-31T05:02:32.851351+0000","last_became_active":"2024-01-31T05:02:31.870686+0000","last_became_peered":"2024-01-31T05:02:31.870686+0000","last_unstale":"2024-01-31T05:02:32.851351+0000","last_undegraded":"2024-01-31T05:02:32.851351+0000","last_fullsized":"2024-01-31T05:02:32.851351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:03.633979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637242+0000","last_change":"2024-01-31T05:02:31.271009+0000","last_active":"2024-01-31T05:02:33.637242+0000","last_peered":"2024-01-31T05:02:33.637242+0000","last_clean":"2024-01-31T05:02:33.637242+0000","last_became_active":"2024-01-31T05:02:31.270689+0000","last_became_peered":"2024-01-31T05:02:31.270689+0000","last_unstale":"2024-01-31T05:02:33.637242+0000","last_undegraded":"2024-01-31T05:02:33.637242+0000","last_fullsized":"2024-01-31T05:02:33.637242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:50:54.221182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637228+0000","last_change":"2024-01-31T05:02:31.864362+0000","last_active":"2024-01-31T05:02:33.637228+0000","last_peered":"2024-01-31T05:02:33.637228+0000","last_clean":"2024-01-31T05:02:33.637228+0000","last_became_active":"2024-01-31T05:02:31.863638+0000","last_became_peered":"2024-01-31T05:02:31.863638+0000","last_unstale":"2024-01-31T05:02:33.637228+0000","last_undegraded":"2024-01-31T05:02:33.637228+0000","last_fullsized":"2024-01-31T05:02:33.637228+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:49:17.512197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848362+0000","last_change":"2024-01-31T05:02:31.266270+0000","last_active":"2024-01-31T05:02:32.848362+0000","last_peered":"2024-01-31T05:02:32.848362+0000","last_clean":"2024-01-31T05:02:32.848362+0000","last_became_active":"2024-01-31T05:02:31.265985+0000","last_became_peered":"2024-01-31T05:02:31.265985+0000","last_unstale":"2024-01-31T05:02:32.848362+0000","last_undegraded":"2024-01-31T05:02:32.848362+0000","last_fullsized":"2024-01-31T05:02:32.848362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:40:22.751913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637708+0000","last_change":"2024-01-31T05:02:31.876731+0000","last_active":"2024-01-31T05:02:33.637708+0000","last_peered":"2024-01-31T05:02:33.637708+0000","last_clean":"2024-01-31T05:02:33.637708+0000","last_became_active":"2024-01-31T05:02:31.875897+0000","last_became_peered":"2024-01-31T05:02:31.875897+0000","last_unstale":"2024-01-31T05:02:33.637708+0000","last_undegraded":"2024-01-31T05:02:33.637708+0000","last_fullsized":"2024-01-31T05:02:33.637708+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:49.577996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389562+0000","last_change":"2024-01-31T05:02:31.267514+0000","last_active":"2024-01-31T05:02:33.389562+0000","last_peered":"2024-01-31T05:02:33.389562+0000","last_clean":"2024-01-31T05:02:33.389562+0000","last_became_active":"2024-01-31T05:02:31.267102+0000","last_became_peered":"2024-01-31T05:02:31.267102+0000","last_unstale":"2024-01-31T05:02:33.389562+0000","last_undegraded":"2024-01-31T05:02:33.389562+0000","last_fullsized":"2024-01-31T05:02:33.389562+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:15:52.267028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.639136+0000","last_change":"2024-01-31T05:02:31.879023+0000","last_active":"2024-01-31T05:02:33.639136+0000","last_peered":"2024-01-31T05:02:33.639136+0000","last_clean":"2024-01-31T05:02:33.639136+0000","last_became_active":"2024-01-31T05:02:31.878756+0000","last_became_peered":"2024-01-31T05:02:31.878756+0000","last_unstale":"2024-01-31T05:02:33.639136+0000","last_undegraded":"2024-01-31T05:02:33.639136+0000","last_fullsized":"2024-01-31T05:02:33.639136+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:57.318680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638781+0000","last_change":"2024-01-31T05:02:31.270602+0000","last_active":"2024-01-31T05:02:33.638781+0000","last_peered":"2024-01-31T05:02:33.638781+0000","last_clean":"2024-01-31T05:02:33.638781+0000","last_became_active":"2024-01-31T05:02:31.269556+0000","last_became_peered":"2024-01-31T05:02:31.269556+0000","last_unstale":"2024-01-31T05:02:33.638781+0000","last_undegraded":"2024-01-31T05:02:33.638781+0000","last_fullsized":"2024-01-31T05:02:33.638781+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:51:47.047325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847470+0000","last_change":"2024-01-31T05:02:31.874593+0000","last_active":"2024-01-31T05:02:32.847470+0000","last_peered":"2024-01-31T05:02:32.847470+0000","last_clean":"2024-01-31T05:02:32.847470+0000","last_became_active":"2024-01-31T05:02:31.874300+0000","last_became_peered":"2024-01-31T05:02:31.874300+0000","last_unstale":"2024-01-31T05:02:32.847470+0000","last_undegraded":"2024-01-31T05:02:32.847470+0000","last_fullsized":"2024-01-31T05:02:32.847470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:18:43.735715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638696+0000","last_change":"2024-01-31T05:02:31.261011+0000","last_active":"2024-01-31T05:02:33.638696+0000","last_peered":"2024-01-31T05:02:33.638696+0000","last_clean":"2024-01-31T05:02:33.638696+0000","last_became_active":"2024-01-31T05:02:31.259901+0000","last_became_peered":"2024-01-31T05:02:31.259901+0000","last_unstale":"2024-01-31T05:02:33.638696+0000","last_undegraded":"2024-01-31T05:02:33.638696+0000","last_fullsized":"2024-01-31T05:02:33.638696+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:26:17.957679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849872+0000","last_change":"2024-01-31T05:02:31.864488+0000","last_active":"2024-01-31T05:02:32.849872+0000","last_peered":"2024-01-31T05:02:32.849872+0000","last_clean":"2024-01-31T05:02:32.849872+0000","last_became_active":"2024-01-31T05:02:31.863826+0000","last_became_peered":"2024-01-31T05:02:31.863826+0000","last_unstale":"2024-01-31T05:02:32.849872+0000","last_undegraded":"2024-01-31T05:02:32.849872+0000","last_fullsized":"2024-01-31T05:02:32.849872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:26:04.599402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.851111+0000","last_change":"2024-01-31T05:02:31.256596+0000","last_active":"2024-01-31T05:02:32.851111+0000","last_peered":"2024-01-31T05:02:32.851111+0000","last_clean":"2024-01-31T05:02:32.851111+0000","last_became_active":"2024-01-31T05:02:31.256269+0000","last_became_peered":"2024-01-31T05:02:31.256269+0000","last_unstale":"2024-01-31T05:02:32.851111+0000","last_undegraded":"2024-01-31T05:02:32.851111+0000","last_fullsized":"2024-01-31T05:02:32.851111+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:44:09.657004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393504+0000","last_change":"2024-01-31T05:02:31.875706+0000","last_active":"2024-01-31T05:02:33.393504+0000","last_peered":"2024-01-31T05:02:33.393504+0000","last_clean":"2024-01-31T05:02:33.393504+0000","last_became_active":"2024-01-31T05:02:31.875030+0000","last_became_peered":"2024-01-31T05:02:31.875030+0000","last_unstale":"2024-01-31T05:02:33.393504+0000","last_undegraded":"2024-01-31T05:02:33.393504+0000","last_fullsized":"2024-01-31T05:02:33.393504+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:43:28.000528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637075+0000","last_change":"2024-01-31T05:02:31.272108+0000","last_active":"2024-01-31T05:02:33.637075+0000","last_peered":"2024-01-31T05:02:33.637075+0000","last_clean":"2024-01-31T05:02:33.637075+0000","last_became_active":"2024-01-31T05:02:31.271812+0000","last_became_peered":"2024-01-31T05:02:31.271812+0000","last_unstale":"2024-01-31T05:02:33.637075+0000","last_undegraded":"2024-01-31T05:02:33.637075+0000","last_fullsized":"2024-01-31T05:02:33.637075+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:00:56.863174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849797+0000","last_change":"2024-01-31T05:02:31.867868+0000","last_active":"2024-01-31T05:02:32.849797+0000","last_peered":"2024-01-31T05:02:32.849797+0000","last_clean":"2024-01-31T05:02:32.849797+0000","last_became_active":"2024-01-31T05:02:31.867474+0000","last_became_peered":"2024-01-31T05:02:31.867474+0000","last_unstale":"2024-01-31T05:02:32.849797+0000","last_undegraded":"2024-01-31T05:02:32.849797+0000","last_fullsized":"2024-01-31T05:02:32.849797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:24:43.970952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.849515+0000","last_change":"2024-01-31T05:02:31.263062+0000","last_active":"2024-01-31T05:02:32.849515+0000","last_peered":"2024-01-31T05:02:32.849515+0000","last_clean":"2024-01-31T05:02:32.849515+0000","last_became_active":"2024-01-31T05:02:31.262782+0000","last_became_peered":"2024-01-31T05:02:31.262782+0000","last_unstale":"2024-01-31T05:02:32.849515+0000","last_undegraded":"2024-01-31T05:02:32.849515+0000","last_fullsized":"2024-01-31T05:02:32.849515+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.037570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.390102+0000","last_change":"2024-01-31T05:02:31.870719+0000","last_active":"2024-01-31T05:02:33.390102+0000","last_peered":"2024-01-31T05:02:33.390102+0000","last_clean":"2024-01-31T05:02:33.390102+0000","last_became_active":"2024-01-31T05:02:31.870400+0000","last_became_peered":"2024-01-31T05:02:31.870400+0000","last_unstale":"2024-01-31T05:02:33.390102+0000","last_undegraded":"2024-01-31T05:02:33.390102+0000","last_fullsized":"2024-01-31T05:02:33.390102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:34:23.316642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848657+0000","last_change":"2024-01-31T05:02:31.260406+0000","last_active":"2024-01-31T05:02:32.848657+0000","last_peered":"2024-01-31T05:02:32.848657+0000","last_clean":"2024-01-31T05:02:32.848657+0000","last_became_active":"2024-01-31T05:02:31.258453+0000","last_became_peered":"2024-01-31T05:02:31.258453+0000","last_unstale":"2024-01-31T05:02:32.848657+0000","last_undegraded":"2024-01-31T05:02:32.848657+0000","last_fullsized":"2024-01-31T05:02:32.848657+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:06:06.654220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393928+0000","last_change":"2024-01-31T05:02:31.873336+0000","last_active":"2024-01-31T05:02:33.393928+0000","last_peered":"2024-01-31T05:02:33.393928+0000","last_clean":"2024-01-31T05:02:33.393928+0000","last_became_active":"2024-01-31T05:02:31.873037+0000","last_became_peered":"2024-01-31T05:02:31.873037+0000","last_unstale":"2024-01-31T05:02:33.393928+0000","last_undegraded":"2024-01-31T05:02:33.393928+0000","last_fullsized":"2024-01-31T05:02:33.393928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:49:45.758767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.392862+0000","last_change":"2024-01-31T05:02:31.264450+0000","last_active":"2024-01-31T05:02:33.392862+0000","last_peered":"2024-01-31T05:02:33.392862+0000","last_clean":"2024-01-31T05:02:33.392862+0000","last_became_active":"2024-01-31T05:02:31.264030+0000","last_became_peered":"2024-01-31T05:02:31.264030+0000","last_unstale":"2024-01-31T05:02:33.392862+0000","last_undegraded":"2024-01-31T05:02:33.392862+0000","last_fullsized":"2024-01-31T05:02:33.392862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:23:15.917063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.638835+0000","last_change":"2024-01-31T05:02:31.874669+0000","last_active":"2024-01-31T05:02:33.638835+0000","last_peered":"2024-01-31T05:02:33.638835+0000","last_clean":"2024-01-31T05:02:33.638835+0000","last_became_active":"2024-01-31T05:02:31.874380+0000","last_became_peered":"2024-01-31T05:02:31.874380+0000","last_unstale":"2024-01-31T05:02:33.638835+0000","last_undegraded":"2024-01-31T05:02:33.638835+0000","last_fullsized":"2024-01-31T05:02:33.638835+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:27:51.934492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389576+0000","last_change":"2024-01-31T05:02:31.267507+0000","last_active":"2024-01-31T05:02:33.389576+0000","last_peered":"2024-01-31T05:02:33.389576+0000","last_clean":"2024-01-31T05:02:33.389576+0000","last_became_active":"2024-01-31T05:02:31.267138+0000","last_became_peered":"2024-01-31T05:02:31.267138+0000","last_unstale":"2024-01-31T05:02:33.389576+0000","last_undegraded":"2024-01-31T05:02:33.389576+0000","last_fullsized":"2024-01-31T05:02:33.389576+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:50:54.221182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.389372+0000","last_change":"2024-01-31T05:02:31.869112+0000","last_active":"2024-01-31T05:02:33.389372+0000","last_peered":"2024-01-31T05:02:33.389372+0000","last_clean":"2024-01-31T05:02:33.389372+0000","last_became_active":"2024-01-31T05:02:31.868821+0000","last_became_peered":"2024-01-31T05:02:31.868821+0000","last_unstale":"2024-01-31T05:02:33.389372+0000","last_undegraded":"2024-01-31T05:02:33.389372+0000","last_fullsized":"2024-01-31T05:02:33.389372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:25:06.523150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850571+0000","last_change":"2024-01-31T05:02:31.267746+0000","last_active":"2024-01-31T05:02:32.850571+0000","last_peered":"2024-01-31T05:02:32.850571+0000","last_clean":"2024-01-31T05:02:32.850571+0000","last_became_active":"2024-01-31T05:02:31.267092+0000","last_became_peered":"2024-01-31T05:02:31.267092+0000","last_unstale":"2024-01-31T05:02:32.850571+0000","last_undegraded":"2024-01-31T05:02:32.850571+0000","last_fullsized":"2024-01-31T05:02:32.850571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:41:31.026855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850731+0000","last_change":"2024-01-31T05:02:31.868631+0000","last_active":"2024-01-31T05:02:32.850731+0000","last_peered":"2024-01-31T05:02:32.850731+0000","last_clean":"2024-01-31T05:02:32.850731+0000","last_became_active":"2024-01-31T05:02:31.868327+0000","last_became_peered":"2024-01-31T05:02:31.868327+0000","last_unstale":"2024-01-31T05:02:32.850731+0000","last_undegraded":"2024-01-31T05:02:32.850731+0000","last_fullsized":"2024-01-31T05:02:32.850731+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:22:17.142232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393597+0000","last_change":"2024-01-31T05:02:31.257680+0000","last_active":"2024-01-31T05:02:33.393597+0000","last_peered":"2024-01-31T05:02:33.393597+0000","last_clean":"2024-01-31T05:02:33.393597+0000","last_became_active":"2024-01-31T05:02:31.257410+0000","last_became_peered":"2024-01-31T05:02:31.257410+0000","last_unstale":"2024-01-31T05:02:33.393597+0000","last_undegraded":"2024-01-31T05:02:33.393597+0000","last_fullsized":"2024-01-31T05:02:33.393597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:46.819224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848040+0000","last_change":"2024-01-31T05:02:31.873278+0000","last_active":"2024-01-31T05:02:32.848040+0000","last_peered":"2024-01-31T05:02:32.848040+0000","last_clean":"2024-01-31T05:02:32.848040+0000","last_became_active":"2024-01-31T05:02:31.873035+0000","last_became_peered":"2024-01-31T05:02:31.873035+0000","last_unstale":"2024-01-31T05:02:32.848040+0000","last_undegraded":"2024-01-31T05:02:32.848040+0000","last_fullsized":"2024-01-31T05:02:32.848040+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:32:37.752542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.847786+0000","last_change":"2024-01-31T05:02:31.263208+0000","last_active":"2024-01-31T05:02:32.847786+0000","last_peered":"2024-01-31T05:02:32.847786+0000","last_clean":"2024-01-31T05:02:32.847786+0000","last_became_active":"2024-01-31T05:02:31.262273+0000","last_became_peered":"2024-01-31T05:02:31.262273+0000","last_unstale":"2024-01-31T05:02:32.847786+0000","last_undegraded":"2024-01-31T05:02:32.847786+0000","last_fullsized":"2024-01-31T05:02:32.847786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:51:42.945351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393451+0000","last_change":"2024-01-31T05:02:31.873839+0000","last_active":"2024-01-31T05:02:33.393451+0000","last_peered":"2024-01-31T05:02:33.393451+0000","last_clean":"2024-01-31T05:02:33.393451+0000","last_became_active":"2024-01-31T05:02:31.873651+0000","last_became_peered":"2024-01-31T05:02:31.873651+0000","last_unstale":"2024-01-31T05:02:33.393451+0000","last_undegraded":"2024-01-31T05:02:33.393451+0000","last_fullsized":"2024-01-31T05:02:33.393451+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:09:25.550905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.850968+0000","last_change":"2024-01-31T05:02:31.264960+0000","last_active":"2024-01-31T05:02:32.850968+0000","last_peered":"2024-01-31T05:02:32.850968+0000","last_clean":"2024-01-31T05:02:32.850968+0000","last_became_active":"2024-01-31T05:02:31.264475+0000","last_became_peered":"2024-01-31T05:02:31.264475+0000","last_unstale":"2024-01-31T05:02:32.850968+0000","last_undegraded":"2024-01-31T05:02:32.850968+0000","last_fullsized":"2024-01-31T05:02:32.850968+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:42:48.566783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:32.848634+0000","last_change":"2024-01-31T05:02:31.884529+0000","last_active":"2024-01-31T05:02:32.848634+0000","last_peered":"2024-01-31T05:02:32.848634+0000","last_clean":"2024-01-31T05:02:32.848634+0000","last_became_active":"2024-01-31T05:02:31.884245+0000","last_became_peered":"2024-01-31T05:02:31.884245+0000","last_unstale":"2024-01-31T05:02:32.848634+0000","last_undegraded":"2024-01-31T05:02:32.848634+0000","last_fullsized":"2024-01-31T05:02:32.848634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:31.220435+0000","last_clean_scrub_stamp":"2024-01-31T05:02:31.220435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:15:35.685367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.393740+0000","last_change":"2024-01-31T05:02:31.258710+0000","last_active":"2024-01-31T05:02:33.393740+0000","last_peered":"2024-01-31T05:02:33.393740+0000","last_clean":"2024-01-31T05:02:33.393740+0000","last_became_active":"2024-01-31T05:02:31.258379+0000","last_became_peered":"2024-01-31T05:02:31.258379+0000","last_unstale":"2024-01-31T05:02:33.393740+0000","last_undegraded":"2024-01-31T05:02:33.393740+0000","last_fullsized":"2024-01-31T05:02:33.393740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:02:30.209223+0000","last_clean_scrub_stamp":"2024-01-31T05:02:30.209223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:13:36.775475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":203,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-01-31T05:02:33.637018+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:02:33.637018+0000","last_peered":"2024-01-31T05:02:33.637018+0000","last_clean":"2024-01-31T05:02:33.637018+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:02:33.637018+0000","last_undegraded":"2024-01-31T05:02:33.637018+0000","last_fullsized":"2024-01-31T05:02:33.637018+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.84599999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.538},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.397},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.494}]},{"osd":3,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.304}]},{"osd":4,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.351,"15min":1.351},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.609},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.5669999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.1859999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.5750000000000002,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.629,"5min":2.629,"15min":2.629},"last":1.3560000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":475136,"data_stored":195813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.147},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.1919999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.2789999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.2270000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.403,"15min":1.403},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.403},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.496}]},{"osd":5,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.878},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":2.5790000000000002,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.0780000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":1.552},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.369}]},{"osd":7,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.6000000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":4.8529999999999998,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.5350000000000001,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.73099999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":1.486},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.84599999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":5.6760000000000002,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":7.2850000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.1120000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.774,"5min":1.774,"15min":1.774},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":8.2850000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.9990000000000001},{"interface":"front","average":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":6.2949999999999999,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.696}]},{"osd":5,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":6.048,"5min":6.048,"15min":6.048},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.875,"5min":1.875,"15min":1.875},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":7.3789999999999996,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.341}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":5.0490000000000004,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.794,"5min":1.794,"15min":1.794},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":5.1669999999999998,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.5449999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.784,"5min":1.784,"15min":1.784},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":4.9050000000000002,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":1.802},{"interface":"front","average":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":7.7839999999999998,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.6080000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":196271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":1.361},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.6200000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.472},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.671}]},{"osd":3,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.502}]},{"osd":4,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":2.0059999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.9470000000000001,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":1.8220000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.421,"15min":1.421},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.992,"5min":2.992,"15min":2.992},"last":2.1320000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.5629999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.425,"5min":1.425,"15min":1.425},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.869}]}]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1052672,"data_stored":784467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.115}]},{"osd":2,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.046}]},{"osd":3,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.294}]},{"osd":4,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.5270000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.8929999999999998,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.248}]},{"osd":5,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":2.722,"5min":2.722,"15min":2.722},"last":1.387},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.204}]},{"osd":6,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.5029999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:39 2024","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.571,"15min":1.571},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.361},{"interface":"front","average":{"1min":1.641,"5min":1.641,"15min":1.641},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.413}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89500000000000002}]},{"osd":1,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.425}]},{"osd":2,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.216}]},{"osd":3,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.339}]},{"osd":4,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.135},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.0780000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.583}]},{"osd":7,"last update":"Wed Jan 31 05:01:44 2024","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.546},{"interface":"front","average":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6819999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1052672,"data_stored":785027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.462},{"interface":"front","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.395}]},{"osd":2,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.272}]},{"osd":3,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.216},{"interface":"front","average":{"1min":1.405,"5min":1.405,"15min":1.405},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.429}]},{"osd":4,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.2429999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.387,"5min":1.387,"15min":1.387},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":3.0960000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.0780000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.3560000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:01:41 2024","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.8370000000000002,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.3120000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":450560,"data_stored":187549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.282,"5min":3.282,"15min":3.282},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.9860000000000002,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.78700000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.262,"5min":3.262,"15min":3.262},"last":1.101},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":1.2050000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:01:38 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.3170000000000002,"5min":3.3170000000000002,"15min":3.3170000000000002},"last":1.4730000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:01:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.897,"5min":1.897,"15min":1.897},"last":1.423},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":3.5569999999999999,"5min":3.5569999999999999,"15min":3.5569999999999999},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:01:42 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.639},{"interface":"front","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":3.597,"5min":3.597,"15min":3.597},"last":1.456}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:02:37.184 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-31T05:02:37.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-31T05:02:37.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.441+0000 7f4d3d800640 1 Processor -- start 2024-01-31T05:02:37.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.442+0000 7f4d3d800640 1 -- start start 2024-01-31T05:02:37.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.443+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 0x7f4d380722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.443+0000 7f4d3d800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d380cbf20 0x7f4d380ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.443+0000 7f4d3d800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3806cd20 con 0x7f4d380abcf0 2024-01-31T05:02:37.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.443+0000 7f4d3d800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d380ce970 con 0x7f4d380cbf20 2024-01-31T05:02:37.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.443+0000 7f4d3d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815b850 con 0x7f4d3815b470 2024-01-31T05:02:37.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.444+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 0x7f4d380722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.444+0000 7f4d37400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d380cbf20 0x7f4d380ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.444+0000 7f4d37400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d380cbf20 0x7f4d380ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:56584/0 (socket says 172.21.15.97:56584) 2024-01-31T05:02:37.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.444+0000 7f4d36a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 0x7f4d380722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:55042/0 (socket says 172.21.15.97:55042) 2024-01-31T05:02:37.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.444+0000 7f4d37400640 1 -- 172.21.15.97:0/2978440070 learned_addr learned my addr 172.21.15.97:0/2978440070 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:37.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.445+0000 7f4d36a00640 1 -- 172.21.15.97:0/2978440070 >> v2:172.21.15.188:3300/0 conn(0x7f4d380cbf20 msgr2=0x7f4d380ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:37.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.445+0000 7f4d36a00640 1 --2- 172.21.15.97:0/2978440070 >> v2:172.21.15.188:3300/0 conn(0x7f4d380cbf20 0x7f4d380ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.445+0000 7f4d36a00640 1 -- 172.21.15.97:0/2978440070 >> v1:172.21.15.188:6789/0 conn(0x7f4d380abcf0 legacy=0x7f4d380cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.445+0000 7f4d36a00640 1 -- 172.21.15.97:0/2978440070 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380ceb60 con 0x7f4d3815b470 2024-01-31T05:02:37.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.447+0000 7f4d36a00640 1 --2- 172.21.15.97:0/2978440070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 0x7f4d380722c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e10 tx=0x7f4d2802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296019ac22eb1d75 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:37.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.447+0000 7f4d36000640 1 -- 172.21.15.97:0/2978440070 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2802cc80 con 0x7f4d3815b470 2024-01-31T05:02:37.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.447+0000 7f4d36000640 1 -- 172.21.15.97:0/2978440070 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2802cde0 con 0x7f4d3815b470 2024-01-31T05:02:37.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.447+0000 7f4d36000640 1 -- 172.21.15.97:0/2978440070 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28034660 con 0x7f4d3815b470 2024-01-31T05:02:37.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 -- 172.21.15.97:0/2978440070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 msgr2=0x7f4d380722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2978440070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 0x7f4d380722c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e10 tx=0x7f4d2802bed0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 -- 172.21.15.97:0/2978440070 shutdown_connections 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2978440070 >> v2:172.21.15.188:3300/0 conn(0x7f4d380cbf20 0x7f4d380ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2978440070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d3815b470 0x7f4d380722c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 -- 172.21.15.97:0/2978440070 >> 172.21.15.97:0/2978440070 conn(0x7f4d380b0a10 msgr2=0x7f4d380b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 -- 172.21.15.97:0/2978440070 shutdown_connections 2024-01-31T05:02:37.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.448+0000 7f4d3d800640 1 -- 172.21.15.97:0/2978440070 wait complete. 2024-01-31T05:02:37.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.451+0000 7f4d3d800640 1 Processor -- start 2024-01-31T05:02:37.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.451+0000 7f4d3d800640 1 -- start start 2024-01-31T05:02:37.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d380abcf0 0x7f4d3815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380cbf20 0x7f4d381f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.452 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d3d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 0x7f4d381f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d3d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815c430 con 0x7f4d3815b470 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d3d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815c710 con 0x7f4d380cbf20 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d3d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3815c9f0 con 0x7f4d380abcf0 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.452+0000 7f4d37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 0x7f4d381f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 0x7f4d381f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54412/0 (socket says 172.21.15.97:54412) 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 -- 172.21.15.97:0/2474916516 learned_addr learned my addr 172.21.15.97:0/2474916516 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d36a00640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d380abcf0 0x7f4d3815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d2de00640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380cbf20 0x7f4d381f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 -- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d380abcf0 msgr2=0x7f4d3815b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d380abcf0 0x7f4d3815b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 -- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380cbf20 msgr2=0x7f4d381f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380cbf20 0x7f4d381f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 -- 172.21.15.97:0/2474916516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d28009a70 con 0x7f4d3815b470 2024-01-31T05:02:37.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d2de00640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380cbf20 0x7f4d381f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.453+0000 7f4d37400640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 0x7f4d381f6c40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e840 tx=0x7f4d3000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.454+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30009660 con 0x7f4d3815b470 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.454+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3000c940 con 0x7f4d3815b470 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.454+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000cd10 con 0x7f4d3815b470 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.454+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3815ccd0 con 0x7f4d3815b470 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.454+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d381f33a0 con 0x7f4d3815b470 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.455+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3806d190 con 0x7f4d3815b470 2024-01-31T05:02:37.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.456+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4d30013470 con 0x7f4d3815b470 2024-01-31T05:02:37.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.457+0000 7f4d2f400640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4d1006aed0 0x7f4d1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.457+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4d300899e0 con 0x7f4d3815b470 2024-01-31T05:02:37.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.457+0000 7f4d36a00640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4d1006aed0 0x7f4d1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.458+0000 7f4d36a00640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4d1006aed0 0x7f4d1006d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e10 tx=0x7f4d28030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:37.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.461+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4d3806d190 con 0x7f4d3815b470 2024-01-31T05:02:37.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.726+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4d3815d1f0 con 0x7f4d3815b470 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.731+0000 7f4d2f400640 1 -- 172.21.15.97:0/2474916516 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4d300210b0 con 0x7f4d3815b470 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-31T05:02:37.731 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:02:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4d1006aed0 msgr2=0x7f4d1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4d1006aed0 0x7f4d1006d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009e10 tx=0x7f4d28030000 comp rx=0 tx=0).stop 2024-01-31T05:02:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 msgr2=0x7f4d381f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 0x7f4d381f6c40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000e840 tx=0x7f4d3000ed00 comp rx=0 tx=0).stop 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 shutdown_connections 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4d1006aed0 0x7f4d1006d390 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d3815b470 0x7f4d381f6c40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d380cbf20 0x7f4d381f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 --2- 172.21.15.97:0/2474916516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d380abcf0 0x7f4d3815b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 >> 172.21.15.97:0/2474916516 conn(0x7f4d380b0a10 msgr2=0x7f4d380cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 shutdown_connections 2024-01-31T05:02:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.735+0000 7f4d3d800640 1 -- 172.21.15.97:0/2474916516 wait complete. 2024-01-31T05:02:37.752 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.959+0000 7fc409400640 1 Processor -- start 2024-01-31T05:02:37.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.959+0000 7fc409400640 1 -- start start 2024-01-31T05:02:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.960+0000 7fc409400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4040ae3b0 0x7fc40406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.960+0000 7fc409400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40406d0d0 0x7fc40415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.960+0000 7fc409400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc404070400 con 0x7fc40415ec70 2024-01-31T05:02:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.960+0000 7fc409400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4040ad8c0 con 0x7fc4040ae3b0 2024-01-31T05:02:37.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.960+0000 7fc409400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4040ca580 con 0x7fc40406d0d0 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.961+0000 7fc3fb200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4040ae3b0 0x7fc40406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.961+0000 7fc3fb200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4040ae3b0 0x7fc40406cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:56602/0 (socket says 172.21.15.97:56602) 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.961+0000 7fc3fb200640 1 -- 172.21.15.97:0/910418275 learned_addr learned my addr 172.21.15.97:0/910418275 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.961+0000 7fc403e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc40415ec70 0x7fc404071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:45636/0 (socket says 172.21.15.97:45636) 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.961+0000 7fc408a00640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40406d0d0 0x7fc40415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.962+0000 7fc3fb200640 1 -- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40406d0d0 msgr2=0x7fc40415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.962+0000 7fc3fb200640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40406d0d0 0x7fc40415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.962+0000 7fc3fb200640 1 -- 172.21.15.97:0/910418275 >> v1:172.21.15.188:6789/0 conn(0x7fc40415ec70 legacy=0x7fc404071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.962+0000 7fc3fb200640 1 -- 172.21.15.97:0/910418275 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc404070650 con 0x7fc4040ae3b0 2024-01-31T05:02:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.962+0000 7fc408a00640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40406d0d0 0x7fc40415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:02:37.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.963+0000 7fc3fb200640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4040ae3b0 0x7fc40406cb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009870 tx=0x7fc3ec02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c4fbf895e3b9a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.963+0000 7fc403400640 1 -- 172.21.15.97:0/910418275 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0317c0 con 0x7fc4040ae3b0 2024-01-31T05:02:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.963+0000 7fc403400640 1 -- 172.21.15.97:0/910418275 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec031920 con 0x7fc4040ae3b0 2024-01-31T05:02:37.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.963+0000 7fc403400640 1 -- 172.21.15.97:0/910418275 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec031cb0 con 0x7fc4040ae3b0 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 -- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4040ae3b0 msgr2=0x7fc40406cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4040ae3b0 0x7fc40406cb90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009870 tx=0x7fc3ec02b860 comp rx=0 tx=0).stop 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 -- 172.21.15.97:0/910418275 shutdown_connections 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4040ae3b0 0x7fc40406cb90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 --2- 172.21.15.97:0/910418275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc40406d0d0 0x7fc40415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 -- 172.21.15.97:0/910418275 >> 172.21.15.97:0/910418275 conn(0x7fc4040cec40 msgr2=0x7fc4040cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 -- 172.21.15.97:0/910418275 shutdown_connections 2024-01-31T05:02:37.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.964+0000 7fc409400640 1 -- 172.21.15.97:0/910418275 wait complete. 2024-01-31T05:02:37.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.966+0000 7fc409400640 1 Processor -- start 2024-01-31T05:02:37.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.966+0000 7fc409400640 1 -- start start 2024-01-31T05:02:37.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc409400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40406d0d0 0x7fc4040b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc409400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4040ae3b0 0x7fc4040cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc409400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 0x7fc4041f0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc409400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40414fce0 con 0x7fc40415ec70 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc409400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4040b4140 con 0x7fc40406d0d0 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc409400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40409a470 con 0x7fc4040ae3b0 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc408a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 0x7fc4041f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc408a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 0x7fc4041f0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54414/0 (socket says 172.21.15.97:54414) 2024-01-31T05:02:37.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.967+0000 7fc408a00640 1 -- 172.21.15.97:0/3178253105 learned_addr learned my addr 172.21.15.97:0/3178253105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc3fb200640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4040ae3b0 0x7fc4040cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc403e00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40406d0d0 0x7fc4040b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc408a00640 1 -- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4040ae3b0 msgr2=0x7fc4040cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc408a00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4040ae3b0 0x7fc4040cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc408a00640 1 -- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40406d0d0 msgr2=0x7fc4040b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc408a00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40406d0d0 0x7fc4040b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc408a00640 1 -- 172.21.15.97:0/3178253105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc009a70 con 0x7fc40415ec70 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc403e00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40406d0d0 0x7fc4040b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc408a00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 0x7fc4041f0400 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00efc0 tx=0x7fc3fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc009cd0 con 0x7fc40415ec70 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc00f040 con 0x7fc40415ec70 2024-01-31T05:02:37.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc003990 con 0x7fc40415ec70 2024-01-31T05:02:37.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.968+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec0094d0 con 0x7fc40415ec70 2024-01-31T05:02:37.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.969+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40409aa30 con 0x7fc40415ec70 2024-01-31T05:02:37.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.970+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4040676b0 con 0x7fc40415ec70 2024-01-31T05:02:37.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.971+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc3fc014480 con 0x7fc40415ec70 2024-01-31T05:02:37.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.972+0000 7fc400c00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3dc06aed0 0x7fc3dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:37.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.972+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc3fc08a3a0 con 0x7fc40415ec70 2024-01-31T05:02:37.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.972+0000 7fc403e00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3dc06aed0 0x7fc3dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:37.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.973+0000 7fc403e00640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3dc06aed0 0x7fc3dc06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000f60 tx=0x7fc3f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:37.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:37.977+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc3fc055130 con 0x7fc40415ec70 2024-01-31T05:02:38.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.211+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4040cfa70 con 0x7fc40415ec70 2024-01-31T05:02:38.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.212+0000 7fc400c00640 1 -- 172.21.15.97:0/3178253105 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fc3fc058750 con 0x7fc40415ec70 2024-01-31T05:02:38.211 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:38.211 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:34.782181+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:02:38.212 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3dc06aed0 msgr2=0x7fc3dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3dc06aed0 0x7fc3dc06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000f60 tx=0x7fc3f4009040 comp rx=0 tx=0).stop 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 msgr2=0x7fc4041f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 0x7fc4041f0400 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00efc0 tx=0x7fc3fc00c490 comp rx=0 tx=0).stop 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 shutdown_connections 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3dc06aed0 0x7fc3dc06d390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc40415ec70 0x7fc4041f0400 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4040ae3b0 0x7fc4040cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 --2- 172.21.15.97:0/3178253105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc40406d0d0 0x7fc4040b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 >> 172.21.15.97:0/3178253105 conn(0x7fc4040cec40 msgr2=0x7fc4040af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 shutdown_connections 2024-01-31T05:02:38.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.215+0000 7fc409400640 1 -- 172.21.15.97:0/3178253105 wait complete. 2024-01-31T05:02:38.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-31T05:02:31.698287+0000', 'modified': '2024-01-31T05:02:34.782181+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5556': {'gid': 5556, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.188:6834/111752619', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 111752619}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 111752619}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-31T05:02:38.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-31T05:02:38.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-31T05:02:38.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.847+0000 7f05ca800640 1 Processor -- start 2024-01-31T05:02:38.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.848+0000 7f05ca800640 1 -- start start 2024-01-31T05:02:38.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05ca800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f05c415b470 0x7f05c40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05ca800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05c40abcf0 0x7f05c40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05ca800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c406cd20 con 0x7f05c40cb810 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05ca800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40ce970 con 0x7f05c40abcf0 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05ca800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c415b850 con 0x7f05c415b470 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05bbe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05c40abcf0 0x7f05c40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05c3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f05c40cb810 0x7f05c40ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58472/0 (socket says 172.21.15.97:58472) 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.849+0000 7f05c3e00640 1 -- 172.21.15.97:0/3132833129 learned_addr learned my addr 172.21.15.97:0/3132833129 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.850+0000 7f05c3400640 1 --2- 172.21.15.97:0/3132833129 >> v2:172.21.15.188:3300/0 conn(0x7f05c415b470 0x7f05c40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.850+0000 7f05bbe00640 1 -- 172.21.15.97:0/3132833129 >> v2:172.21.15.188:3300/0 conn(0x7f05c415b470 msgr2=0x7f05c40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.850+0000 7f05bbe00640 1 --2- 172.21.15.97:0/3132833129 >> v2:172.21.15.188:3300/0 conn(0x7f05c415b470 0x7f05c40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.850+0000 7f05bbe00640 1 -- 172.21.15.97:0/3132833129 >> v1:172.21.15.97:6789/0 conn(0x7f05c40cb810 legacy=0x7f05c40ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.850+0000 7f05bbe00640 1 -- 172.21.15.97:0/3132833129 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40ceb60 con 0x7f05c40abcf0 2024-01-31T05:02:38.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.850+0000 7f05c2a00640 1 -- 172.21.15.97:0/3132833129 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253031448 0 0) 0x7f05c406cd20 con 0x7f05c40cb810 2024-01-31T05:02:38.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.851+0000 7f05bbe00640 1 --2- 172.21.15.97:0/3132833129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40abcf0 0x7f05c40ce430 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05c4067370 tx=0x7f05ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6953d8465c2108 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.855+0000 7f05c2a00640 1 -- 172.21.15.97:0/3132833129 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac031830 con 0x7f05c40abcf0 2024-01-31T05:02:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.855+0000 7f05c2a00640 1 -- 172.21.15.97:0/3132833129 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac031990 con 0x7f05c40abcf0 2024-01-31T05:02:38.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.855+0000 7f05c2a00640 1 -- 172.21.15.97:0/3132833129 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac031d20 con 0x7f05c40abcf0 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 -- 172.21.15.97:0/3132833129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40abcf0 msgr2=0x7f05c40ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 --2- 172.21.15.97:0/3132833129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40abcf0 0x7f05c40ce430 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05c4067370 tx=0x7f05ac02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 -- 172.21.15.97:0/3132833129 shutdown_connections 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 --2- 172.21.15.97:0/3132833129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40abcf0 0x7f05c40ce430 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 --2- 172.21.15.97:0/3132833129 >> v2:172.21.15.188:3300/0 conn(0x7f05c415b470 0x7f05c40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 -- 172.21.15.97:0/3132833129 >> 172.21.15.97:0/3132833129 conn(0x7f05c40b0a10 msgr2=0x7f05c40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 -- 172.21.15.97:0/3132833129 shutdown_connections 2024-01-31T05:02:38.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.856+0000 7f05ca800640 1 -- 172.21.15.97:0/3132833129 wait complete. 2024-01-31T05:02:38.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.858+0000 7f05ca800640 1 Processor -- start 2024-01-31T05:02:38.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.858+0000 7f05ca800640 1 -- start start 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05ca800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05c40abcf0 0x7f05c41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05ca800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 0x7f05c40ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05ca800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05c415b470 0x7f05c414f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05ca800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4155910 con 0x7f05c40cb810 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05ca800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41ef6c0 con 0x7f05c40abcf0 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05ca800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41538d0 con 0x7f05c415b470 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 0x7f05c40ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 0x7f05c40ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54426/0 (socket says 172.21.15.97:54426) 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05bbe00640 1 -- 172.21.15.97:0/2717137783 learned_addr learned my addr 172.21.15.97:0/2717137783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.859+0000 7f05c3400640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05c40abcf0 0x7f05c41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05c3e00640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05c415b470 0x7f05c414f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bbe00640 1 -- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05c415b470 msgr2=0x7f05c414f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bbe00640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05c415b470 0x7f05c414f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bbe00640 1 -- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05c40abcf0 msgr2=0x7f05c41543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bbe00640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05c40abcf0 0x7f05c41543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bbe00640 1 -- 172.21.15.97:0/2717137783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009400 con 0x7f05c40cb810 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bbe00640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 0x7f05c40ad930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f05ac02bee0 tx=0x7f05ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bb400640 1 -- 172.21.15.97:0/2717137783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac03a6f0 con 0x7f05c40cb810 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bb400640 1 -- 172.21.15.97:0/2717137783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac03a870 con 0x7f05c40cb810 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05bb400640 1 -- 172.21.15.97:0/2717137783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac03ac20 con 0x7f05c40cb810 2024-01-31T05:02:38.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c4153bb0 con 0x7f05c40cb810 2024-01-31T05:02:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.860+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c41548f0 con 0x7f05c40cb810 2024-01-31T05:02:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.862+0000 7f05bb400640 1 -- 172.21.15.97:0/2717137783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f05ac04c3b0 con 0x7f05c40cb810 2024-01-31T05:02:38.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.863+0000 7f05bb400640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f059c06aed0 0x7f059c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.863+0000 7f05bb400640 1 -- 172.21.15.97:0/2717137783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f05ac0ab490 con 0x7f05c40cb810 2024-01-31T05:02:38.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.863+0000 7f05c3400640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f059c06aed0 0x7f059c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.864+0000 7f05c3400640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f059c06aed0 0x7f059c06d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05b400b9f0 tx=0x7f05b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.864+0000 7f05cd76f740 1 Processor -- start 2024-01-31T05:02:38.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.864+0000 7f05cd76f740 1 -- start start 2024-01-31T05:02:38.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x56079857ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985a9e60 0x5607985ac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56079857cf60 0x5607985acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05cd76f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607984eff50 con 0x5607985a9a80 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607984ed640 con 0x56079857cf60 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607984e8f30 con 0x5607985a9e60 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x56079857ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x56079857ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54438/0 (socket says 172.21.15.97:54438) 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.865+0000 7f05c3400640 1 -- 172.21.15.97:0/1428523448 learned_addr learned my addr 172.21.15.97:0/1428523448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985a9e60 0x5607985ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3400640 1 -- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985a9e60 msgr2=0x5607985ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3400640 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985a9e60 0x5607985ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3e00640 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56079857cf60 0x5607985acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3400640 1 -- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56079857cf60 msgr2=0x5607985acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3400640 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56079857cf60 0x5607985acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3400640 1 -- 172.21.15.97:0/1428523448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607984ab2d0 con 0x5607985a9a80 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05c3400640 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x56079857ca20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f05b400fab0 tx=0x7f05b4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63128efb2289736c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05b3e00640 1 -- 172.21.15.97:0/1428523448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4024070 con 0x5607985a9a80 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05b3e00640 1 -- 172.21.15.97:0/1428523448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b4019070 con 0x5607985a9a80 2024-01-31T05:02:38.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.866+0000 7f05b3e00640 1 -- 172.21.15.97:0/1428523448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4014ce0 con 0x5607985a9a80 2024-01-31T05:02:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 -- 172.21.15.97:0/1428523448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 msgr2=0x56079857ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x56079857ca20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f05b400fab0 tx=0x7f05b4012000 comp rx=0 tx=0).stop 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 -- 172.21.15.97:0/1428523448 shutdown_connections 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56079857cf60 0x5607985acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985a9e60 0x5607985ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1428523448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x56079857ca20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 -- 172.21.15.97:0/1428523448 >> 172.21.15.97:0/1428523448 conn(0x56079857ec90 msgr2=0x5607985c7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 -- 172.21.15.97:0/1428523448 shutdown_connections 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.867+0000 7f05cd76f740 1 -- 172.21.15.97:0/1428523448 wait complete. 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.868+0000 7f05cd76f740 1 -- start start 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.868+0000 7f05cd76f740 1 Processor -- start 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x560798599c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607984a26b0 con 0x5607985a9e60 2024-01-31T05:02:38.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607984a2350 con 0x5607985a9a80 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56079824af20 con 0x5607985b01b0 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05cd76f740 10 client.0 _ll_register_callbacks cb 0x5607984a49c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54442/0 (socket says 172.21.15.97:54442) 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05bbe00640 1 -- 172.21.15.97:0/1359671913 learned_addr learned my addr 172.21.15.97:0/1359671913 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05c3400640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x560798599c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.869+0000 7f05c3e00640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05bbe00640 1 -- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 msgr2=0x5607985d7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05bbe00640 1 -- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 msgr2=0x560798599c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x560798599c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05bbe00640 1 -- 172.21.15.97:0/1359671913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b400e040 con 0x5607985a9e60 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05c3e00640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:38.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05c3400640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x560798599c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985afc70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0764f0 tx=0x7f05ac07a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f602d722910d790 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05b0c00640 1 -- 172.21.15.97:0/1359671913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac076700 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05b0c00640 1 -- 172.21.15.97:0/1359671913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f05ac076860 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560798510e60 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05b0c00640 1 -- 172.21.15.97:0/1359671913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05ac0800b0 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05cd76f740 10 client.5718 fetch_fsmap learned FSMap version 17 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5607984e75b0 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.870+0000 7f05b0c00640 1 -- 172.21.15.97:0/1359671913 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0a8070 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05b0c00640 1 -- 172.21.15.97:0/1359671913 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f05ac0ab0b0 con 0x5607985a9e60 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05cd76f740 10 client.5718 fetch_fsmap finished waiting for FSMap version 17 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05cd76f740 10 client.5718 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05cd76f740 20 client.5718 populate_metadata read hostname 'smithi097' 2024-01-31T05:02:38.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x5607985d7d90 0x5607985e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05cd76f740 4 client.5718 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:02:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 --> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607984bac40 con 0x5607985d7d90 2024-01-31T05:02:38.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.871+0000 7f05c3400640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x5607985d7d90 0x5607985e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.872+0000 7f05c3400640 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x5607985d7d90 0x5607985e4ac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e01c6d17565259 server_cookie=3a98438dda95f073 in_seq=0 out_seq=0 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.872+0000 7f05b0c00640 10 client.5718 ms_handle_connect on v2:172.21.15.188:6832/111752619 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05b0c00640 1 -- 172.21.15.97:0/1359671913 <== mds.0 v2:172.21.15.188:6832/111752619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607984bac40 con 0x5607985d7d90 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05b0c00640 10 client.5718 handle_command_reply: tid=0 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 client.5718 shutdown 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 msgr2=0x5607985afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985afc70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0764f0 tx=0x7f05ac07a830 comp rx=0 tx=0).stop 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 shutdown_connections 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x5607985d7d90 0x5607985e4ac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985afc70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1359671913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x560798599c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.874+0000 7f05b0c00640 0 client.5718 ms_handle_reset on v2:172.21.15.188:6832/111752619 2024-01-31T05:02:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.875+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 >> 172.21.15.97:0/1359671913 conn(0x56079857cf60 msgr2=0x5607985cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.875+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 shutdown_connections 2024-01-31T05:02:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.875+0000 7f05cd76f740 1 -- 172.21.15.97:0/1359671913 wait complete. 2024-01-31T05:02:38.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.875+0000 7f05cd76f740 20 client.5718 trim_cache size 0 max 16384 2024-01-31T05:02:38.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.891+0000 7f05cd76f740 1 Processor -- start 2024-01-31T05:02:38.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.891+0000 7f05cd76f740 1 -- start start 2024-01-31T05:02:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.892+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x5607985af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.892+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.892+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.892+0000 7f05cd76f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607985c8510 con 0x5607985a9e60 2024-01-31T05:02:38.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.892+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x56079857e590 con 0x5607985a9a80 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.892+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607985c7740 con 0x5607985b01b0 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54454/0 (socket says 172.21.15.97:54454) 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 -- 172.21.15.97:0/1363837279 learned_addr learned my addr 172.21.15.97:0/1363837279 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 -- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 msgr2=0x5607985d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05c3e00640 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 -- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 msgr2=0x5607985af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x5607985af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 -- 172.21.15.97:0/1363837279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607984841e0 con 0x5607985a9e60 2024-01-31T05:02:38.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05bbe00640 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985d6d80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05ac079f10 tx=0x7f05ac035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ff4a4eaad25ed3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05b3400640 1 -- 172.21.15.97:0/1363837279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac02d040 con 0x5607985a9e60 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05b3400640 1 -- 172.21.15.97:0/1363837279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0a8070 con 0x5607985a9e60 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.893+0000 7f05b3400640 1 -- 172.21.15.97:0/1363837279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac079950 con 0x5607985a9e60 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 -- 172.21.15.97:0/1363837279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 msgr2=0x5607985d6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985d6d80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f05ac079f10 tx=0x7f05ac035000 comp rx=0 tx=0).stop 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 -- 172.21.15.97:0/1363837279 shutdown_connections 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 --2- 172.21.15.97:0/1363837279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9a80 0x5607985af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 -- 172.21.15.97:0/1363837279 >> 172.21.15.97:0/1363837279 conn(0x56079857cf60 msgr2=0x560798583560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 -- 172.21.15.97:0/1363837279 shutdown_connections 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.894+0000 7f05cd76f740 1 -- 172.21.15.97:0/1363837279 wait complete. 2024-01-31T05:02:38.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.895+0000 7f05cd76f740 1 -- start start 2024-01-31T05:02:38.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.895+0000 7f05cd76f740 1 Processor -- start 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x560798630be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607985aa9c0 con 0x5607985a9a80 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607985eea00 con 0x5607985a9e60 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560798632720 con 0x5607985b01b0 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05cd76f740 10 client.0 _ll_register_callbacks cb 0x7f05ac076330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:02:38.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x560798630be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05c3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x560798630be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54456/0 (socket says 172.21.15.97:54456) 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.896+0000 7f05c3400640 1 -- 172.21.15.97:0/186316749 learned_addr learned my addr 172.21.15.97:0/186316749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05c3400640 1 -- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 msgr2=0x5607985d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05c3400640 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05c3400640 1 -- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9e60 msgr2=0x5607985d6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05c3400640 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05c3400640 1 -- 172.21.15.97:0/186316749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607984e75b0 con 0x5607985a9a80 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05c3400640 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x560798630be0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f05b400fa80 tx=0x7f05b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d330288eb0d32fb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05b0c00640 1 -- 172.21.15.97:0/186316749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b402f070 con 0x5607985a9a80 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05b0c00640 1 -- 172.21.15.97:0/186316749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f05b4005990 con 0x5607985a9a80 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05b4019070 con 0x5607985a9a80 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05b0c00640 1 -- 172.21.15.97:0/186316749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b4024040 con 0x5607985a9a80 2024-01-31T05:02:38.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05cd76f740 10 client.5724 fetch_fsmap learned FSMap version 17 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.897+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5607984a9b30 con 0x5607985a9a80 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05b0c00640 1 -- 172.21.15.97:0/186316749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b4019070 con 0x5607985a9a80 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05b0c00640 1 -- 172.21.15.97:0/186316749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f05b4005b50 con 0x5607985a9a80 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05cd76f740 10 client.5724 fetch_fsmap finished waiting for FSMap version 17 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05cd76f740 10 client.5724 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05cd76f740 20 client.5724 populate_metadata read hostname 'smithi097' 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05cd76f740 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x560798631970 0x5607985984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05cd76f740 4 client.5724 mds_command: new command op to 5556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 --> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5607984e1dc0 con 0x560798631970 2024-01-31T05:02:38.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.898+0000 7f05bbe00640 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x560798631970 0x5607985984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.899+0000 7f05b0c00640 10 client.5724 ms_handle_connect on v2:172.21.15.188:6832/111752619 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.899+0000 7f05bbe00640 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x560798631970 0x5607985984a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef71fc4b8b115812 server_cookie=8ff49fae4a6ed326 in_seq=0 out_seq=0 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05b0c00640 1 -- 172.21.15.97:0/186316749 <== mds.0 v2:172.21.15.188:6832/111752619 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5607984e1dc0 con 0x560798631970 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05b0c00640 10 client.5724 handle_command_reply: tid=0 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 client.5724 shutdown 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 msgr2=0x560798630be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x560798630be0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f05b400fa80 tx=0x7f05b40046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 shutdown_connections 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:6832/111752619,v1:172.21.15.188:6834/111752619] conn(0x560798631970 0x5607985984a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5607985b01b0 0x5607985d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607985a9e60 0x5607985d6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05b0c00640 0 client.5724 ms_handle_reset on v2:172.21.15.188:6832/111752619 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 --2- 172.21.15.97:0/186316749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5607985a9a80 0x560798630be0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 >> 172.21.15.97:0/186316749 conn(0x56079857cf60 msgr2=0x5607985e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 shutdown_connections 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 1 -- 172.21.15.97:0/186316749 wait complete. 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.900+0000 7f05cd76f740 20 client.5724 trim_cache size 0 max 16384 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "dc34a409-d027-49db-a591-dbd89a074581", 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5556, 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-01-31T05:02:38.903 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-01-31T05:02:38.904 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 6.6316572479999998, 2024-01-31T05:02:38.904 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 17, 2024-01-31T05:02:38.904 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 40, 2024-01-31T05:02:38.904 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 40, 2024-01-31T05:02:38.904 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 15.520006321 2024-01-31T05:02:38.904 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f059c06aed0 msgr2=0x7f059c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f059c06aed0 0x7f059c06d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05b400b9f0 tx=0x7f05b4008040 comp rx=0 tx=0).stop 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 msgr2=0x7f05c40ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 0x7f05c40ad930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f05ac02bee0 tx=0x7f05ac0046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 shutdown_connections 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05c415b470 0x7f05c414f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f059c06aed0 0x7f059c06d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05c40cb810 0x7f05c40ad930 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 --2- 172.21.15.97:0/2717137783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05c40abcf0 0x7f05c41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 >> 172.21.15.97:0/2717137783 conn(0x7f05c40b0a10 msgr2=0x7f05c4156310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 shutdown_connections 2024-01-31T05:02:38.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:38.901+0000 7f05ca800640 1 -- 172.21.15.97:0/2717137783 wait complete. 2024-01-31T05:02:38.916 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:02:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.114+0000 7f0afe600640 1 Processor -- start 2024-01-31T05:02:39.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.115+0000 7f0afe600640 1 -- start start 2024-01-31T05:02:39.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.116+0000 7f0afe600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0af815b400 0x7f0af806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:39.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.116+0000 7f0afe600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0af806db70 0x7f0af80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:39.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.116+0000 7f0afe600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af806d1a0 con 0x7f0af80cb810 2024-01-31T05:02:39.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.116+0000 7f0afe600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80cc960 con 0x7f0af806db70 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.116+0000 7f0afe600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af806c600 con 0x7f0af815b400 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0af806db70 0x7f0af80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0af806db70 0x7f0af80ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54468/0 (socket says 172.21.15.97:54468) 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 -- 172.21.15.97:0/3737650908 learned_addr learned my addr 172.21.15.97:0/3737650908 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0aefe00640 1 --2- 172.21.15.97:0/3737650908 >> v2:172.21.15.188:3300/0 conn(0x7f0af815b400 0x7f0af806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 -- 172.21.15.97:0/3737650908 >> v2:172.21.15.188:3300/0 conn(0x7f0af815b400 msgr2=0x7f0af806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 --2- 172.21.15.97:0/3737650908 >> v2:172.21.15.188:3300/0 conn(0x7f0af815b400 0x7f0af806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 -- 172.21.15.97:0/3737650908 >> v1:172.21.15.188:6789/0 conn(0x7f0af80cb810 legacy=0x7f0af80cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:39.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.117+0000 7f0af7e00640 1 -- 172.21.15.97:0/3737650908 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af806c870 con 0x7f0af806db70 2024-01-31T05:02:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.118+0000 7f0af7e00640 1 --2- 172.21.15.97:0/3737650908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af806db70 0x7f0af80ccd00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0af80ae830 tx=0x7f0af002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ba3741881d4401 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.118+0000 7f0af6a00640 1 -- 172.21.15.97:0/3737650908 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af002ca80 con 0x7f0af806db70 2024-01-31T05:02:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0af6a00640 1 -- 172.21.15.97:0/3737650908 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af002cbe0 con 0x7f0af806db70 2024-01-31T05:02:39.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0af6a00640 1 -- 172.21.15.97:0/3737650908 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af00343f0 con 0x7f0af806db70 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 -- 172.21.15.97:0/3737650908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af806db70 msgr2=0x7f0af80ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 --2- 172.21.15.97:0/3737650908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af806db70 0x7f0af80ccd00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0af80ae830 tx=0x7f0af002bee0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 -- 172.21.15.97:0/3737650908 shutdown_connections 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 --2- 172.21.15.97:0/3737650908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af806db70 0x7f0af80ccd00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 --2- 172.21.15.97:0/3737650908 >> v2:172.21.15.188:3300/0 conn(0x7f0af815b400 0x7f0af806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 -- 172.21.15.97:0/3737650908 >> 172.21.15.97:0/3737650908 conn(0x7f0af80b0a10 msgr2=0x7f0af80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 -- 172.21.15.97:0/3737650908 shutdown_connections 2024-01-31T05:02:39.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.119+0000 7f0afe600640 1 -- 172.21.15.97:0/3737650908 wait complete. 2024-01-31T05:02:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.121+0000 7f0afe600640 1 Processor -- start 2024-01-31T05:02:39.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.121+0000 7f0afe600640 1 -- start start 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0afe600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af806db70 0x7f0af815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0afe600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cb810 0x7f0af814ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0afe600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 0x7f0af80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0afe600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815c810 con 0x7f0af815b400 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0afe600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815caf0 con 0x7f0af80cb810 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0afe600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815cdd0 con 0x7f0af806db70 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0af7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 0x7f0af80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0af7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 0x7f0af80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54482/0 (socket says 172.21.15.97:54482) 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.122+0000 7f0af7e00640 1 -- 172.21.15.97:0/2733830461 learned_addr learned my addr 172.21.15.97:0/2733830461 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7400640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af806db70 0x7f0af815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:39.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0aefe00640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cb810 0x7f0af814ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7e00640 1 -- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af806db70 msgr2=0x7f0af815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7e00640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af806db70 0x7f0af815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7e00640 1 -- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cb810 msgr2=0x7f0af814ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7e00640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cb810 0x7f0af814ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7e00640 1 -- 172.21.15.97:0/2733830461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af0009b30 con 0x7f0af815b400 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0af7e00640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 0x7f0af80ce740 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0af0036040 tx=0x7f0af0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0009510 con 0x7f0af815b400 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af003e450 con 0x7f0af815b400 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af003e800 con 0x7f0af815b400 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.123+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af815d0b0 con 0x7f0af815b400 2024-01-31T05:02:39.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.124+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af815ddf0 con 0x7f0af815b400 2024-01-31T05:02:39.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.125+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af80677c0 con 0x7f0af815b400 2024-01-31T05:02:39.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.126+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f0af002cd50 con 0x7f0af815b400 2024-01-31T05:02:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.127+0000 7f0aef400640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ad006aed0 0x7f0ad006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.127+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0af00ac390 con 0x7f0af815b400 2024-01-31T05:02:39.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.127+0000 7f0af7400640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ad006aed0 0x7f0ad006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:39.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.128+0000 7f0af7400640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ad006aed0 0x7f0ad006d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8004ea0 tx=0x7f0ae8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:39.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.131+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f0af0077080 con 0x7f0af815b400 2024-01-31T05:02:39.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.350+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0af80cfbf0 con 0x7f0af815b400 2024-01-31T05:02:39.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.351+0000 7f0aef400640 1 -- 172.21.15.97:0/2733830461 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0af007a6a0 con 0x7f0af815b400 2024-01-31T05:02:39.350 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ad006aed0 msgr2=0x7f0ad006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ad006aed0 0x7f0ad006d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8004ea0 tx=0x7f0ae8009040 comp rx=0 tx=0).stop 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 msgr2=0x7f0af80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 0x7f0af80ce740 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0af0036040 tx=0x7f0af0004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 shutdown_connections 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0ad006aed0 0x7f0ad006d390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af815b400 0x7f0af80ce740 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80cb810 0x7f0af814ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 --2- 172.21.15.97:0/2733830461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0af806db70 0x7f0af815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 >> 172.21.15.97:0/2733830461 conn(0x7f0af80b0a10 msgr2=0x7f0af81559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.353+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 shutdown_connections 2024-01-31T05:02:39.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:39.354+0000 7f0afe600640 1 -- 172.21.15.97:0/2733830461 wait complete. 2024-01-31T05:02:39.370 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:02:39.370 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:02:39.370 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:02:39.370 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:02:39.370 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:39.371 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:02:39.371 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:02:39.371 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:02:39.371 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:02:39.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:39.371 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:02:39.415 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:02:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:02:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:02:39.415 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42365sec preferred_lft 42365sec 2024-01-31T05:02:39.416 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether ca:84:a9:c5:b2:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c884:a9ff:fec5:b219/64 scope link 2024-01-31T05:02:39.417 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:39.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:39.418 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:02:39.540 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:02:39.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:39.545 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:02:39.597 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:02:39.597 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:02:39.597 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:02:39.598 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:02:39.598 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:02:39.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:39.598 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:02:39.598 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:02:39.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:02:39.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:02:39.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:02:39.598 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:02:39.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:39.771 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:02:39.820 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:02:39.820 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:02:39.821 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:02:39.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:39.822 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:02:39.917 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:02:39.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:02:39.917 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:39.963 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:02:39.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:02:39.963 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:40.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:02:40.143 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:02:40.199 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:02:40.199 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:02:40.199 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:02:40.199 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:02:40.200 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:40.201 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:40.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:40.202 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:02:40.255 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:02:40.255 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:02:40.262 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:02:40.340 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:02:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:02:40.383 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:02:40.384 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:40.385 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:40.386 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:40.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:40.386 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:02:40.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:02:40.449+0000 7f2f33c47480 -1 init, newargv = 0x563f96475ca0 newargc=15 2024-01-31T05:02:40.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[52785]: starting ceph client 2024-01-31T05:02:40.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[52785]: starting fuse 2024-01-31T05:02:40.468 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:02:40.468 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:02:40.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:02:40.468 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:02:40.468 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:02:40.468 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:02:40.469 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:02:40.470 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:02:40.660 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.52785.asok 2024-01-31T05:02:40.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:02:40.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52785.asok 2024-01-31T05:02:40.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:40.677 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52785.asok status 2024-01-31T05:02:40.912 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:02:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:02:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:02:40.912 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "52785", 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5736, 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 5736 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:40.913 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1620533631 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1620533631 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.5736 192.168.0.1:0/1620533631", 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/1620533631", 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 17, 2024-01-31T05:02:40.914 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 40, 2024-01-31T05:02:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:02:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:02:40.915 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:02:40.915 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:02:40.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:40.928 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:02:40.928 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:40.977 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:02:40.977 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:40.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:40.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:41.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-01-31T05:02:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.272+0000 7fb28cc00640 1 Processor -- start 2024-01-31T05:02:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.273+0000 7fb28cc00640 1 -- start start 2024-01-31T05:02:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.274+0000 7fb28cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb2880ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb28cc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2880cab10 con 0x7fb2880c9430 2024-01-31T05:02:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb28cc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2880c99f0 con 0x7fb28806e1e0 2024-01-31T05:02:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb28cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28806c720 con 0x7fb28815a230 2024-01-31T05:02:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb285400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb2880c9430 0x7fb28806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58480/0 (socket says 172.21.15.97:58480) 2024-01-31T05:02:41.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb285400640 1 -- 172.21.15.97:0/1710286154 learned_addr learned my addr 172.21.15.97:0/1710286154 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:41.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb285e00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb2880ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303865222 0 0) 0x7fb2880cab10 con 0x7fb2880c9430 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.275+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb268005730 con 0x7fb2880c9430 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1726623305 0 0) 0x7fb268005730 con 0x7fb2880c9430 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 msgr2=0x7fb2880ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb2880ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 >> v1:172.21.15.188:6789/0 conn(0x7fb28806e1e0 legacy=0x7fb288071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880ae050 con 0x7fb2880c9430 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fb270001670 con 0x7fb2880c9430 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 >> v1:172.21.15.97:6789/0 conn(0x7fb2880c9430 legacy=0x7fb28806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb268005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb268006130 0x7fb2680069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb268006f10 0x7fb268007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28806c720 con 0x7fb268006f10 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26800dd20 con 0x7fb268006130 2024-01-31T05:02:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26800dee0 con 0x7fb28815a230 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb286800640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb268006f10 0x7fb268007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.276+0000 7fb285e00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb268005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb285400640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb268006130 0x7fb2680069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb286800640 1 -- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 msgr2=0x7fb268005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb286800640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb268005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb286800640 1 -- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb268006130 msgr2=0x7fb2680069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb286800640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb268006130 0x7fb2680069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb286800640 1 -- 172.21.15.97:0/1710286154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268005730 con 0x7fb268006f10 2024-01-31T05:02:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.277+0000 7fb285e00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb28815a230 0x7fb268005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:02:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.278+0000 7fb286800640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb268006f10 0x7fb268007390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb280001720 tx=0x7fb268007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00a6d5118f7c45c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.278+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800052d0 con 0x7fb268006f10 2024-01-31T05:02:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.278+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28002e050 con 0x7fb268006f10 2024-01-31T05:02:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.278+0000 7fb284a00640 1 -- 172.21.15.97:0/1710286154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800035f0 con 0x7fb268006f10 2024-01-31T05:02:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb28cc00640 1 -- 172.21.15.97:0/1710286154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb268006f10 msgr2=0x7fb268007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb28cc00640 1 --2- 172.21.15.97:0/1710286154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb268006f10 0x7fb268007390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb280001720 tx=0x7fb268007a00 comp rx=0 tx=0).stop 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb286800640 1 -- 172.21.15.97:0/1710286154 reap_dead start 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb28cc00640 1 -- 172.21.15.97:0/1710286154 shutdown_connections 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb28cc00640 1 -- 172.21.15.97:0/1710286154 >> 172.21.15.97:0/1710286154 conn(0x7fb2880cede0 msgr2=0x7fb2880cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb28cc00640 1 -- 172.21.15.97:0/1710286154 shutdown_connections 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.279+0000 7fb28cc00640 1 -- 172.21.15.97:0/1710286154 wait complete. 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.281+0000 7fb28cc00640 1 Processor -- start 2024-01-31T05:02:41.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.281+0000 7fb28cc00640 1 -- start start 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb28cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb268006f10 0x7fb2880999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb28cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 0x7fb28809aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb28cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2880c9ba0 0x7fb2880ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb28cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880ca560 con 0x7fb288148870 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb28cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880c6050 con 0x7fb2880c9ba0 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb28cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880c62f0 con 0x7fb268006f10 2024-01-31T05:02:41.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb285400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 0x7fb28809aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb285400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 0x7fb28809aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54486/0 (socket says 172.21.15.97:54486) 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb285400640 1 -- 172.21.15.97:0/2372527882 learned_addr learned my addr 172.21.15.97:0/2372527882 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb285e00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb268006f10 0x7fb2880999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.282+0000 7fb286800640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2880c9ba0 0x7fb2880ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285400640 1 -- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb268006f10 msgr2=0x7fb2880999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285400640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb268006f10 0x7fb2880999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285400640 1 -- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2880c9ba0 msgr2=0x7fb2880ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285400640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2880c9ba0 0x7fb2880ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285400640 1 -- 172.21.15.97:0/2372527882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880c65d0 con 0x7fb288148870 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285e00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb268006f10 0x7fb2880999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb286800640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2880c9ba0 0x7fb2880ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:41.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb285400640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 0x7fb28809aa20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e800 tx=0x7fb27000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000c8d0 con 0x7fb288148870 2024-01-31T05:02:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27000ca50 con 0x7fb288148870 2024-01-31T05:02:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2880c68b0 con 0x7fb288148870 2024-01-31T05:02:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000ce00 con 0x7fb288148870 2024-01-31T05:02:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.283+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288099f00 con 0x7fb288148870 2024-01-31T05:02:41.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.285+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2880c6dd0 con 0x7fb288148870 2024-01-31T05:02:41.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.286+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb27001d460 con 0x7fb288148870 2024-01-31T05:02:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.286+0000 7fb27e000640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb26006af00 0x7fb26006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.286+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb2700897a0 con 0x7fb288148870 2024-01-31T05:02:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.287+0000 7fb285e00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb26006af00 0x7fb26006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:41.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.287+0000 7fb285e00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb26006af00 0x7fb26006d3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb278007520 tx=0x7fb27800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:41.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.291+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb2880c6dd0 con 0x7fb288148870 2024-01-31T05:02:41.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.485+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7fb2880c6dd0 con 0x7fb288148870 2024-01-31T05:02:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.876+0000 7fb27e000640 1 -- 172.21.15.97:0/2372527882 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v41) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb2880c6dd0 con 0x7fb288148870 2024-01-31T05:02:41.875 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'data_new' created 2024-01-31T05:02:41.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.879+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb26006af00 msgr2=0x7fb26006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.879+0000 7fb28cc00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb26006af00 0x7fb26006d3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb278007520 tx=0x7fb27800b040 comp rx=0 tx=0).stop 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.879+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 msgr2=0x7fb28809aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.879+0000 7fb28cc00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 0x7fb28809aa20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb27000e800 tx=0x7fb27000ecc0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 shutdown_connections 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2880c9ba0 0x7fb2880ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb26006af00 0x7fb26006d3c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb288148870 0x7fb28809aa20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 --2- 172.21.15.97:0/2372527882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb268006f10 0x7fb2880999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 >> 172.21.15.97:0/2372527882 conn(0x7fb2880cede0 msgr2=0x7fb28806c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 shutdown_connections 2024-01-31T05:02:41.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:41.880+0000 7fb28cc00640 1 -- 172.21.15.97:0/2372527882 wait complete. 2024-01-31T05:02:41.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-01-31T05:02:42.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.093+0000 7fe65a200640 1 Processor -- start 2024-01-31T05:02:42.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.094+0000 7fe65a200640 1 -- start start 2024-01-31T05:02:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.095+0000 7fe65a200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe65415d450 0x7fe6540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.095+0000 7fe65a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6540d39f0 0x7fe6540cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.095+0000 7fe65a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6540d2190 con 0x7fe6540d1680 2024-01-31T05:02:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.095+0000 7fe65a200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6540b3060 con 0x7fe65415d450 2024-01-31T05:02:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.095+0000 7fe65a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540b2a30 con 0x7fe6540d39f0 2024-01-31T05:02:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.095+0000 7fe658e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe65415d450 0x7fe6540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.096+0000 7fe658e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe65415d450 0x7fe6540cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54496/0 (socket says 172.21.15.97:54496) 2024-01-31T05:02:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.096+0000 7fe658e00640 1 -- 172.21.15.97:0/2396923384 learned_addr learned my addr 172.21.15.97:0/2396923384 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.096+0000 7fe658e00640 1 -- 172.21.15.97:0/2396923384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6540d39f0 msgr2=0x7fe6540cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.096+0000 7fe658e00640 1 --2- 172.21.15.97:0/2396923384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6540d39f0 0x7fe6540cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.096+0000 7fe658e00640 1 -- 172.21.15.97:0/2396923384 >> v1:172.21.15.188:6789/0 conn(0x7fe6540d1680 legacy=0x7fe6540cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.096+0000 7fe658e00640 1 -- 172.21.15.97:0/2396923384 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540d1c30 con 0x7fe65415d450 2024-01-31T05:02:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.097+0000 7fe658e00640 1 --2- 172.21.15.97:0/2396923384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe65415d450 0x7fe6540cf930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009870 tx=0x7fe63c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917433c68232971 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe653400640 1 -- 172.21.15.97:0/2396923384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c031640 con 0x7fe65415d450 2024-01-31T05:02:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe653400640 1 -- 172.21.15.97:0/2396923384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c0317a0 con 0x7fe65415d450 2024-01-31T05:02:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe653400640 1 -- 172.21.15.97:0/2396923384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c031b30 con 0x7fe65415d450 2024-01-31T05:02:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe65a200640 1 -- 172.21.15.97:0/2396923384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe65415d450 msgr2=0x7fe6540cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe65a200640 1 --2- 172.21.15.97:0/2396923384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe65415d450 0x7fe6540cf930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009870 tx=0x7fe63c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe65a200640 1 -- 172.21.15.97:0/2396923384 shutdown_connections 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe65a200640 1 --2- 172.21.15.97:0/2396923384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6540d39f0 0x7fe6540cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe65a200640 1 --2- 172.21.15.97:0/2396923384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe65415d450 0x7fe6540cf930 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.098+0000 7fe65a200640 1 -- 172.21.15.97:0/2396923384 >> 172.21.15.97:0/2396923384 conn(0x7fe65409c400 msgr2=0x7fe6540d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.099+0000 7fe65a200640 1 -- 172.21.15.97:0/2396923384 shutdown_connections 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.099+0000 7fe65a200640 1 -- 172.21.15.97:0/2396923384 wait complete. 2024-01-31T05:02:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.100+0000 7fe65a200640 1 Processor -- start 2024-01-31T05:02:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.100+0000 7fe65a200640 1 -- start start 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.101+0000 7fe65a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6540d1680 0x7fe6540a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.101+0000 7fe65a200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 0x7fe65415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.101+0000 7fe65a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe65415d450 0x7fe6540c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.101+0000 7fe65a200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65415a0b0 con 0x7fe6540d39f0 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.101+0000 7fe65a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540ccf60 con 0x7fe6540d1680 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.101+0000 7fe65a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540ca340 con 0x7fe65415d450 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 0x7fe65415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 0x7fe65415a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54500/0 (socket says 172.21.15.97:54500) 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 -- 172.21.15.97:0/3242806649 learned_addr learned my addr 172.21.15.97:0/3242806649 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe658e00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6540d1680 0x7fe6540a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe659800640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe65415d450 0x7fe6540c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 -- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe65415d450 msgr2=0x7fe6540c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe65415d450 0x7fe6540c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 -- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6540d1680 msgr2=0x7fe6540a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6540d1680 0x7fe6540a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.102+0000 7fe653e00640 1 -- 172.21.15.97:0/3242806649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0094d0 con 0x7fe6540d39f0 2024-01-31T05:02:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.103+0000 7fe653e00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 0x7fe65415a220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe64400ba80 tx=0x7fe64400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.103+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64400cc00 con 0x7fe6540d39f0 2024-01-31T05:02:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.103+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64400cd80 con 0x7fe6540d39f0 2024-01-31T05:02:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.103+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe644014680 con 0x7fe6540d39f0 2024-01-31T05:02:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.103+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6540ca620 con 0x7fe6540d39f0 2024-01-31T05:02:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.103+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6540cabe0 con 0x7fe6540d39f0 2024-01-31T05:02:42.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.104+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6540cae00 con 0x7fe6540d39f0 2024-01-31T05:02:42.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.106+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe64401d3b0 con 0x7fe6540d39f0 2024-01-31T05:02:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.106+0000 7fe650c00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe62c06aed0 0x7fe62c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.107+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6392+0+0 (secure 0 0 0) 0x7fe644089950 con 0x7fe6540d39f0 2024-01-31T05:02:42.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.107+0000 7fe658e00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe62c06aed0 0x7fe62c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:42.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.107+0000 7fe658e00640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe62c06aed0 0x7fe62c06d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe63c036000 tx=0x7fe63c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:42.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.112+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe6440544d0 con 0x7fe6540d39f0 2024-01-31T05:02:42.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.344+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7fe6540ceec0 con 0x7fe6540d39f0 2024-01-31T05:02:42.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:02:42.814 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:02:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.888+0000 7fe650c00640 1 -- 172.21.15.97:0/3242806649 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v18) v1 ==== 131+0+0 (secure 0 0 0) 0x7fe6440190b0 con 0x7fe6540d39f0 2024-01-31T05:02:42.888 INFO:teuthology.orchestra.run.smithi097.stderr:added data pool 7 to fsmap 2024-01-31T05:02:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.892+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe62c06aed0 msgr2=0x7fe62c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.892+0000 7fe65a200640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe62c06aed0 0x7fe62c06d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe63c036000 tx=0x7fe63c03c040 comp rx=0 tx=0).stop 2024-01-31T05:02:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.892+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 msgr2=0x7fe65415a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:42.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.892+0000 7fe65a200640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 0x7fe65415a220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe64400ba80 tx=0x7fe64400bf40 comp rx=0 tx=0).stop 2024-01-31T05:02:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.893+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 shutdown_connections 2024-01-31T05:02:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.893+0000 7fe65a200640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe65415d450 0x7fe6540c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.893+0000 7fe65a200640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe62c06aed0 0x7fe62c06d390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.893+0000 7fe65a200640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6540d39f0 0x7fe65415a220 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.893+0000 7fe65a200640 1 --2- 172.21.15.97:0/3242806649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6540d1680 0x7fe6540a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:42.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.893+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 >> 172.21.15.97:0/3242806649 conn(0x7fe65409c400 msgr2=0x7fe65406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:42.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.895+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 shutdown_connections 2024-01-31T05:02:42.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:42.897+0000 7fe65a200640 1 -- 172.21.15.97:0/3242806649 wait complete. 2024-01-31T05:02:42.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:02:43.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.101+0000 7f6427200640 1 Processor -- start 2024-01-31T05:02:43.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.101+0000 7f6427200640 1 -- start start 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6427200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f642815d4e0 0x7f64280b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6427200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64280d1c70 0x7f64280b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6427200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642815d8c0 con 0x7f64280d0d60 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6427200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280b0b20 con 0x7f64280d1c70 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6427200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64280b22f0 con 0x7f642815d4e0 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64280d1c70 0x7f64280b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f64280d1c70 0x7f64280b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54508/0 (socket says 172.21.15.97:54508) 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 -- 172.21.15.97:0/4070168132 learned_addr learned my addr 172.21.15.97:0/4070168132 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f641fe00640 1 --1- 172.21.15.97:0/4070168132 >> v1:172.21.15.97:6789/0 conn(0x7f64280d0d60 0x7f64280d1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58496/0 (socket says 172.21.15.97:58496) 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6425e00640 1 --2- 172.21.15.97:0/4070168132 >> v2:172.21.15.188:3300/0 conn(0x7f642815d4e0 0x7f64280b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 -- 172.21.15.97:0/4070168132 >> v2:172.21.15.188:3300/0 conn(0x7f642815d4e0 msgr2=0x7f64280b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 --2- 172.21.15.97:0/4070168132 >> v2:172.21.15.188:3300/0 conn(0x7f642815d4e0 0x7f64280b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 -- 172.21.15.97:0/4070168132 >> v1:172.21.15.97:6789/0 conn(0x7f64280d0d60 legacy=0x7f64280d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6426800640 1 -- 172.21.15.97:0/4070168132 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280b1920 con 0x7f64280d1c70 2024-01-31T05:02:43.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.103+0000 7f6425e00640 1 --2- 172.21.15.97:0/4070168132 >> v2:172.21.15.188:3300/0 conn(0x7f642815d4e0 0x7f64280b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6426800640 1 --2- 172.21.15.97:0/4070168132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64280d1c70 0x7f64280b1db0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6428074590 tx=0x7f642002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516ade4db8a74503 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:43.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6425400640 1 -- 172.21.15.97:0/4070168132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642002cbb0 con 0x7f64280d1c70 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6425400640 1 -- 172.21.15.97:0/4070168132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642002cd10 con 0x7f64280d1c70 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6425400640 1 -- 172.21.15.97:0/4070168132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420034610 con 0x7f64280d1c70 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6427200640 1 -- 172.21.15.97:0/4070168132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64280d1c70 msgr2=0x7f64280b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6427200640 1 --2- 172.21.15.97:0/4070168132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64280d1c70 0x7f64280b1db0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6428074590 tx=0x7f642002c010 comp rx=0 tx=0).stop 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6427200640 1 -- 172.21.15.97:0/4070168132 shutdown_connections 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6427200640 1 --2- 172.21.15.97:0/4070168132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64280d1c70 0x7f64280b1db0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.105+0000 7f6427200640 1 --2- 172.21.15.97:0/4070168132 >> v2:172.21.15.188:3300/0 conn(0x7f642815d4e0 0x7f64280b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.106+0000 7f6427200640 1 -- 172.21.15.97:0/4070168132 >> 172.21.15.97:0/4070168132 conn(0x7f64280d3010 msgr2=0x7f64280d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.106+0000 7f6427200640 1 -- 172.21.15.97:0/4070168132 shutdown_connections 2024-01-31T05:02:43.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.106+0000 7f6427200640 1 -- 172.21.15.97:0/4070168132 wait complete. 2024-01-31T05:02:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.107+0000 7f6427200640 1 Processor -- start 2024-01-31T05:02:43.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.107+0000 7f6427200640 1 -- start start 2024-01-31T05:02:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.108+0000 7f6427200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64280d0d60 0x7f6428152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.108+0000 7f6427200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64280d1c70 0x7f64280cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.108+0000 7f6427200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 0x7f64280ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.108+0000 7f6427200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642815b890 con 0x7f642815d4e0 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.108+0000 7f6427200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64280b2430 con 0x7f64280d1c70 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.108+0000 7f6427200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64280ccd00 con 0x7f64280d0d60 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 0x7f64280ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 0x7f64280ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54518/0 (socket says 172.21.15.97:54518) 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 -- 172.21.15.97:0/1600724092 learned_addr learned my addr 172.21.15.97:0/1600724092 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6425e00640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64280d0d60 0x7f6428152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f641fe00640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64280d1c70 0x7f64280cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 -- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64280d0d60 msgr2=0x7f6428152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64280d0d60 0x7f6428152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 -- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64280d1c70 msgr2=0x7f64280cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64280d1c70 0x7f64280cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 -- 172.21.15.97:0/1600724092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420009b30 con 0x7f642815d4e0 2024-01-31T05:02:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6425e00640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64280d0d60 0x7f6428152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f641fe00640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64280d1c70 0x7f64280cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:43.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.109+0000 7f6426800640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 0x7f64280ce060 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6420036040 tx=0x7f6420004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.110+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6420009510 con 0x7f642815d4e0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.110+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642003e450 con 0x7f642815d4e0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.110+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642003e800 con 0x7f642815d4e0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.110+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64280b26f0 con 0x7f642815d4e0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.110+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64280b2cb0 con 0x7f642815d4e0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.111+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64280676b0 con 0x7f642815d4e0 2024-01-31T05:02:43.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.112+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f642003e960 con 0x7f642815d4e0 2024-01-31T05:02:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.113+0000 7f641e000640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f640006aed0 0x7f640006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.113+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f64200ac720 con 0x7f642815d4e0 2024-01-31T05:02:43.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.113+0000 7f6425e00640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f640006aed0 0x7f640006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.114+0000 7f6425e00640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f640006aed0 0x7f640006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6418006fd0 tx=0x7f6418008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:43.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.118+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6420077460 con 0x7f642815d4e0 2024-01-31T05:02:43.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.348+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f642806ae60 con 0x7f642815d4e0 2024-01-31T05:02:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.348+0000 7f641e000640 1 -- 172.21.15.97:0/1600724092 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f642007aa80 con 0x7f642815d4e0 2024-01-31T05:02:43.348 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:43.348 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:42.346423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:02:43.348 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f640006aed0 msgr2=0x7f640006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f640006aed0 0x7f640006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6418006fd0 tx=0x7f6418008040 comp rx=0 tx=0).stop 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 msgr2=0x7f64280ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 0x7f64280ce060 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6420036040 tx=0x7f6420004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 shutdown_connections 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f640006aed0 0x7f640006d390 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f642815d4e0 0x7f64280ce060 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64280d1c70 0x7f64280cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 --2- 172.21.15.97:0/1600724092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64280d0d60 0x7f6428152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 >> 172.21.15.97:0/1600724092 conn(0x7f64280d3010 msgr2=0x7f642806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 shutdown_connections 2024-01-31T05:02:43.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.351+0000 7f6427200640 1 -- 172.21.15.97:0/1600724092 wait complete. 2024-01-31T05:02:43.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:43.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.585+0000 7fd0d9600640 1 Processor -- start 2024-01-31T05:02:43.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.585+0000 7fd0d9600640 1 -- start start 2024-01-31T05:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.586+0000 7fd0d9600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0d415f400 0x7fd0d415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.586+0000 7fd0d9600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0d415ecd0 0x7fd0d40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.586+0000 7fd0d9600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d40cc940 con 0x7fd0d40b25b0 2024-01-31T05:02:43.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.586+0000 7fd0d9600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d40ccc20 con 0x7fd0d415ecd0 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.586+0000 7fd0d9600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d40ccf00 con 0x7fd0d415f400 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.586+0000 7fd0d3400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0d415ecd0 0x7fd0d40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d3400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0d415ecd0 0x7fd0d40cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54530/0 (socket says 172.21.15.97:54530) 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d2000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0d40b25b0 0x7fd0d415e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58498/0 (socket says 172.21.15.97:58498) 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d3400640 1 -- 172.21.15.97:0/280497797 learned_addr learned my addr 172.21.15.97:0/280497797 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d2a00640 1 --2- 172.21.15.97:0/280497797 >> v2:172.21.15.188:3300/0 conn(0x7fd0d415f400 0x7fd0d415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d3400640 1 -- 172.21.15.97:0/280497797 >> v2:172.21.15.188:3300/0 conn(0x7fd0d415f400 msgr2=0x7fd0d415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d3400640 1 --2- 172.21.15.97:0/280497797 >> v2:172.21.15.188:3300/0 conn(0x7fd0d415f400 0x7fd0d415f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d3400640 1 -- 172.21.15.97:0/280497797 >> v1:172.21.15.97:6789/0 conn(0x7fd0d40b25b0 legacy=0x7fd0d415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.587+0000 7fd0d3400640 1 -- 172.21.15.97:0/280497797 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d40b09d0 con 0x7fd0d415ecd0 2024-01-31T05:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.588+0000 7fd0d3400640 1 --2- 172.21.15.97:0/280497797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415ecd0 0x7fd0d40cf030 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4067260 tx=0x7fd0cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f03e6b9165740 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.588+0000 7fd0d1600640 1 -- 172.21.15.97:0/280497797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc030860 con 0x7fd0d415ecd0 2024-01-31T05:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d1600640 1 -- 172.21.15.97:0/280497797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc0309c0 con 0x7fd0d415ecd0 2024-01-31T05:02:43.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d1600640 1 -- 172.21.15.97:0/280497797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc030d50 con 0x7fd0d415ecd0 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 -- 172.21.15.97:0/280497797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415ecd0 msgr2=0x7fd0d40cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 --2- 172.21.15.97:0/280497797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415ecd0 0x7fd0d40cf030 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4067260 tx=0x7fd0cc02b9a0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 -- 172.21.15.97:0/280497797 shutdown_connections 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 --2- 172.21.15.97:0/280497797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415ecd0 0x7fd0d40cf030 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 --2- 172.21.15.97:0/280497797 >> v2:172.21.15.188:3300/0 conn(0x7fd0d415f400 0x7fd0d415f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 -- 172.21.15.97:0/280497797 >> 172.21.15.97:0/280497797 conn(0x7fd0d40b2b90 msgr2=0x7fd0d40b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 -- 172.21.15.97:0/280497797 shutdown_connections 2024-01-31T05:02:43.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.589+0000 7fd0d9600640 1 -- 172.21.15.97:0/280497797 wait complete. 2024-01-31T05:02:43.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.591+0000 7fd0d9600640 1 Processor -- start 2024-01-31T05:02:43.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.591+0000 7fd0d9600640 1 -- start start 2024-01-31T05:02:43.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d40b25b0 0x7fd0d40c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d415ecd0 0x7fd0d40c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 0x7fd0d406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d9600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40c9ba0 con 0x7fd0d415f400 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d9600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40c9e80 con 0x7fd0d40b25b0 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40c69a0 con 0x7fd0d415ecd0 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 0x7fd0d406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.592+0000 7fd0d3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 0x7fd0d406b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54536/0 (socket says 172.21.15.97:54536) 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 -- 172.21.15.97:0/3792394671 learned_addr learned my addr 172.21.15.97:0/3792394671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d2000640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d415ecd0 0x7fd0d40c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d2a00640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d40b25b0 0x7fd0d40c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 -- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d415ecd0 msgr2=0x7fd0d40c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d415ecd0 0x7fd0d40c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 -- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d40b25b0 msgr2=0x7fd0d40c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d40b25b0 0x7fd0d40c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 -- 172.21.15.97:0/3792394671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0094d0 con 0x7fd0d415f400 2024-01-31T05:02:43.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.593+0000 7fd0d3400640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 0x7fd0d406b820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0090b0 tx=0x7fd0cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:43.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.594+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc039a90 con 0x7fd0d415f400 2024-01-31T05:02:43.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.594+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc039bf0 con 0x7fd0d415f400 2024-01-31T05:02:43.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.594+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc043410 con 0x7fd0d415f400 2024-01-31T05:02:43.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.594+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d40c6c80 con 0x7fd0d415f400 2024-01-31T05:02:43.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.594+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d415b010 con 0x7fd0d415f400 2024-01-31T05:02:43.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.595+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d40c6800 con 0x7fd0d415f400 2024-01-31T05:02:43.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.596+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd0cc04c3b0 con 0x7fd0d415f400 2024-01-31T05:02:43.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.597+0000 7fd0caa00640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0ac06aed0 0x7fd0ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:43.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.597+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fd0cc0abee0 con 0x7fd0d415f400 2024-01-31T05:02:43.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.597+0000 7fd0d2a00640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0ac06aed0 0x7fd0ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:43.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.598+0000 7fd0d2a00640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0ac06aed0 0x7fd0ac06d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006fd0 tx=0x7fd0c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:43.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.602+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd0d40c6800 con 0x7fd0d415f400 2024-01-31T05:02:43.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.796+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0d40c6800 con 0x7fd0d415f400 2024-01-31T05:02:43.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.799+0000 7fd0caa00640 1 -- 172.21.15.97:0/3792394671 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18563 (secure 0 0 0) 0x7fd0d40c6800 con 0x7fd0d415f400 2024-01-31T05:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:43.799 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":42,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:42.346174+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0ac06aed0 msgr2=0x7fd0ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0ac06aed0 0x7fd0ac06d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006fd0 tx=0x7fd0c4008040 comp rx=0 tx=0).stop 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 msgr2=0x7fd0d406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 0x7fd0d406b820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc0090b0 tx=0x7fd0cc004730 comp rx=0 tx=0).stop 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 shutdown_connections 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd0ac06aed0 0x7fd0ac06d390 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0d415f400 0x7fd0d406b820 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0d415ecd0 0x7fd0d40c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 --2- 172.21.15.97:0/3792394671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0d40b25b0 0x7fd0d40c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 >> 172.21.15.97:0/3792394671 conn(0x7fd0d40b2b90 msgr2=0x7fd0d406e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 shutdown_connections 2024-01-31T05:02:43.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:43.802+0000 7fd0d9600640 1 -- 172.21.15.97:0/3792394671 wait complete. 2024-01-31T05:02:43.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:43.821 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-01-31T05:02:43.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:43.886 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-01-31T05:02:43.945 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-01-31T05:02:44.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.153+0000 7f1194000640 1 Processor -- start 2024-01-31T05:02:44.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.154+0000 7f1194000640 1 -- start start 2024-01-31T05:02:44.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.155+0000 7f1194000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f118c0cbf20 0x7f118c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.155+0000 7f1194000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c06cd20 con 0x7f118c0abcf0 2024-01-31T05:02:44.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.155+0000 7f1194000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c0ceb60 con 0x7f118c15b470 2024-01-31T05:02:44.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.155+0000 7f1194000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c15b850 con 0x7f118c0cbf20 2024-01-31T05:02:44.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f1192000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f118c0abcf0 0x7f118c06d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58512/0 (socket says 172.21.15.97:58512) 2024-01-31T05:02:44.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f1192000640 1 -- 172.21.15.97:0/2752886807 learned_addr learned my addr 172.21.15.97:0/2752886807 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f1190c00640 1 --2- 172.21.15.97:0/2752886807 >> v2:172.21.15.188:3300/0 conn(0x7f118c0cbf20 0x7f118c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999572722 0 0) 0x7f118c06cd20 con 0x7f118c0abcf0 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f116c003520 con 0x7f118c0abcf0 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674252173 0 0) 0x7f116c003520 con 0x7f118c0abcf0 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 >> v2:172.21.15.188:3300/0 conn(0x7f118c0cbf20 msgr2=0x7f118c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 --2- 172.21.15.97:0/2752886807 >> v2:172.21.15.188:3300/0 conn(0x7f118c0cbf20 0x7f118c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 >> v1:172.21.15.188:6789/0 conn(0x7f118c15b470 legacy=0x7f118c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c06c3f0 con 0x7f118c0abcf0 2024-01-31T05:02:44.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f1184001670 con 0x7f118c0abcf0 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 >> v1:172.21.15.97:6789/0 conn(0x7f118c0abcf0 legacy=0x7f118c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c003970 0x7f116c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0047c0 0x7f116c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.156+0000 7f118be00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c005180 0x7f116c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c15b850 con 0x7f116c0047c0 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c00c000 con 0x7f116c003970 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c00c1c0 con 0x7f116c005180 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1190c00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0047c0 0x7f116c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1192000640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c005180 0x7f116c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1190c00640 1 -- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c005180 msgr2=0x7f116c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1190c00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c005180 0x7f116c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1190c00640 1 -- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c003970 msgr2=0x7f116c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1190c00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c003970 0x7f116c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1190c00640 1 -- 172.21.15.97:0/2752886807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c003520 con 0x7f116c0047c0 2024-01-31T05:02:44.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.157+0000 7f1192000640 1 -- 172.21.15.97:0/2752886807 reap_dead start 2024-01-31T05:02:44.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.158+0000 7f1190c00640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0047c0 0x7f116c004c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ac0 tx=0x7f116c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c6a929c1ff2011 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117402c6b0 con 0x7f116c0047c0 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117402c810 con 0x7f116c0047c0 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f118be00640 1 -- 172.21.15.97:0/2752886807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117402cba0 con 0x7f116c0047c0 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 -- 172.21.15.97:0/2752886807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0047c0 msgr2=0x7f116c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0047c0 0x7f116c004c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1174009ac0 tx=0x7f116c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 -- 172.21.15.97:0/2752886807 shutdown_connections 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 --2- 172.21.15.97:0/2752886807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0047c0 0x7f116c004c40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 -- 172.21.15.97:0/2752886807 >> 172.21.15.97:0/2752886807 conn(0x7f118c0b0a10 msgr2=0x7f118c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 -- 172.21.15.97:0/2752886807 shutdown_connections 2024-01-31T05:02:44.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.159+0000 7f1194000640 1 -- 172.21.15.97:0/2752886807 wait complete. 2024-01-31T05:02:44.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.161+0000 7f1194000640 1 Processor -- start 2024-01-31T05:02:44.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.161+0000 7f1194000640 1 -- start start 2024-01-31T05:02:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1194000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c0047c0 0x7f118c09aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1194000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 0x7f118c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1194000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f118c152e20 0x7f118c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1194000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c09a640 con 0x7f118c1529a0 2024-01-31T05:02:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1194000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0999c0 con 0x7f118c152e20 2024-01-31T05:02:44.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1194000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c099ca0 con 0x7f116c0047c0 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1190c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 0x7f118c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1190c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 0x7f118c06d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54540/0 (socket says 172.21.15.97:54540) 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.162+0000 7f1190c00640 1 -- 172.21.15.97:0/506933533 learned_addr learned my addr 172.21.15.97:0/506933533 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1191600640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c0047c0 0x7f118c09aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1192000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f118c152e20 0x7f118c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1190c00640 1 -- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c0047c0 msgr2=0x7f118c09aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1190c00640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c0047c0 0x7f118c09aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1190c00640 1 -- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f118c152e20 msgr2=0x7f118c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1190c00640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f118c152e20 0x7f118c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1190c00640 1 -- 172.21.15.97:0/506933533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c099f80 con 0x7f118c1529a0 2024-01-31T05:02:44.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1192000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f118c152e20 0x7f118c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1190c00640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 0x7f118c06d1b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1174000c00 tx=0x7f1174004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:44.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174003810 con 0x7f118c1529a0 2024-01-31T05:02:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1174003970 con 0x7f118c1529a0 2024-01-31T05:02:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.163+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174003d20 con 0x7f118c1529a0 2024-01-31T05:02:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.164+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c09a260 con 0x7f118c1529a0 2024-01-31T05:02:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.164+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c09b270 con 0x7f118c1529a0 2024-01-31T05:02:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.165+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0cb650 con 0x7f118c1529a0 2024-01-31T05:02:44.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.166+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1174034440 con 0x7f118c1529a0 2024-01-31T05:02:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.167+0000 7f1189600640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f116406aed0 0x7f116406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.167+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f11740ab930 con 0x7f118c1529a0 2024-01-31T05:02:44.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.167+0000 7f1191600640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f116406aed0 0x7f116406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.168+0000 7f1191600640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f116406aed0 0x7f116406d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f117c005540 tx=0x7f117c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:44.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.172+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1174076660 con 0x7f118c1529a0 2024-01-31T05:02:44.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.434+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f118c067830 con 0x7f118c1529a0 2024-01-31T05:02:44.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.439+0000 7f1189600640 1 -- 172.21.15.97:0/506933533 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v12) v1 ==== 210+0+167 (secure 0 0 0) 0x7f11740044e0 con 0x7f118c1529a0 2024-01-31T05:02:44.439 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:02:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:02:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow rw" 2024-01-31T05:02:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:02:44.439 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-01-31T05:02:44.439 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f116406aed0 msgr2=0x7f116406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f116406aed0 0x7f116406d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f117c005540 tx=0x7f117c009040 comp rx=0 tx=0).stop 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 msgr2=0x7f118c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 0x7f118c06d1b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1174000c00 tx=0x7f1174004730 comp rx=0 tx=0).stop 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 shutdown_connections 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f118c152e20 0x7f118c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f116406aed0 0x7f116406d390 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f118c1529a0 0x7f118c06d1b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 --2- 172.21.15.97:0/506933533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f116c0047c0 0x7f118c09aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.442+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 >> 172.21.15.97:0/506933533 conn(0x7f118c0b0a10 msgr2=0x7f118c0cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.443+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 shutdown_connections 2024-01-31T05:02:44.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.443+0000 7f1194000640 1 -- 172.21.15.97:0/506933533 wait complete. 2024-01-31T05:02:44.461 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:02:44.461 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:44.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:44.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.758+0000 7f1a28200640 1 Processor -- start 2024-01-31T05:02:44.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.759+0000 7f1a28200640 1 -- start start 2024-01-31T05:02:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a28200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a200cb810 0x7f1a200cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a28200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a200ccd00 0x7f1a200cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a28200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2006d1a0 con 0x7f1a2015b400 2024-01-31T05:02:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a28200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a200cc960 con 0x7f1a200cb810 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a28200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a200cd6f0 con 0x7f1a200ccd00 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a25800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a200cb810 0x7f1a200cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a25800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a200cb810 0x7f1a200cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54548/0 (socket says 172.21.15.97:54548) 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.760+0000 7f1a25800640 1 -- 172.21.15.97:0/2384410759 learned_addr learned my addr 172.21.15.97:0/2384410759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.761+0000 7f1a25800640 1 -- 172.21.15.97:0/2384410759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a200ccd00 msgr2=0x7f1a200cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.761+0000 7f1a25800640 1 --2- 172.21.15.97:0/2384410759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a200ccd00 0x7f1a200cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.761+0000 7f1a25800640 1 -- 172.21.15.97:0/2384410759 >> v1:172.21.15.188:6789/0 conn(0x7f1a2015b400 legacy=0x7f1a2006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:44.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.761+0000 7f1a25800640 1 -- 172.21.15.97:0/2384410759 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2006c600 con 0x7f1a200cb810 2024-01-31T05:02:44.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.762+0000 7f1a25800640 1 --2- 172.21.15.97:0/2384410759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a200cbbf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009e50 tx=0x7f1a1002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76edc6da0da7916 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:44.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.762+0000 7f1a1fe00640 1 -- 172.21.15.97:0/2384410759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1002ca80 con 0x7f1a200cb810 2024-01-31T05:02:44.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.762+0000 7f1a1fe00640 1 -- 172.21.15.97:0/2384410759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1002cbe0 con 0x7f1a200cb810 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.762+0000 7f1a1fe00640 1 -- 172.21.15.97:0/2384410759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a100343f0 con 0x7f1a200cb810 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 -- 172.21.15.97:0/2384410759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 msgr2=0x7f1a200cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 --2- 172.21.15.97:0/2384410759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a200cbbf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009e50 tx=0x7f1a1002bee0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 -- 172.21.15.97:0/2384410759 shutdown_connections 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 --2- 172.21.15.97:0/2384410759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a200ccd00 0x7f1a200cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 --2- 172.21.15.97:0/2384410759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a200cbbf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 -- 172.21.15.97:0/2384410759 >> 172.21.15.97:0/2384410759 conn(0x7f1a200b0a10 msgr2=0x7f1a200b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 -- 172.21.15.97:0/2384410759 shutdown_connections 2024-01-31T05:02:44.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.763+0000 7f1a28200640 1 -- 172.21.15.97:0/2384410759 wait complete. 2024-01-31T05:02:44.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.765+0000 7f1a28200640 1 Processor -- start 2024-01-31T05:02:44.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.765+0000 7f1a28200640 1 -- start start 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a28200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a201f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a28200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a200ccd00 0x7f1a20099de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a28200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2015b400 0x7f1a2015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a28200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2015d970 con 0x7f1a200cb810 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a28200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2015cd80 con 0x7f1a200ccd00 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a28200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2015d040 con 0x7f1a2015b400 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a201f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a201f04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54552/0 (socket says 172.21.15.97:54552) 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 -- 172.21.15.97:0/474645897 learned_addr learned my addr 172.21.15.97:0/474645897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:44.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 -- 172.21.15.97:0/474645897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2015b400 msgr2=0x7f1a2015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2015b400 0x7f1a2015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 -- 172.21.15.97:0/474645897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a200ccd00 msgr2=0x7f1a20099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a200ccd00 0x7f1a20099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.766+0000 7f1a25800640 1 -- 172.21.15.97:0/474645897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10009b30 con 0x7f1a200cb810 2024-01-31T05:02:44.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.767+0000 7f1a25800640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a201f04e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009e50 tx=0x7f1a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.767+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1003e450 con 0x7f1a200cb810 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.767+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1003e5d0 con 0x7f1a200cb810 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.767+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1003e980 con 0x7f1a200cb810 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.767+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2015d320 con 0x7f1a200cb810 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.767+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a20099b10 con 0x7f1a200cb810 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.768+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a200677c0 con 0x7f1a200cb810 2024-01-31T05:02:44.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.769+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1a1002cd50 con 0x7f1a200cb810 2024-01-31T05:02:44.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.770+0000 7f1a1d600640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f19f806aed0 0x7f19f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:44.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.770+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f1a100ac610 con 0x7f1a200cb810 2024-01-31T05:02:44.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.770+0000 7f1a24e00640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f19f806aed0 0x7f19f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:44.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.771+0000 7f1a24e00640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f19f806aed0 0x7f19f806d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a200af500 tx=0x7f1a08006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:44.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.775+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1a10077170 con 0x7f1a200cb810 2024-01-31T05:02:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.954+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a200ada10 con 0x7f1a200cb810 2024-01-31T05:02:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.956+0000 7f1a1d600640 1 -- 172.21.15.97:0/474645897 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f1a1007a790 con 0x7f1a200cb810 2024-01-31T05:02:44.956 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:44.956 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f19f806aed0 msgr2=0x7f19f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f19f806aed0 0x7f19f806d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a200af500 tx=0x7f1a08006ba0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 msgr2=0x7f1a201f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a201f04e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009e50 tx=0x7f1a10004a40 comp rx=0 tx=0).stop 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 shutdown_connections 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a2015b400 0x7f1a2015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a200ccd00 0x7f1a20099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f19f806aed0 0x7f19f806d390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 --2- 172.21.15.97:0/474645897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a200cb810 0x7f1a201f04e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.958+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 >> 172.21.15.97:0/474645897 conn(0x7f1a200b0a10 msgr2=0x7f1a200ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.959+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 shutdown_connections 2024-01-31T05:02:44.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:44.959+0000 7f1a28200640 1 -- 172.21.15.97:0/474645897 wait complete. 2024-01-31T05:02:44.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:02:44.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:45.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.181+0000 7f6b38a00640 1 Processor -- start 2024-01-31T05:02:45.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.181+0000 7f6b38a00640 1 -- start start 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b38a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b300cbf20 0x7f6b300722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b38a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b300abcf0 0x7f6b300ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b38a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3006cd20 con 0x7f6b3015b470 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b38a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300ce970 con 0x7f6b300abcf0 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b38a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3015b850 con 0x7f6b300cbf20 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b36000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6b3015b470 0x7f6b300cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56058/0 (socket says 172.21.15.97:56058) 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b36a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b300abcf0 0x7f6b300ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.183+0000 7f6b36000640 1 -- 172.21.15.97:0/4237567249 learned_addr learned my addr 172.21.15.97:0/4237567249 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:45.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b35600640 1 --2- 172.21.15.97:0/4237567249 >> v2:172.21.15.188:3300/0 conn(0x7f6b300cbf20 0x7f6b300722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b36a00640 1 -- 172.21.15.97:0/4237567249 >> v2:172.21.15.188:3300/0 conn(0x7f6b300cbf20 msgr2=0x7f6b300722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:45.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b36a00640 1 --2- 172.21.15.97:0/4237567249 >> v2:172.21.15.188:3300/0 conn(0x7f6b300cbf20 0x7f6b300722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b36a00640 1 -- 172.21.15.97:0/4237567249 >> v1:172.21.15.97:6789/0 conn(0x7f6b3015b470 legacy=0x7f6b300cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b36a00640 1 -- 172.21.15.97:0/4237567249 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300ceb60 con 0x7f6b300abcf0 2024-01-31T05:02:45.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b34c00640 1 -- 172.21.15.97:0/4237567249 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386890504 0 0) 0x7f6b3006cd20 con 0x7f6b3015b470 2024-01-31T05:02:45.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.184+0000 7f6b35600640 1 --2- 172.21.15.97:0/4237567249 >> v2:172.21.15.188:3300/0 conn(0x7f6b300cbf20 0x7f6b300722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:45.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.185+0000 7f6b36a00640 1 --2- 172.21.15.97:0/4237567249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b300ce430 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6b30072d90 tx=0x7f6b2802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8056bb337adf84ed server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.185+0000 7f6b34c00640 1 -- 172.21.15.97:0/4237567249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2802cc60 con 0x7f6b300abcf0 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.185+0000 7f6b34c00640 1 -- 172.21.15.97:0/4237567249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2802cdc0 con 0x7f6b300abcf0 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.185+0000 7f6b34c00640 1 -- 172.21.15.97:0/4237567249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28034660 con 0x7f6b300abcf0 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 -- 172.21.15.97:0/4237567249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 msgr2=0x7f6b300ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 --2- 172.21.15.97:0/4237567249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b300ce430 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6b30072d90 tx=0x7f6b2802beb0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 -- 172.21.15.97:0/4237567249 shutdown_connections 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 --2- 172.21.15.97:0/4237567249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b300ce430 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 --2- 172.21.15.97:0/4237567249 >> v2:172.21.15.188:3300/0 conn(0x7f6b300cbf20 0x7f6b300722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 -- 172.21.15.97:0/4237567249 >> 172.21.15.97:0/4237567249 conn(0x7f6b300b0a10 msgr2=0x7f6b300b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:45.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 -- 172.21.15.97:0/4237567249 shutdown_connections 2024-01-31T05:02:45.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.186+0000 7f6b38a00640 1 -- 172.21.15.97:0/4237567249 wait complete. 2024-01-31T05:02:45.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.188+0000 7f6b38a00640 1 Processor -- start 2024-01-31T05:02:45.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.188+0000 7f6b38a00640 1 -- start start 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b38a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b3009c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b38a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b300cbf20 0x7f6b30147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b38a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b3015b470 0x7f6b30148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b38a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3009b4c0 con 0x7f6b300abcf0 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b38a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3009ca10 con 0x7f6b3015b470 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b38a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3009a560 con 0x7f6b300cbf20 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b36000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b3009c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b36000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b3009c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35574/0 (socket says 172.21.15.97:35574) 2024-01-31T05:02:45.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.189+0000 7f6b36000640 1 -- 172.21.15.97:0/985529687 learned_addr learned my addr 172.21.15.97:0/985529687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:45.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b3015b470 0x7f6b30148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36000640 1 -- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b300cbf20 msgr2=0x7f6b30147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36000640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b300cbf20 0x7f6b30147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36000640 1 -- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b3015b470 msgr2=0x7f6b30148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36000640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b3015b470 0x7f6b30148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36000640 1 -- 172.21.15.97:0/985529687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28009a70 con 0x7f6b300abcf0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b36000640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b3009c4d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6b20001eb0 tx=0x7f6b2000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200074e0 con 0x7f6b300abcf0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20007640 con 0x7f6b300abcf0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3009a840 con 0x7f6b300abcf0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.190+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b200059e0 con 0x7f6b300abcf0 2024-01-31T05:02:45.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.191+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3009ae00 con 0x7f6b300abcf0 2024-01-31T05:02:45.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.192+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3006d190 con 0x7f6b300abcf0 2024-01-31T05:02:45.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.192+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6b20005b40 con 0x7f6b300abcf0 2024-01-31T05:02:45.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.194+0000 7f6b2e000640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b0806aed0 0x7f6b0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.194+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f6b20088ef0 con 0x7f6b300abcf0 2024-01-31T05:02:45.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.194+0000 7f6b35600640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b0806aed0 0x7f6b0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.195+0000 7f6b35600640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b0806aed0 0x7f6b0806d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006fd0 tx=0x7f6b18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:45.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.198+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6b3006d190 con 0x7f6b300abcf0 2024-01-31T05:02:45.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.391+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b3006d190 con 0x7f6b300abcf0 2024-01-31T05:02:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.392+0000 7f6b2e000640 1 -- 172.21.15.97:0/985529687 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b3006d190 con 0x7f6b300abcf0 2024-01-31T05:02:45.392 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:45.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b0806aed0 msgr2=0x7f6b0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b0806aed0 0x7f6b0806d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006fd0 tx=0x7f6b18008040 comp rx=0 tx=0).stop 2024-01-31T05:02:45.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 msgr2=0x7f6b3009c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b3009c4d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6b20001eb0 tx=0x7f6b2000c890 comp rx=0 tx=0).stop 2024-01-31T05:02:45.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 shutdown_connections 2024-01-31T05:02:45.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b3015b470 0x7f6b30148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b300cbf20 0x7f6b30147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6b0806aed0 0x7f6b0806d390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 --2- 172.21.15.97:0/985529687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b300abcf0 0x7f6b3009c4d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 >> 172.21.15.97:0/985529687 conn(0x7f6b300b0a10 msgr2=0x7f6b300cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 shutdown_connections 2024-01-31T05:02:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.395+0000 7f6b38a00640 1 -- 172.21.15.97:0/985529687 wait complete. 2024-01-31T05:02:45.411 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:45.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.621+0000 7fa1a8000640 1 Processor -- start 2024-01-31T05:02:45.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.621+0000 7fa1a8000640 1 -- start start 2024-01-31T05:02:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a8000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1a015a350 0x7fa1a00abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a8000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a015abc0 con 0x7fa1a015a730 2024-01-31T05:02:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a8000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a015aea0 con 0x7fa1a00cde10 2024-01-31T05:02:45.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a8000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a00b2c90 con 0x7fa1a015a350 2024-01-31T05:02:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a4c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa1a015a730 0x7fa1a006da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56072/0 (socket says 172.21.15.97:56072) 2024-01-31T05:02:45.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a4c00640 1 -- 172.21.15.97:0/2059878163 learned_addr learned my addr 172.21.15.97:0/2059878163 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa1a5600640 1 --2- 172.21.15.97:0/2059878163 >> v2:172.21.15.188:3300/0 conn(0x7fa1a015a350 0x7fa1a00abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.623+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382066812 0 0) 0x7fa1a015abc0 con 0x7fa1a015a730 2024-01-31T05:02:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa180003520 con 0x7fa1a015a730 2024-01-31T05:02:45.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152872462 0 0) 0x7fa180003520 con 0x7fa1a015a730 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 >> v2:172.21.15.188:3300/0 conn(0x7fa1a015a350 msgr2=0x7fa1a00abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 --2- 172.21.15.97:0/2059878163 >> v2:172.21.15.188:3300/0 conn(0x7fa1a015a350 0x7fa1a00abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 >> v1:172.21.15.188:6789/0 conn(0x7fa1a00cde10 legacy=0x7fa1a00cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a015b180 con 0x7fa1a015a730 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa188001670 con 0x7fa1a015a730 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 >> v1:172.21.15.97:6789/0 conn(0x7fa1a015a730 legacy=0x7fa1a006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800039c0 0x7fa180003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1800051d0 0x7fa18000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00b2c90 con 0x7fa1800047c0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000c100 con 0x7fa1800039c0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18000c2c0 con 0x7fa1800051d0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.624+0000 7fa1a4c00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a5600640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800039c0 0x7fa180003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a6000640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1800051d0 0x7fa18000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a4c00640 1 -- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1800051d0 msgr2=0x7fa18000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a4c00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1800051d0 0x7fa18000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a4c00640 1 -- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800039c0 msgr2=0x7fa180003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a4c00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800039c0 0x7fa180003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a4c00640 1 -- 172.21.15.97:0/2059878163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180003520 con 0x7fa1800047c0 2024-01-31T05:02:45.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a5600640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800039c0 0x7fa180003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a6000640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1800051d0 0x7fa18000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.625+0000 7fa1a6000640 1 -- 172.21.15.97:0/2059878163 reap_dead start 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.626+0000 7fa1a4c00640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c160 tx=0x7fa1800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89763cc13d7286 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.626+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188005ad0 con 0x7fa1800047c0 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.626+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188005c30 con 0x7fa1800047c0 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.626+0000 7fa19fe00640 1 -- 172.21.15.97:0/2059878163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18802c7c0 con 0x7fa1800047c0 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 -- 172.21.15.97:0/2059878163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 msgr2=0x7fa180004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c160 tx=0x7fa1800057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 -- 172.21.15.97:0/2059878163 shutdown_connections 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 --2- 172.21.15.97:0/2059878163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1800047c0 0x7fa180004c40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 -- 172.21.15.97:0/2059878163 >> 172.21.15.97:0/2059878163 conn(0x7fa1a00b0a10 msgr2=0x7fa1a00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 -- 172.21.15.97:0/2059878163 shutdown_connections 2024-01-31T05:02:45.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.627+0000 7fa1a8000640 1 -- 172.21.15.97:0/2059878163 wait complete. 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.629+0000 7fa1a8000640 1 Processor -- start 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.629+0000 7fa1a8000640 1 -- start start 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.629+0000 7fa1a8000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800047c0 0x7fa1a0099f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a8000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a00999c0 0x7fa1a0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a8000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 0x7fa1a00aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a8000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a009b2a0 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a8000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00acf60 con 0x7fa1800047c0 2024-01-31T05:02:45.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a8000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00c95d0 con 0x7fa1a00999c0 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 0x7fa1a00aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 0x7fa1a00aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35588/0 (socket says 172.21.15.97:35588) 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 -- 172.21.15.97:0/3799569030 learned_addr learned my addr 172.21.15.97:0/3799569030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a5600640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800047c0 0x7fa1a0099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a4c00640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a00999c0 0x7fa1a0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 -- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a00999c0 msgr2=0x7fa1a0157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a00999c0 0x7fa1a0157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 -- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800047c0 msgr2=0x7fa1a0099f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800047c0 0x7fa1a0099f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a6000640 1 -- 172.21.15.97:0/3799569030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a009a450 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a5600640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800047c0 0x7fa1a0099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.630+0000 7fa1a4c00640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a00999c0 0x7fa1a0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:45.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.631+0000 7fa1a6000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 0x7fa1a00aca20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00ac230 tx=0x7fa19800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:45.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.631+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198018070 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.631+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa198006b50 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.631+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a009a720 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.631+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198004980 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.631+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a009ace0 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.632+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a015aea0 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.633+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa198012600 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.634+0000 7fa19d600640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa17806aed0 0x7fa17806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:45.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.634+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa198088e50 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.634+0000 7fa1a5600640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa17806aed0 0x7fa17806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:45.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.635+0000 7fa1a5600640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa17806aed0 0x7fa17806d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa190009b10 tx=0x7fa190008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:45.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.638+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa198053a40 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.826+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1a00c9790 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.827+0000 7fa19d600640 1 -- 172.21.15.97:0/3799569030 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa198057060 con 0x7fa1a00ac5c0 2024-01-31T05:02:45.827 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:45.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.829+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa17806aed0 msgr2=0x7fa17806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.829+0000 7fa1a8000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa17806aed0 0x7fa17806d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa190009b10 tx=0x7fa190008040 comp rx=0 tx=0).stop 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.829+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 msgr2=0x7fa1a00aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.829+0000 7fa1a8000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 0x7fa1a00aca20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00ac230 tx=0x7fa19800d9d0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 shutdown_connections 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa17806aed0 0x7fa17806d390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1a00ac5c0 0x7fa1a00aca20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a00999c0 0x7fa1a0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 --2- 172.21.15.97:0/3799569030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1800047c0 0x7fa1a0099f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 >> 172.21.15.97:0/3799569030 conn(0x7fa1a00b0a10 msgr2=0x7fa1a0157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 shutdown_connections 2024-01-31T05:02:45.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:45.830+0000 7fa1a8000640 1 -- 172.21.15.97:0/3799569030 wait complete. 2024-01-31T05:02:45.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:46.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.053+0000 7fcf80600640 1 Processor -- start 2024-01-31T05:02:46.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.053+0000 7fcf80600640 1 -- start start 2024-01-31T05:02:46.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf80600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 0x7fcf7815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf80600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf780cff20 0x7fcf78161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf80600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf780cded0 con 0x7fcf780cfae0 2024-01-31T05:02:46.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf80600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7815d970 con 0x7fcf780cff20 2024-01-31T05:02:46.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf80600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780cf210 con 0x7fcf7815d590 2024-01-31T05:02:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf7d200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcf780cfae0 0x7fcf78161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56074/0 (socket says 172.21.15.97:56074) 2024-01-31T05:02:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.055+0000 7fcf7d200640 1 -- 172.21.15.97:0/1864715832 learned_addr learned my addr 172.21.15.97:0/1864715832 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773844805 0 0) 0x7fcf780cded0 con 0x7fcf780cfae0 2024-01-31T05:02:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 0x7fcf7815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf58003520 con 0x7fcf780cfae0 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf7e600640 1 --2- 172.21.15.97:0/1864715832 >> v2:172.21.15.188:3300/0 conn(0x7fcf780cff20 0x7fcf78161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3838862554 0 0) 0x7fcf58003520 con 0x7fcf780cfae0 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 msgr2=0x7fcf7815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 0x7fcf7815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 >> v2:172.21.15.188:3300/0 conn(0x7fcf780cff20 msgr2=0x7fcf78161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 --2- 172.21.15.97:0/1864715832 >> v2:172.21.15.188:3300/0 conn(0x7fcf780cff20 0x7fcf78161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780ce170 con 0x7fcf780cfae0 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fcf60001670 con 0x7fcf780cfae0 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 >> v1:172.21.15.97:6789/0 conn(0x7fcf780cfae0 legacy=0x7fcf78161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf58003970 0x7fcf58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 0x7fcf580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf58004d00 0x7fcf58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7815d970 con 0x7fcf58003970 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780cf210 con 0x7fcf58004d00 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5800bab0 con 0x7fcf7815d590 2024-01-31T05:02:46.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.056+0000 7fcf7e600640 1 --2- 172.21.15.97:0/1864715832 >> v2:172.21.15.188:3300/0 conn(0x7fcf780cff20 0x7fcf78161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:02:46.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.057+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf58003970 0x7fcf58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.057+0000 7fcf7dc00640 1 -- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 msgr2=0x7fcf580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.057+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf7815d590 0x7fcf580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.057+0000 7fcf7dc00640 1 -- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf58004d00 msgr2=0x7fcf58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.057+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf58004d00 0x7fcf58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.057+0000 7fcf7dc00640 1 -- 172.21.15.97:0/1864715832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58003520 con 0x7fcf58003970 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.058+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf58003970 0x7fcf58003d90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007360 tx=0x7fcf580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e25668e9a4b9562 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.058+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6802ccf0 con 0x7fcf58003970 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.058+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6802ce50 con 0x7fcf58003970 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.058+0000 7fcf77e00640 1 -- 172.21.15.97:0/1864715832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68034660 con 0x7fcf58003970 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf80600640 1 -- 172.21.15.97:0/1864715832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf58003970 msgr2=0x7fcf58003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf80600640 1 --2- 172.21.15.97:0/1864715832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf58003970 0x7fcf58003d90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcf68007360 tx=0x7fcf580057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf7e600640 1 -- 172.21.15.97:0/1864715832 reap_dead start 2024-01-31T05:02:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf80600640 1 -- 172.21.15.97:0/1864715832 shutdown_connections 2024-01-31T05:02:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf80600640 1 -- 172.21.15.97:0/1864715832 >> 172.21.15.97:0/1864715832 conn(0x7fcf7809c560 msgr2=0x7fcf780d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf80600640 1 -- 172.21.15.97:0/1864715832 shutdown_connections 2024-01-31T05:02:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.059+0000 7fcf80600640 1 -- 172.21.15.97:0/1864715832 wait complete. 2024-01-31T05:02:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.061+0000 7fcf80600640 1 Processor -- start 2024-01-31T05:02:46.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.061+0000 7fcf80600640 1 -- start start 2024-01-31T05:02:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf80600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf58003970 0x7fcf7814a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf80600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 0x7fcf781f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf80600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf78147580 0x7fcf78147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf80600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7809d6c0 con 0x7fcf7809ced0 2024-01-31T05:02:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf80600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7814d0d0 con 0x7fcf78147580 2024-01-31T05:02:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf80600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7814d8e0 con 0x7fcf58003970 2024-01-31T05:02:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf7d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 0x7fcf781f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf7d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 0x7fcf781f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35604/0 (socket says 172.21.15.97:35604) 2024-01-31T05:02:46.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf7d200640 1 -- 172.21.15.97:0/1088575611 learned_addr learned my addr 172.21.15.97:0/1088575611 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.062+0000 7fcf7e600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf78147580 0x7fcf78147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7d200640 1 -- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf58003970 msgr2=0x7fcf7814a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7d200640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf58003970 0x7fcf7814a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7d200640 1 -- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf78147580 msgr2=0x7fcf78147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7d200640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf78147580 0x7fcf78147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7d200640 1 -- 172.21.15.97:0/1088575611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf781f65b0 con 0x7fcf7809ced0 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7e600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf78147580 0x7fcf78147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf7d200640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 0x7fcf781f2780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c7b0 tx=0x7fcf6000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600074e0 con 0x7fcf7809ced0 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60007640 con 0x7fcf7809ced0 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf781f2cc0 con 0x7fcf7809ced0 2024-01-31T05:02:46.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60004a00 con 0x7fcf7809ced0 2024-01-31T05:02:46.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.063+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7814d600 con 0x7fcf7809ced0 2024-01-31T05:02:46.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.064+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf78067830 con 0x7fcf7809ced0 2024-01-31T05:02:46.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.065+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fcf60004b60 con 0x7fcf7809ced0 2024-01-31T05:02:46.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.066+0000 7fcf75600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcf5006aed0 0x7fcf5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.066+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fcf60089150 con 0x7fcf7809ced0 2024-01-31T05:02:46.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.066+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcf5006aed0 0x7fcf5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.067+0000 7fcf7dc00640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcf5006aed0 0x7fcf5006d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf68036000 tx=0x7fcf6802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.071+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fcf60053d40 con 0x7fcf7809ced0 2024-01-31T05:02:46.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.248+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf7806bec0 con 0x7fcf7809ced0 2024-01-31T05:02:46.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.249+0000 7fcf75600640 1 -- 172.21.15.97:0/1088575611 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf60057360 con 0x7fcf7809ced0 2024-01-31T05:02:46.250 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:46.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcf5006aed0 msgr2=0x7fcf5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcf5006aed0 0x7fcf5006d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcf68036000 tx=0x7fcf6802e000 comp rx=0 tx=0).stop 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 msgr2=0x7fcf781f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 0x7fcf781f2780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c7b0 tx=0x7fcf6000cc70 comp rx=0 tx=0).stop 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 shutdown_connections 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf78147580 0x7fcf78147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fcf5006aed0 0x7fcf5006d390 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf7809ced0 0x7fcf781f2780 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 --2- 172.21.15.97:0/1088575611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf58003970 0x7fcf7814a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 >> 172.21.15.97:0/1088575611 conn(0x7fcf7809c560 msgr2=0x7fcf780cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 shutdown_connections 2024-01-31T05:02:46.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.252+0000 7fcf80600640 1 -- 172.21.15.97:0/1088575611 wait complete. 2024-01-31T05:02:46.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:02:46.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.471+0000 7f39a3a00640 1 Processor -- start 2024-01-31T05:02:46.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.472+0000 7f39a3a00640 1 -- start start 2024-01-31T05:02:46.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.473+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f399c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.473+0000 7f39a3a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c06cd20 con 0x7f399c15b470 2024-01-31T05:02:46.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.473+0000 7f39a3a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399c0ceb60 con 0x7f399c0cbf20 2024-01-31T05:02:46.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.473+0000 7f39a3a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c15b850 con 0x7f399c0abcf0 2024-01-31T05:02:46.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.473+0000 7f39a1000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f399c15b470 0x7f399c0cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56084/0 (socket says 172.21.15.97:56084) 2024-01-31T05:02:46.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.473+0000 7f39a1000640 1 -- 172.21.15.97:0/4178879940 learned_addr learned my addr 172.21.15.97:0/4178879940 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697706621 0 0) 0x7f399c06cd20 con 0x7f399c15b470 2024-01-31T05:02:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f397c003520 con 0x7f399c15b470 2024-01-31T05:02:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f39a1a00640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f399c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3568907093 0 0) 0x7f397c003520 con 0x7f399c15b470 2024-01-31T05:02:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 msgr2=0x7f399c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f399c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 >> v1:172.21.15.188:6789/0 conn(0x7f399c0cbf20 legacy=0x7f399c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c06c3f0 con 0x7f399c15b470 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f39840017f0 con 0x7f399c15b470 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 >> v1:172.21.15.97:6789/0 conn(0x7f399c15b470 legacy=0x7f399c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0039a0 0x7f397c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f397c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f397c004d00 0x7f397c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c15b850 con 0x7f397c0039a0 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c00bb10 con 0x7f397c004d00 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c00bcd0 con 0x7f399c0abcf0 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.474+0000 7f39a1000640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0039a0 0x7f397c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1a00640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f397c004d00 0x7f397c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f399be00640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f397c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1000640 1 -- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 msgr2=0x7f397c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1000640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f397c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1000640 1 -- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f397c004d00 msgr2=0x7f397c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1000640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f397c004d00 0x7f397c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1000640 1 -- 172.21.15.97:0/4178879940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c003520 con 0x7f397c0039a0 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f39a1a00640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f397c004d00 0x7f397c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.475+0000 7f399be00640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399c0abcf0 0x7f397c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.476+0000 7f39a1000640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0039a0 0x7f397c003da0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f398400c0e0 tx=0x7f397c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e023909aff8d1c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.476+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984007270 con 0x7f397c0039a0 2024-01-31T05:02:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.476+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39840073d0 con 0x7f397c0039a0 2024-01-31T05:02:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.476+0000 7f399b400640 1 -- 172.21.15.97:0/4178879940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398402c810 con 0x7f397c0039a0 2024-01-31T05:02:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a3a00640 1 -- 172.21.15.97:0/4178879940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0039a0 msgr2=0x7f397c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a3a00640 1 --2- 172.21.15.97:0/4178879940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f397c0039a0 0x7f397c003da0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f398400c0e0 tx=0x7f397c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a1a00640 1 -- 172.21.15.97:0/4178879940 reap_dead start 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a3a00640 1 -- 172.21.15.97:0/4178879940 shutdown_connections 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a3a00640 1 -- 172.21.15.97:0/4178879940 >> 172.21.15.97:0/4178879940 conn(0x7f399c0b0a10 msgr2=0x7f399c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a3a00640 1 -- 172.21.15.97:0/4178879940 shutdown_connections 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.477+0000 7f39a3a00640 1 -- 172.21.15.97:0/4178879940 wait complete. 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.479+0000 7f39a3a00640 1 Processor -- start 2024-01-31T05:02:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.479+0000 7f39a3a00640 1 -- start start 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f397c0039a0 0x7f399c0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399c1469d0 0x7f399c143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a3a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 0x7f399c1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a3a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c143a70 con 0x7f399c0ac3f0 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a3a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0accf0 con 0x7f399c1469d0 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a3a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c143cf0 con 0x7f397c0039a0 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 0x7f399c1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 0x7f399c1592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35622/0 (socket says 172.21.15.97:35622) 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 -- 172.21.15.97:0/882180387 learned_addr learned my addr 172.21.15.97:0/882180387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 -- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f397c0039a0 msgr2=0x7f399c0ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f397c0039a0 0x7f399c0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f399be00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399c1469d0 0x7f399c143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 -- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399c1469d0 msgr2=0x7f399c143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399c1469d0 0x7f399c143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.480+0000 7f39a1a00640 1 -- 172.21.15.97:0/882180387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c143f40 con 0x7f399c0ac3f0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.481+0000 7f39a1a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 0x7f399c1592a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f399c0711d0 tx=0x7f399400e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.481+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399400c070 con 0x7f399c0ac3f0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.481+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3994013610 con 0x7f399c0ac3f0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.481+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39940139c0 con 0x7f399c0ac3f0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.481+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c144220 con 0x7f399c0ac3f0 2024-01-31T05:02:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.481+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c1447e0 con 0x7f399c0ac3f0 2024-01-31T05:02:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.482+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c144a00 con 0x7f399c0ac3f0 2024-01-31T05:02:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.483+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3994013b20 con 0x7f399c0ac3f0 2024-01-31T05:02:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.484+0000 7f3998c00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f397406aed0 0x7f397406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.484+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f3994089070 con 0x7f399c0ac3f0 2024-01-31T05:02:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.484+0000 7f39a1000640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f397406aed0 0x7f397406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.485+0000 7f39a1000640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f397406aed0 0x7f397406d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3984034000 tx=0x7f3984007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.488+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3994053c60 con 0x7f399c0ac3f0 2024-01-31T05:02:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.680+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f399c067830 con 0x7f399c0ac3f0 2024-01-31T05:02:46.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.681+0000 7f3998c00640 1 -- 172.21.15.97:0/882180387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7f3994006c30 con 0x7f399c0ac3f0 2024-01-31T05:02:46.680 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f397406aed0 msgr2=0x7f397406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f397406aed0 0x7f397406d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3984034000 tx=0x7f3984007000 comp rx=0 tx=0).stop 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 msgr2=0x7f399c1592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 0x7f399c1592a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f399c0711d0 tx=0x7f399400e8b0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 shutdown_connections 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f397406aed0 0x7f397406d390 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f399c0ac3f0 0x7f399c1592a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399c1469d0 0x7f399c143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 --2- 172.21.15.97:0/882180387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f397c0039a0 0x7f399c0ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.683+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 >> 172.21.15.97:0/882180387 conn(0x7f399c0b0a10 msgr2=0x7f399c0cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.684+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 shutdown_connections 2024-01-31T05:02:46.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.684+0000 7f39a3a00640 1 -- 172.21.15.97:0/882180387 wait complete. 2024-01-31T05:02:46.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.919+0000 7f6a15e00640 1 Processor -- start 2024-01-31T05:02:46.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.920+0000 7f6a15e00640 1 -- start start 2024-01-31T05:02:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.921+0000 7f6a15e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 0x7f6a100722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.921+0000 7f6a15e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1006cd20 con 0x7f6a1015b470 2024-01-31T05:02:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.921+0000 7f6a15e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a100ceb60 con 0x7f6a100cbf20 2024-01-31T05:02:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.921+0000 7f6a15e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1015b850 con 0x7f6a100abcf0 2024-01-31T05:02:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.921+0000 7f6a0ea00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6a1015b470 0x7f6a100cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56086/0 (socket says 172.21.15.97:56086) 2024-01-31T05:02:46.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.921+0000 7f6a0ea00640 1 -- 172.21.15.97:0/4196322075 learned_addr learned my addr 172.21.15.97:0/4196322075 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023725762 0 0) 0x7f6a1006cd20 con 0x7f6a1015b470 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f0003520 con 0x7f6a1015b470 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0f400640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 0x7f6a100722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643293805 0 0) 0x7f69f0003520 con 0x7f6a1015b470 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 msgr2=0x7f6a100722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 0x7f6a100722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 >> v1:172.21.15.188:6789/0 conn(0x7f6a100cbf20 legacy=0x7f6a1006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1006c3f0 con 0x7f6a1015b470 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f6a080017f0 con 0x7f6a1015b470 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 >> v1:172.21.15.97:6789/0 conn(0x7f6a1015b470 legacy=0x7f6a100cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f00039a0 0x7f69f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69f00047c0 0x7f69f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 0x7f69f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1015b850 con 0x7f69f00039a0 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f000bb10 con 0x7f69f00047c0 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.922+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f000bcd0 con 0x7f6a100abcf0 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0ea00640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f00039a0 0x7f69f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0e000640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69f00047c0 0x7f69f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0ea00640 1 -- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 msgr2=0x7f69f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0ea00640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a100abcf0 0x7f69f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0ea00640 1 -- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69f00047c0 msgr2=0x7f69f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0ea00640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69f00047c0 0x7f69f0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.923+0000 7f6a0ea00640 1 -- 172.21.15.97:0/4196322075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0003520 con 0x7f69f00039a0 2024-01-31T05:02:46.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.924+0000 7f6a0ea00640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f00039a0 0x7f69f0003dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c0e0 tx=0x7f69f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46f4aaefb68aba server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.924+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08007270 con 0x7f69f00039a0 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.924+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a080073d0 con 0x7f69f00039a0 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.924+0000 7f6a0d600640 1 -- 172.21.15.97:0/4196322075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0802c7d0 con 0x7f69f00039a0 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a15e00640 1 -- 172.21.15.97:0/4196322075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f00039a0 msgr2=0x7f69f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a15e00640 1 --2- 172.21.15.97:0/4196322075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f00039a0 0x7f69f0003dc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c0e0 tx=0x7f69f00057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a0f400640 1 -- 172.21.15.97:0/4196322075 reap_dead start 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a15e00640 1 -- 172.21.15.97:0/4196322075 shutdown_connections 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a15e00640 1 -- 172.21.15.97:0/4196322075 >> 172.21.15.97:0/4196322075 conn(0x7f6a100b0a10 msgr2=0x7f6a100b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a15e00640 1 -- 172.21.15.97:0/4196322075 shutdown_connections 2024-01-31T05:02:46.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.925+0000 7f6a15e00640 1 -- 172.21.15.97:0/4196322075 wait complete. 2024-01-31T05:02:46.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.927+0000 7f6a15e00640 1 Processor -- start 2024-01-31T05:02:46.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.927+0000 7f6a15e00640 1 -- start start 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a15e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00039a0 0x7f6a100999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a15e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 0x7f6a1014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a15e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a1015e270 0x7f6a1015c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a15e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100b21a0 con 0x7f6a101505e0 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a15e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100b2480 con 0x7f6a1015e270 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a15e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100b2760 con 0x7f69f00039a0 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 0x7f6a1014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 0x7f6a1014fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35630/0 (socket says 172.21.15.97:35630) 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.928+0000 7f6a0e000640 1 -- 172.21.15.97:0/1709922529 learned_addr learned my addr 172.21.15.97:0/1709922529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a0e000640 1 -- 172.21.15.97:0/1709922529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00039a0 msgr2=0x7f6a100999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a0e000640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00039a0 0x7f6a100999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a0e000640 1 -- 172.21.15.97:0/1709922529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a1015e270 msgr2=0x7f6a1015c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a0e000640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a1015e270 0x7f6a1015c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a0e000640 1 -- 172.21.15.97:0/1709922529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a100b2a40 con 0x7f6a101505e0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a0e000640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 0x7f6a1014fe00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f69f800d860 tx=0x7f69f800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f8003da0 con 0x7f6a101505e0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f800b6b0 con 0x7f6a101505e0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f800ba80 con 0x7f6a101505e0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a100b2d20 con 0x7f6a101505e0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.929+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1015e6f0 con 0x7f6a101505e0 2024-01-31T05:02:46.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.930+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a10067830 con 0x7f6a101505e0 2024-01-31T05:02:46.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.932+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f69f800bbe0 con 0x7f6a101505e0 2024-01-31T05:02:46.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.932+0000 7f6a07400640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69e806aed0 0x7f69e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:46.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.933+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f69f8089ac0 con 0x7f6a101505e0 2024-01-31T05:02:46.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.933+0000 7f6a0ea00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69e806aed0 0x7f69e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:46.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.933+0000 7f6a0ea00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69e806aed0 0x7f69e806d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c0e0 tx=0x7f6a08007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:46.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:46.937+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f69f8054750 con 0x7f6a101505e0 2024-01-31T05:02:47.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.149+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6a10071dc0 con 0x7f6a101505e0 2024-01-31T05:02:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.149+0000 7f6a07400640 1 -- 172.21.15.97:0/1709922529 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f69f8057d70 con 0x7f6a101505e0 2024-01-31T05:02:47.149 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69e806aed0 msgr2=0x7f69e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69e806aed0 0x7f69e806d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c0e0 tx=0x7f6a08007000 comp rx=0 tx=0).stop 2024-01-31T05:02:47.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 msgr2=0x7f6a1014fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 0x7f6a1014fe00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f69f800d860 tx=0x7f69f800dd20 comp rx=0 tx=0).stop 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 shutdown_connections 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a1015e270 0x7f6a1015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.152+0000 7f6a15e00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f69e806aed0 0x7f69e806d390 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.153+0000 7f6a15e00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a101505e0 0x7f6a1014fe00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.153+0000 7f6a15e00640 1 --2- 172.21.15.97:0/1709922529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69f00039a0 0x7f6a100999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.153+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 >> 172.21.15.97:0/1709922529 conn(0x7f6a100b0a10 msgr2=0x7f6a100cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.153+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 shutdown_connections 2024-01-31T05:02:47.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.153+0000 7f6a15e00640 1 -- 172.21.15.97:0/1709922529 wait complete. 2024-01-31T05:02:47.170 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:02:47.170 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:47.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.467+0000 7f699fe00640 1 Processor -- start 2024-01-31T05:02:47.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.468+0000 7f699fe00640 1 -- start start 2024-01-31T05:02:47.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69980cb810 0x7f69980cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f699815b400 0x7f699806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699fe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699806d1a0 con 0x7f699806db70 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699fe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69980cc960 con 0x7f69980cb810 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699806c600 con 0x7f699815b400 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69980cb810 0x7f69980cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699de00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f699806db70 0x7f69980ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56094/0 (socket says 172.21.15.97:56094) 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.469+0000 7f699de00640 1 -- 172.21.15.97:0/1354230535 learned_addr learned my addr 172.21.15.97:0/1354230535 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.470+0000 7f699ca00640 1 --2- 172.21.15.97:0/1354230535 >> v2:172.21.15.188:3300/0 conn(0x7f699815b400 0x7f699806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.470+0000 7f699d400640 1 -- 172.21.15.97:0/1354230535 >> v2:172.21.15.188:3300/0 conn(0x7f699815b400 msgr2=0x7f699806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.470+0000 7f699d400640 1 --2- 172.21.15.97:0/1354230535 >> v2:172.21.15.188:3300/0 conn(0x7f699815b400 0x7f699806d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.470+0000 7f699d400640 1 -- 172.21.15.97:0/1354230535 >> v1:172.21.15.97:6789/0 conn(0x7f699806db70 legacy=0x7f69980ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.470+0000 7f699d400640 1 -- 172.21.15.97:0/1354230535 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69980cc130 con 0x7f69980cb810 2024-01-31T05:02:47.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.470+0000 7f6997e00640 1 -- 172.21.15.97:0/1354230535 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367513448 0 0) 0x7f699806d1a0 con 0x7f699806db70 2024-01-31T05:02:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.471+0000 7f699d400640 1 --2- 172.21.15.97:0/1354230535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69980cb810 0x7f69980cbbf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f69980ae830 tx=0x7f698802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c772e051701317a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.471+0000 7f6997e00640 1 -- 172.21.15.97:0/1354230535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988031830 con 0x7f69980cb810 2024-01-31T05:02:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.471+0000 7f6997e00640 1 -- 172.21.15.97:0/1354230535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988031990 con 0x7f69980cb810 2024-01-31T05:02:47.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.471+0000 7f6997e00640 1 -- 172.21.15.97:0/1354230535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988031d20 con 0x7f69980cb810 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 -- 172.21.15.97:0/1354230535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69980cb810 msgr2=0x7f69980cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 --2- 172.21.15.97:0/1354230535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69980cb810 0x7f69980cbbf0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f69980ae830 tx=0x7f698802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 -- 172.21.15.97:0/1354230535 shutdown_connections 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 --2- 172.21.15.97:0/1354230535 >> v2:172.21.15.188:3300/0 conn(0x7f699815b400 0x7f699806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 --2- 172.21.15.97:0/1354230535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69980cb810 0x7f69980cbbf0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 -- 172.21.15.97:0/1354230535 >> 172.21.15.97:0/1354230535 conn(0x7f69980b0a10 msgr2=0x7f69980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 -- 172.21.15.97:0/1354230535 shutdown_connections 2024-01-31T05:02:47.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.472+0000 7f699fe00640 1 -- 172.21.15.97:0/1354230535 wait complete. 2024-01-31T05:02:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.474+0000 7f699fe00640 1 Processor -- start 2024-01-31T05:02:47.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.474+0000 7f699fe00640 1 -- start start 2024-01-31T05:02:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 0x7f69981f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69980cb810 0x7f69981f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f699815b400 0x7f699809bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980cd260 con 0x7f699806db70 2024-01-31T05:02:47.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981f1840 con 0x7f699815b400 2024-01-31T05:02:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69981f1a40 con 0x7f69980cb810 2024-01-31T05:02:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 0x7f69981f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 0x7f69981f0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35642/0 (socket says 172.21.15.97:35642) 2024-01-31T05:02:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 -- 172.21.15.97:0/2987036778 learned_addr learned my addr 172.21.15.97:0/2987036778 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699de00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f699815b400 0x7f699809bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 -- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69980cb810 msgr2=0x7f69981f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69980cb810 0x7f69981f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 -- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f699815b400 msgr2=0x7f699809bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.475+0000 7f699d400640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f699815b400 0x7f699809bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f699d400640 1 -- 172.21.15.97:0/2987036778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988009400 con 0x7f699806db70 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f699d400640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 0x7f69981f0dc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6988009890 tx=0x7f69880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69880317d0 con 0x7f699806db70 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698803ba70 con 0x7f699806db70 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698803be20 con 0x7f699806db70 2024-01-31T05:02:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69981f1d20 con 0x7f699806db70 2024-01-31T05:02:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.476+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69981f2300 con 0x7f699806db70 2024-01-31T05:02:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.477+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69980677c0 con 0x7f699806db70 2024-01-31T05:02:47.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.478+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f69880453f0 con 0x7f699806db70 2024-01-31T05:02:47.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.479+0000 7f6995600640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697006af00 0x7f697006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.479+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f69880abbb0 con 0x7f699806db70 2024-01-31T05:02:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.479+0000 7f699ca00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697006af00 0x7f697006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.480+0000 7f699ca00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697006af00 0x7f697006d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6980006fd0 tx=0x7f6980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:47.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.484+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6988076830 con 0x7f699806db70 2024-01-31T05:02:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.680+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f699815ab80 con 0x7f699806db70 2024-01-31T05:02:47.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.683+0000 7f6995600640 1 -- 172.21.15.97:0/2987036778 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f6988035020 con 0x7f699806db70 2024-01-31T05:02:47.683 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:47.684 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:47.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.686+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697006af00 msgr2=0x7f697006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.686+0000 7f699fe00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697006af00 0x7f697006d3c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6980006fd0 tx=0x7f6980008040 comp rx=0 tx=0).stop 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.686+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 msgr2=0x7f69981f0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.686+0000 7f699fe00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 0x7f69981f0dc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6988009890 tx=0x7f69880046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 shutdown_connections 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f699815b400 0x7f699809bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69980cb810 0x7f69981f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f697006af00 0x7f697006d3c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 --2- 172.21.15.97:0/2987036778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f699806db70 0x7f69981f0dc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 >> 172.21.15.97:0/2987036778 conn(0x7f69980b0a10 msgr2=0x7f69981558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 shutdown_connections 2024-01-31T05:02:47.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.687+0000 7f699fe00640 1 -- 172.21.15.97:0/2987036778 wait complete. 2024-01-31T05:02:47.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:02:47.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.923+0000 7fa776200640 1 Processor -- start 2024-01-31T05:02:47.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.923+0000 7fa776200640 1 -- start start 2024-01-31T05:02:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.924+0000 7fa776200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa7700722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.924+0000 7fa776200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa7700cbf20 0x7fa7700ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa776200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77006cd20 con 0x7fa7700abcf0 2024-01-31T05:02:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa776200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700ce970 con 0x7fa7700cbf20 2024-01-31T05:02:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa776200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77015b850 con 0x7fa77015b470 2024-01-31T05:02:47.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa76ea00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa7700abcf0 0x7fa7700cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56098/0 (socket says 172.21.15.97:56098) 2024-01-31T05:02:47.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa76ea00640 1 -- 172.21.15.97:0/1010142101 learned_addr learned my addr 172.21.15.97:0/1010142101 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1010142101 >> v2:172.21.15.188:3300/0 conn(0x7fa7700cbf20 0x7fa7700ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778506157 0 0) 0x7fa77006cd20 con 0x7fa7700abcf0 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa76f400640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa7700722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.925+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa750003520 con 0x7fa7700abcf0 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727675826 0 0) 0x7fa750003520 con 0x7fa7700abcf0 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 msgr2=0x7fa7700722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa7700722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 >> v2:172.21.15.188:3300/0 conn(0x7fa7700cbf20 msgr2=0x7fa7700ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 --2- 172.21.15.97:0/1010142101 >> v2:172.21.15.188:3300/0 conn(0x7fa7700cbf20 0x7fa7700ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700ceb60 con 0x7fa7700abcf0 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa758001670 con 0x7fa7700abcf0 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 >> v1:172.21.15.97:6789/0 conn(0x7fa7700abcf0 legacy=0x7fa7700cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7500039a0 0x7fa750003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7500047c0 0x7fa750004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa750005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77015b850 con 0x7fa7500039a0 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75000b980 con 0x7fa7500047c0 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75000bac0 con 0x7fa77015b470 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.926+0000 7fa76f400640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7500039a0 0x7fa750003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76ea00640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7500047c0 0x7fa750004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa750005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76f400640 1 -- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 msgr2=0x7fa750005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76f400640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa750005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76f400640 1 -- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7500047c0 msgr2=0x7fa750004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76f400640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7500047c0 0x7fa750004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76f400640 1 -- 172.21.15.97:0/1010142101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa750003520 con 0x7fa7500039a0 2024-01-31T05:02:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76ea00640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7500047c0 0x7fa750004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.927+0000 7fa76fe00640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa77015b470 0x7fa750005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.928+0000 7fa76f400640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7500039a0 0x7fa750003dc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ab0 tx=0x7fa7500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525dfacbe6823e0e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.928+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760005990 con 0x7fa7500039a0 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.928+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760005af0 con 0x7fa7500039a0 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.928+0000 7fa76e000640 1 -- 172.21.15.97:0/1010142101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760005ea0 con 0x7fa7500039a0 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.928+0000 7fa776200640 1 -- 172.21.15.97:0/1010142101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7500039a0 msgr2=0x7fa750003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.928+0000 7fa776200640 1 --2- 172.21.15.97:0/1010142101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7500039a0 0x7fa750003dc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ab0 tx=0x7fa7500057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.929+0000 7fa76fe00640 1 -- 172.21.15.97:0/1010142101 reap_dead start 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.929+0000 7fa776200640 1 -- 172.21.15.97:0/1010142101 shutdown_connections 2024-01-31T05:02:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.929+0000 7fa776200640 1 -- 172.21.15.97:0/1010142101 >> 172.21.15.97:0/1010142101 conn(0x7fa7700b0a10 msgr2=0x7fa7700b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.929+0000 7fa776200640 1 -- 172.21.15.97:0/1010142101 shutdown_connections 2024-01-31T05:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.929+0000 7fa776200640 1 -- 172.21.15.97:0/1010142101 wait complete. 2024-01-31T05:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.930+0000 7fa776200640 1 Processor -- start 2024-01-31T05:02:47.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.931+0000 7fa776200640 1 -- start start 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.931+0000 7fa776200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7500039a0 0x7fa7701f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa776200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7701f14b0 0x7fa7701f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa776200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 0x7fa77015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa776200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77015e560 con 0x7fa77015dba0 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa776200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770152360 con 0x7fa7701f14b0 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa776200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa770152600 con 0x7fa7500039a0 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 0x7fa77015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 0x7fa77015e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35648/0 (socket says 172.21.15.97:35648) 2024-01-31T05:02:47.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 -- 172.21.15.97:0/2384562173 learned_addr learned my addr 172.21.15.97:0/2384562173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76f400640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7500039a0 0x7fa7701f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 -- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7500039a0 msgr2=0x7fa7701f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76ea00640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7701f14b0 0x7fa7701f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7500039a0 0x7fa7701f1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 -- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7701f14b0 msgr2=0x7fa7701f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7701f14b0 0x7fa7701f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.932+0000 7fa76fe00640 1 -- 172.21.15.97:0/2384562173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7701528e0 con 0x7fa77015dba0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.933+0000 7fa76fe00640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 0x7fa77015e020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa770072f30 tx=0x7fa76800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.933+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768017400 con 0x7fa77015dba0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.933+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768011040 con 0x7fa77015dba0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.933+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768017780 con 0x7fa77015dba0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.933+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa770152bc0 con 0x7fa77015dba0 2024-01-31T05:02:47.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.933+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa770153180 con 0x7fa77015dba0 2024-01-31T05:02:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.934+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7701533a0 con 0x7fa77015dba0 2024-01-31T05:02:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.935+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa7680203b0 con 0x7fa77015dba0 2024-01-31T05:02:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.936+0000 7fa767400640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa74806af00 0x7fa74806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:47.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.936+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa768089af0 con 0x7fa77015dba0 2024-01-31T05:02:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.936+0000 7fa76f400640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa74806af00 0x7fa74806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:47.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.937+0000 7fa76f400640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa74806af00 0x7fa74806d3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa760035000 tx=0x7fa760030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:47.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:47.941+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa768054780 con 0x7fa77015dba0 2024-01-31T05:02:48.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.134+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa770067830 con 0x7fa77015dba0 2024-01-31T05:02:48.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.135+0000 7fa767400640 1 -- 172.21.15.97:0/2384562173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa768089020 con 0x7fa77015dba0 2024-01-31T05:02:48.135 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa74806af00 msgr2=0x7fa74806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa74806af00 0x7fa74806d3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa760035000 tx=0x7fa760030000 comp rx=0 tx=0).stop 2024-01-31T05:02:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 msgr2=0x7fa77015e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 0x7fa77015e020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa770072f30 tx=0x7fa76800f4a0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 shutdown_connections 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa74806af00 0x7fa74806d3c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77015dba0 0x7fa77015e020 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7701f14b0 0x7fa7701f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 --2- 172.21.15.97:0/2384562173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7500039a0 0x7fa7701f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.138+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 >> 172.21.15.97:0/2384562173 conn(0x7fa7700b0a10 msgr2=0x7fa77006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.139+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 shutdown_connections 2024-01-31T05:02:48.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.139+0000 7fa776200640 1 -- 172.21.15.97:0/2384562173 wait complete. 2024-01-31T05:02:48.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.357+0000 7f25dca00640 1 Processor -- start 2024-01-31T05:02:48.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.357+0000 7f25dca00640 1 -- start start 2024-01-31T05:02:48.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.358+0000 7f25dca00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f25d40ce000 0x7f25d40cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.358+0000 7f25dca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.358+0000 7f25dca00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d40cb760 con 0x7f25d40b2570 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.358+0000 7f25dca00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d406ce00 con 0x7f25d40ce000 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.358+0000 7f25dca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40cd4f0 con 0x7f25d40d1b90 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.358+0000 7f25da000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f25d40b2570 0x7f25d40cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56114/0 (socket says 172.21.15.97:56114) 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25da000640 1 -- 172.21.15.97:0/3727473671 learned_addr learned my addr 172.21.15.97:0/3727473671 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25d9600640 1 --2- 172.21.15.97:0/3727473671 >> v2:172.21.15.97:3300/0 conn(0x7f25d40ce000 0x7f25d40cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25daa00640 1 --2- 172.21.15.97:0/3727473671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25d8c00640 1 -- 172.21.15.97:0/3727473671 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103455360 0 0) 0x7f25d40cb760 con 0x7f25d40b2570 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25d9600640 1 -- 172.21.15.97:0/3727473671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 msgr2=0x7f25d415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25d9600640 1 --2- 172.21.15.97:0/3727473671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25d9600640 1 -- 172.21.15.97:0/3727473671 >> v1:172.21.15.97:6789/0 conn(0x7f25d40b2570 legacy=0x7f25d40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.359+0000 7f25d9600640 1 -- 172.21.15.97:0/3727473671 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d40b2950 con 0x7f25d40ce000 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.360+0000 7f25d9600640 1 --2- 172.21.15.97:0/3727473671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d40cf3e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0097a0 tx=0x7f25bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a940898cf3a90c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.360+0000 7f25d8c00640 1 -- 172.21.15.97:0/3727473671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc02e070 con 0x7f25d40ce000 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.360+0000 7f25d8c00640 1 -- 172.21.15.97:0/3727473671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc0315a0 con 0x7f25d40ce000 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.360+0000 7f25d8c00640 1 -- 172.21.15.97:0/3727473671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc031930 con 0x7f25d40ce000 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 -- 172.21.15.97:0/3727473671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 msgr2=0x7f25d40cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 --2- 172.21.15.97:0/3727473671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d40cf3e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0097a0 tx=0x7f25bc02b840 comp rx=0 tx=0).stop 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 -- 172.21.15.97:0/3727473671 shutdown_connections 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 --2- 172.21.15.97:0/3727473671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 --2- 172.21.15.97:0/3727473671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d40cf3e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 -- 172.21.15.97:0/3727473671 >> 172.21.15.97:0/3727473671 conn(0x7f25d40d1070 msgr2=0x7f25d40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 -- 172.21.15.97:0/3727473671 shutdown_connections 2024-01-31T05:02:48.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.361+0000 7f25dca00640 1 -- 172.21.15.97:0/3727473671 wait complete. 2024-01-31T05:02:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.363+0000 7f25dca00640 1 Processor -- start 2024-01-31T05:02:48.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.363+0000 7f25dca00640 1 -- start start 2024-01-31T05:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25dca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d40b2570 0x7f25d41f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25dca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d41e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25dca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25dca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41f2d90 con 0x7f25d40ce000 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25dca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41e2bf0 con 0x7f25d40b2570 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25dca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40c8bb0 con 0x7f25d40d1b90 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25d9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d41e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25d9600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d41e2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35676/0 (socket says 172.21.15.97:35676) 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.364+0000 7f25d9600640 1 -- 172.21.15.97:0/1751011886 learned_addr learned my addr 172.21.15.97:0/1751011886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25daa00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25da000640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d40b2570 0x7f25d41f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d9600640 1 -- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 msgr2=0x7f25d40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d9600640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d40aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d9600640 1 -- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d40b2570 msgr2=0x7f25d41f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d9600640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d40b2570 0x7f25d41f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d9600640 1 -- 172.21.15.97:0/1751011886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009400 con 0x7f25d40ce000 2024-01-31T05:02:48.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d9600640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d41e2370 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f25bc034040 tx=0x7f25bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.365+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc032900 con 0x7f25d40ce000 2024-01-31T05:02:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.366+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc032a80 con 0x7f25d40ce000 2024-01-31T05:02:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.366+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d40c8e90 con 0x7f25d40ce000 2024-01-31T05:02:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.366+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc032e30 con 0x7f25d40ce000 2024-01-31T05:02:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.366+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d40c9450 con 0x7f25d40ce000 2024-01-31T05:02:48.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.367+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d40c9640 con 0x7f25d40ce000 2024-01-31T05:02:48.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.368+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f25bc04e430 con 0x7f25d40ce000 2024-01-31T05:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.368+0000 7f25d2000640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25ac06aed0 0x7f25ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.368+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f25bc0aba90 con 0x7f25d40ce000 2024-01-31T05:02:48.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.369+0000 7f25da000640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25ac06aed0 0x7f25ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.369+0000 7f25da000640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25ac06aed0 0x7f25ac06d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000f60 tx=0x7f25c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:48.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.373+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f25bc0765f0 con 0x7f25d40ce000 2024-01-31T05:02:48.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.563+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25d4067830 con 0x7f25d40ce000 2024-01-31T05:02:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.564+0000 7f25d2000640 1 -- 172.21.15.97:0/1751011886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25bc079c10 con 0x7f25d40ce000 2024-01-31T05:02:48.564 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25ac06aed0 msgr2=0x7f25ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25ac06aed0 0x7f25ac06d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000f60 tx=0x7f25c4009040 comp rx=0 tx=0).stop 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 msgr2=0x7f25d41e2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d41e2370 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f25bc034040 tx=0x7f25bc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 shutdown_connections 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25d40d1b90 0x7f25d40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25ac06aed0 0x7f25ac06d390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25d40ce000 0x7f25d41e2370 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 --2- 172.21.15.97:0/1751011886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25d40b2570 0x7f25d41f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 >> 172.21.15.97:0/1751011886 conn(0x7f25d40d1070 msgr2=0x7f25d40b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 shutdown_connections 2024-01-31T05:02:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.567+0000 7f25dca00640 1 -- 172.21.15.97:0/1751011886 wait complete. 2024-01-31T05:02:48.584 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:02:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.791+0000 7f8ad9800640 1 Processor -- start 2024-01-31T05:02:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.792+0000 7f8ad9800640 1 -- start start 2024-01-31T05:02:48.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.793+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad40abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.793+0000 7f8ad9800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8ad415a730 0x7f8ad406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.793+0000 7f8ad9800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad415abc0 con 0x7f8ad406dfc0 2024-01-31T05:02:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.793+0000 7f8ad9800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40b2c90 con 0x7f8ad415a730 2024-01-31T05:02:48.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.793+0000 7f8ad9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad415aea0 con 0x7f8ad415a350 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad40abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad40abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:55928/0 (socket says 172.21.15.97:55928) 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 -- 172.21.15.97:0/2764196193 learned_addr learned my addr 172.21.15.97:0/2764196193 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8acbe00640 1 --2- 172.21.15.97:0/2764196193 >> v2:172.21.15.188:3300/0 conn(0x7f8ad415a730 0x7f8ad406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 -- 172.21.15.97:0/2764196193 >> v2:172.21.15.188:3300/0 conn(0x7f8ad415a730 msgr2=0x7f8ad406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 --2- 172.21.15.97:0/2764196193 >> v2:172.21.15.188:3300/0 conn(0x7f8ad415a730 0x7f8ad406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 -- 172.21.15.97:0/2764196193 >> v1:172.21.15.188:6789/0 conn(0x7f8ad406dfc0 legacy=0x7f8ad40cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.794+0000 7f8ad2a00640 1 -- 172.21.15.97:0/2764196193 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad415b180 con 0x7f8ad415a350 2024-01-31T05:02:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.795+0000 7f8ad2a00640 1 --2- 172.21.15.97:0/2764196193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad40abcf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009850 tx=0x7f8ac402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc94f7395da152f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.796+0000 7f8ad2000640 1 -- 172.21.15.97:0/2764196193 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4004230 con 0x7f8ad415a350 2024-01-31T05:02:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.796+0000 7f8ad2000640 1 -- 172.21.15.97:0/2764196193 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4004390 con 0x7f8ad415a350 2024-01-31T05:02:48.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.796+0000 7f8ad2000640 1 -- 172.21.15.97:0/2764196193 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4003810 con 0x7f8ad415a350 2024-01-31T05:02:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.796+0000 7f8ad9800640 1 -- 172.21.15.97:0/2764196193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 msgr2=0x7f8ad40abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 --2- 172.21.15.97:0/2764196193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad40abcf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009850 tx=0x7f8ac402b960 comp rx=0 tx=0).stop 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 -- 172.21.15.97:0/2764196193 shutdown_connections 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 --2- 172.21.15.97:0/2764196193 >> v2:172.21.15.188:3300/0 conn(0x7f8ad415a730 0x7f8ad406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 --2- 172.21.15.97:0/2764196193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad40abcf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 -- 172.21.15.97:0/2764196193 >> 172.21.15.97:0/2764196193 conn(0x7f8ad40b0a10 msgr2=0x7f8ad40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 -- 172.21.15.97:0/2764196193 shutdown_connections 2024-01-31T05:02:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.797+0000 7f8ad9800640 1 -- 172.21.15.97:0/2764196193 wait complete. 2024-01-31T05:02:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.799+0000 7f8ad9800640 1 Processor -- start 2024-01-31T05:02:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.799+0000 7f8ad9800640 1 -- start start 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad406dfc0 0x7f8ad414c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad4154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 0x7f8ad4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad9800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40aed00 con 0x7f8ad40b0390 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad41534c0 con 0x7f8ad406dfc0 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40ae710 con 0x7f8ad415a350 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 0x7f8ad4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 0x7f8ad4154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35688/0 (socket says 172.21.15.97:35688) 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad3400640 1 -- 172.21.15.97:0/3100053157 learned_addr learned my addr 172.21.15.97:0/3100053157 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8acbe00640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.800+0000 7f8ad2a00640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad406dfc0 0x7f8ad414c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad3400640 1 -- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 msgr2=0x7f8ad4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad3400640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad3400640 1 -- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad406dfc0 msgr2=0x7f8ad414c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad3400640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad406dfc0 0x7f8ad414c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad3400640 1 -- 172.21.15.97:0/3100053157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4009460 con 0x7f8ad40b0390 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad3400640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 0x7f8ad4154dd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c710 tx=0x7f8acc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc006d50 con 0x7f8ad40b0390 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00ce20 con 0x7f8ad40b0390 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad4155310 con 0x7f8ad40b0390 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.801+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc003bb0 con 0x7f8ad40b0390 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.802+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad41531e0 con 0x7f8ad40b0390 2024-01-31T05:02:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.803+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad40aee40 con 0x7f8ad40b0390 2024-01-31T05:02:48.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.804+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8acc003d10 con 0x7f8ad40b0390 2024-01-31T05:02:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.804+0000 7f8acaa00640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8aac06af00 0x7f8aac06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.805+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f8acc089430 con 0x7f8ad40b0390 2024-01-31T05:02:48.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.805+0000 7f8ad2a00640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8aac06af00 0x7f8aac06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:48.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.805+0000 7f8ad2a00640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8aac06af00 0x7f8aac06d3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4035000 tx=0x7f8ac4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:48.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:48.810+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8ad40aee40 con 0x7f8ad40b0390 2024-01-31T05:02:49.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.000+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ad40aee40 con 0x7f8ad40b0390 2024-01-31T05:02:49.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.001+0000 7f8acaa00640 1 -- 172.21.15.97:0/3100053157 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ad40aee40 con 0x7f8ad40b0390 2024-01-31T05:02:49.001 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:02:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8aac06af00 msgr2=0x7f8aac06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8aac06af00 0x7f8aac06d3c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4035000 tx=0x7f8ac4008030 comp rx=0 tx=0).stop 2024-01-31T05:02:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 msgr2=0x7f8ad4154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 0x7f8ad4154dd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00c710 tx=0x7f8acc00cbd0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 shutdown_connections 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8aac06af00 0x7f8aac06d3c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad40b0390 0x7f8ad4154dd0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ad415a350 0x7f8ad4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 --2- 172.21.15.97:0/3100053157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ad406dfc0 0x7f8ad414c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.003+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 >> 172.21.15.97:0/3100053157 conn(0x7f8ad40b0a10 msgr2=0x7f8ad40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.004+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 shutdown_connections 2024-01-31T05:02:49.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.004+0000 7f8ad9800640 1 -- 172.21.15.97:0/3100053157 wait complete. 2024-01-31T05:02:49.020 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:02:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.230+0000 7f5b98c00640 1 Processor -- start 2024-01-31T05:02:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.230+0000 7f5b98c00640 1 -- start start 2024-01-31T05:02:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b98c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b9415b020 0x7f5b9406a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b98c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9415c410 0x7f5b9406ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b98c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b94071500 con 0x7f5b940b4ab0 2024-01-31T05:02:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b98c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b94071da0 con 0x7f5b9415b020 2024-01-31T05:02:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b98c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940b4f90 con 0x7f5b9415c410 2024-01-31T05:02:49.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b93400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b9415b020 0x7f5b9406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b93400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5b9415b020 0x7f5b9406a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35700/0 (socket says 172.21.15.97:35700) 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.232+0000 7f5b93400640 1 -- 172.21.15.97:0/2504453724 learned_addr learned my addr 172.21.15.97:0/2504453724 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.233+0000 7f5b93e00640 1 --2- 172.21.15.97:0/2504453724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9415c410 0x7f5b9406ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.233+0000 7f5b93400640 1 -- 172.21.15.97:0/2504453724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9415c410 msgr2=0x7f5b9406ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.233+0000 7f5b93400640 1 --2- 172.21.15.97:0/2504453724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9415c410 0x7f5b9406ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.233+0000 7f5b93400640 1 -- 172.21.15.97:0/2504453724 >> v1:172.21.15.188:6789/0 conn(0x7f5b940b4ab0 legacy=0x7f5b940ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:49.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.233+0000 7f5b93400640 1 -- 172.21.15.97:0/2504453724 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94071750 con 0x7f5b9415b020 2024-01-31T05:02:49.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.234+0000 7f5b93400640 1 --2- 172.21.15.97:0/2504453724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b9406a710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009870 tx=0x7f5b8402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985aed6a91c8881b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:49.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.234+0000 7f5b92a00640 1 -- 172.21.15.97:0/2504453724 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840317c0 con 0x7f5b9415b020 2024-01-31T05:02:49.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.234+0000 7f5b92a00640 1 -- 172.21.15.97:0/2504453724 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b84031920 con 0x7f5b9415b020 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.234+0000 7f5b92a00640 1 -- 172.21.15.97:0/2504453724 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84031cb0 con 0x7f5b9415b020 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 -- 172.21.15.97:0/2504453724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 msgr2=0x7f5b9406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 --2- 172.21.15.97:0/2504453724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b9406a710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009870 tx=0x7f5b8402b860 comp rx=0 tx=0).stop 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 -- 172.21.15.97:0/2504453724 shutdown_connections 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 --2- 172.21.15.97:0/2504453724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9415c410 0x7f5b9406ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 --2- 172.21.15.97:0/2504453724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b9406a710 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 -- 172.21.15.97:0/2504453724 >> 172.21.15.97:0/2504453724 conn(0x7f5b940b1b00 msgr2=0x7f5b940b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 -- 172.21.15.97:0/2504453724 shutdown_connections 2024-01-31T05:02:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.235+0000 7f5b98c00640 1 -- 172.21.15.97:0/2504453724 wait complete. 2024-01-31T05:02:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.237+0000 7f5b98c00640 1 Processor -- start 2024-01-31T05:02:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.237+0000 7f5b98c00640 1 -- start start 2024-01-31T05:02:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b98c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b940b4ab0 0x7f5b94158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b98c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b940cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b98c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9415a2d0 0x7f5b94153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b98c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406d3a0 con 0x7f5b9415b020 2024-01-31T05:02:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b98c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9406d680 con 0x7f5b9415a2d0 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b98c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b94159190 con 0x7f5b940b4ab0 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b940cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b940cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35708/0 (socket says 172.21.15.97:35708) 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 -- 172.21.15.97:0/1217303944 learned_addr learned my addr 172.21.15.97:0/1217303944 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b93400640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b940b4ab0 0x7f5b94158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 -- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b940b4ab0 msgr2=0x7f5b94158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b940b4ab0 0x7f5b94158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 -- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9415a2d0 msgr2=0x7f5b94153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9415a2d0 0x7f5b94153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.238+0000 7f5b8be00640 1 -- 172.21.15.97:0/1217303944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840094d0 con 0x7f5b9415b020 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.239+0000 7f5b8be00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b940cceb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00c460 tx=0x7f5b7c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:49.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.239+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c012830 con 0x7f5b9415b020 2024-01-31T05:02:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.239+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c0129d0 con 0x7f5b9415b020 2024-01-31T05:02:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.239+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c012d80 con 0x7f5b9415b020 2024-01-31T05:02:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.239+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b94159470 con 0x7f5b9415b020 2024-01-31T05:02:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.239+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b94158e80 con 0x7f5b9415b020 2024-01-31T05:02:49.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.240+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9406d810 con 0x7f5b9415b020 2024-01-31T05:02:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.241+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5b7c01a3b0 con 0x7f5b9415b020 2024-01-31T05:02:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.242+0000 7f5b8b400640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5b6c06aed0 0x7f5b6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.242+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f5b7c089620 con 0x7f5b9415b020 2024-01-31T05:02:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.242+0000 7f5b93400640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5b6c06aed0 0x7f5b6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.243+0000 7f5b93400640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5b6c06aed0 0x7f5b6c06d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b84036000 tx=0x7f5b8403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.246+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5b7c054180 con 0x7f5b9415b020 2024-01-31T05:02:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.426+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f5b94067dc0 con 0x7f5b9415b020 2024-01-31T05:02:49.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.427+0000 7f5b8b400640 1 -- 172.21.15.97:0/1217303944 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7f5b7c0577a0 con 0x7f5b9415b020 2024-01-31T05:02:49.426 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5b6c06aed0 msgr2=0x7f5b6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5b6c06aed0 0x7f5b6c06d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5b84036000 tx=0x7f5b8403d040 comp rx=0 tx=0).stop 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 msgr2=0x7f5b940cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b940cceb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c00c460 tx=0x7f5b7c00c920 comp rx=0 tx=0).stop 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 shutdown_connections 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9415a2d0 0x7f5b94153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5b6c06aed0 0x7f5b6c06d390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b9415b020 0x7f5b940cceb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 --2- 172.21.15.97:0/1217303944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b940b4ab0 0x7f5b94158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.429+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 >> 172.21.15.97:0/1217303944 conn(0x7f5b940b1b00 msgr2=0x7f5b940cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:49.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.430+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 shutdown_connections 2024-01-31T05:02:49.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.430+0000 7f5b98c00640 1 -- 172.21.15.97:0/1217303944 wait complete. 2024-01-31T05:02:49.448 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:02:49.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.646+0000 7f5305a00640 1 Processor -- start 2024-01-31T05:02:49.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.647+0000 7f5305a00640 1 -- start start 2024-01-31T05:02:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f5305a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f530006f280 0x7f5300073010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f5305a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f53000cb6a0 0x7f53000cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f5305a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5300073710 con 0x7f530006d050 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f5305a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000ca6f0 con 0x7f530006f280 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f5305a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530006f6e0 con 0x7f53000cb6a0 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f52fe000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f530006f280 0x7f5300073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f52fea00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f530006d050 0x7f530006d430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:56122/0 (socket says 172.21.15.97:56122) 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f52fe000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f530006f280 0x7f5300073010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35716/0 (socket says 172.21.15.97:35716) 2024-01-31T05:02:49.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f52fea00640 1 -- 172.21.15.97:0/1960090128 learned_addr learned my addr 172.21.15.97:0/1960090128 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:49.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f52fe000640 1 -- 172.21.15.97:0/1960090128 >> v2:172.21.15.188:3300/0 conn(0x7f53000cb6a0 msgr2=0x7f53000cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:49.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.648+0000 7f52fe000640 1 --2- 172.21.15.97:0/1960090128 >> v2:172.21.15.188:3300/0 conn(0x7f53000cb6a0 0x7f53000cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.649+0000 7f52fe000640 1 -- 172.21.15.97:0/1960090128 >> v1:172.21.15.97:6789/0 conn(0x7f530006d050 legacy=0x7f530006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.649+0000 7f52fe000640 1 -- 172.21.15.97:0/1960090128 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000aed80 con 0x7f530006f280 2024-01-31T05:02:49.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f52fe000640 1 --2- 172.21.15.97:0/1960090128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006f280 0x7f5300073010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5300070d90 tx=0x7f52e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b098513b01861c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f52fd600640 1 -- 172.21.15.97:0/1960090128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8031830 con 0x7f530006f280 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f52fd600640 1 -- 172.21.15.97:0/1960090128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e8031990 con 0x7f530006f280 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f52fd600640 1 -- 172.21.15.97:0/1960090128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8031d20 con 0x7f530006f280 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f5305a00640 1 -- 172.21.15.97:0/1960090128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006f280 msgr2=0x7f5300073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f5305a00640 1 --2- 172.21.15.97:0/1960090128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006f280 0x7f5300073010 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5300070d90 tx=0x7f52e802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f5305a00640 1 -- 172.21.15.97:0/1960090128 shutdown_connections 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f5305a00640 1 --2- 172.21.15.97:0/1960090128 >> v2:172.21.15.188:3300/0 conn(0x7f53000cb6a0 0x7f53000cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f5305a00640 1 --2- 172.21.15.97:0/1960090128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006f280 0x7f5300073010 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.650+0000 7f5305a00640 1 -- 172.21.15.97:0/1960090128 >> 172.21.15.97:0/1960090128 conn(0x7f53000cf6e0 msgr2=0x7f53000cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:49.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.651+0000 7f5305a00640 1 -- 172.21.15.97:0/1960090128 shutdown_connections 2024-01-31T05:02:49.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.651+0000 7f5305a00640 1 -- 172.21.15.97:0/1960090128 wait complete. 2024-01-31T05:02:49.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.652+0000 7f5305a00640 1 Processor -- start 2024-01-31T05:02:49.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.652+0000 7f5305a00640 1 -- start start 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f5305a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 0x7f530015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f5305a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f530006f280 0x7f5300152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f5305a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000cb6a0 0x7f5300152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f5305a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530015bb80 con 0x7f530006d050 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f5305a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530015b5c0 con 0x7f53000cb6a0 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f5305a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530015b8a0 con 0x7f530006f280 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f52fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 0x7f530015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f52fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 0x7f530015ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35720/0 (socket says 172.21.15.97:35720) 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.653+0000 7f52fea00640 1 -- 172.21.15.97:0/783110173 learned_addr learned my addr 172.21.15.97:0/783110173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52fea00640 1 -- 172.21.15.97:0/783110173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f530006f280 msgr2=0x7f5300152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52fea00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f530006f280 0x7f5300152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52fea00640 1 -- 172.21.15.97:0/783110173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000cb6a0 msgr2=0x7f5300152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52fea00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000cb6a0 0x7f5300152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52fea00640 1 -- 172.21.15.97:0/783110173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8009400 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52fea00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 0x7f530015ad70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f52f000cb10 tx=0x7f52f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000ac70 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f000add0 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0012620 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5300159a40 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.654+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530015b2b0 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.655+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53000738a0 con 0x7f530006d050 2024-01-31T05:02:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.656+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f52f0007070 con 0x7f530006d050 2024-01-31T05:02:49.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.657+0000 7f52f6a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f52d806aed0 0x7f52d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:49.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.657+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f52f00898f0 con 0x7f530006d050 2024-01-31T05:02:49.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.657+0000 7f52fe000640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f52d806aed0 0x7f52d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:49.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.658+0000 7f52fe000640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f52d806aed0 0x7f52d806d390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52e8036000 tx=0x7f52e8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.661+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f52f00544e0 con 0x7f530006d050 2024-01-31T05:02:49.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.867+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5300159f60 con 0x7f530006d050 2024-01-31T05:02:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.868+0000 7f52f6a00640 1 -- 172.21.15.97:0/783110173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f52f0017020 con 0x7f530006d050 2024-01-31T05:02:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f52d806aed0 msgr2=0x7f52d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f52d806aed0 0x7f52d806d390 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52e8036000 tx=0x7f52e8030000 comp rx=0 tx=0).stop 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 msgr2=0x7f530015ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 0x7f530015ad70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f52f000cb10 tx=0x7f52f000a060 comp rx=0 tx=0).stop 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 shutdown_connections 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000cb6a0 0x7f5300152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f530006f280 0x7f5300152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f52d806aed0 0x7f52d806d390 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 --2- 172.21.15.97:0/783110173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f530006d050 0x7f530015ad70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 >> 172.21.15.97:0/783110173 conn(0x7f53000cf6e0 msgr2=0x7f53000cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 shutdown_connections 2024-01-31T05:02:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:49.871+0000 7f5305a00640 1 -- 172.21.15.97:0/783110173 wait complete. 2024-01-31T05:02:49.890 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:02:49.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:49.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:49.977 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[52785]: fuse finished with error 0 and tester_r 0 2024-01-31T05:02:49.982 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:02:56.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:02:56.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:56.084 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:02:56.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:02:56.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.353+0000 7f6654600640 1 Processor -- start 2024-01-31T05:02:56.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.354+0000 7f6654600640 1 -- start start 2024-01-31T05:02:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.355+0000 7f6654600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f664c0cb810 0x7f664c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.355+0000 7f6654600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c15b400 0x7f664c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.355+0000 7f6654600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f664c06db70 0x7f664c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.355+0000 7f6654600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0cc960 con 0x7f664c0cb810 2024-01-31T05:02:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.355+0000 7f6654600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c06d1a0 con 0x7f664c06db70 2024-01-31T05:02:56.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.355+0000 7f6654600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c06c600 con 0x7f664c15b400 2024-01-31T05:02:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f664c0cb810 0x7f664c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f664c0cb810 0x7f664c0cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49736/0 (socket says 172.21.15.97:49736) 2024-01-31T05:02:56.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 -- 172.21.15.97:0/2445232441 learned_addr learned my addr 172.21.15.97:0/2445232441 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:56.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6652600640 1 --2- 172.21.15.97:0/2445232441 >> v2:172.21.15.188:3300/0 conn(0x7f664c06db70 0x7f664c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 -- 172.21.15.97:0/2445232441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c15b400 msgr2=0x7f664c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:56.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651200640 1 --2- 172.21.15.97:0/2445232441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c15b400 0x7f664c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 --2- 172.21.15.97:0/2445232441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c15b400 0x7f664c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 -- 172.21.15.97:0/2445232441 >> v2:172.21.15.188:3300/0 conn(0x7f664c06db70 msgr2=0x7f664c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:56.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 --2- 172.21.15.97:0/2445232441 >> v2:172.21.15.188:3300/0 conn(0x7f664c06db70 0x7f664c0ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6651c00640 1 -- 172.21.15.97:0/2445232441 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0cc130 con 0x7f664c0cb810 2024-01-31T05:02:56.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.356+0000 7f6652600640 1 --2- 172.21.15.97:0/2445232441 >> v2:172.21.15.188:3300/0 conn(0x7f664c06db70 0x7f664c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:56.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.358+0000 7f6651c00640 1 --2- 172.21.15.97:0/2445232441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0cbbf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f664c06c1c0 tx=0x7f663c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731735f8a55581a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:56.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.358+0000 7f664be00640 1 -- 172.21.15.97:0/2445232441 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c031870 con 0x7f664c0cb810 2024-01-31T05:02:56.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.358+0000 7f664be00640 1 -- 172.21.15.97:0/2445232441 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663c0319d0 con 0x7f664c0cb810 2024-01-31T05:02:56.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.358+0000 7f664be00640 1 -- 172.21.15.97:0/2445232441 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663c031d60 con 0x7f664c0cb810 2024-01-31T05:02:56.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 -- 172.21.15.97:0/2445232441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 msgr2=0x7f664c0cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:56.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 --2- 172.21.15.97:0/2445232441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0cbbf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f664c06c1c0 tx=0x7f663c02b910 comp rx=0 tx=0).stop 2024-01-31T05:02:56.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 -- 172.21.15.97:0/2445232441 shutdown_connections 2024-01-31T05:02:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 --2- 172.21.15.97:0/2445232441 >> v2:172.21.15.188:3300/0 conn(0x7f664c06db70 0x7f664c0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 --2- 172.21.15.97:0/2445232441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c15b400 0x7f664c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 --2- 172.21.15.97:0/2445232441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0cbbf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 -- 172.21.15.97:0/2445232441 >> 172.21.15.97:0/2445232441 conn(0x7f664c0b0a10 msgr2=0x7f664c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 -- 172.21.15.97:0/2445232441 shutdown_connections 2024-01-31T05:02:56.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.359+0000 7f6654600640 1 -- 172.21.15.97:0/2445232441 wait complete. 2024-01-31T05:02:56.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.361+0000 7f6654600640 1 Processor -- start 2024-01-31T05:02:56.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.361+0000 7f6654600640 1 -- start start 2024-01-31T05:02:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.362+0000 7f6654600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c06db70 0x7f664c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.362+0000 7f6654600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.362+0000 7f6654600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f664c15b400 0x7f664c0ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.362+0000 7f6654600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c09c870 con 0x7f664c0cb810 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.362+0000 7f6654600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c153020 con 0x7f664c15b400 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6654600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c15b9b0 con 0x7f664c06db70 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49744/0 (socket says 172.21.15.97:49744) 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 -- 172.21.15.97:0/2339624911 learned_addr learned my addr 172.21.15.97:0/2339624911 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651c00640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c06db70 0x7f664c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6652600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f664c15b400 0x7f664c0ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 -- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c06db70 msgr2=0x7f664c152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:56.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c06db70 0x7f664c152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 -- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f664c15b400 msgr2=0x7f664c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f664c15b400 0x7f664c0ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651200640 1 -- 172.21.15.97:0/2339624911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0094d0 con 0x7f664c0cb810 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6651c00640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c06db70 0x7f664c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.363+0000 7f6652600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f664c15b400 0x7f664c0ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.364+0000 7f6651200640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0abcf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f663400c7c0 tx=0x7f663400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.364+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634007820 con 0x7f664c0cb810 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.364+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634007980 con 0x7f664c0cb810 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.364+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634007cf0 con 0x7f664c0cb810 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.364+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c15bc50 con 0x7f664c0cb810 2024-01-31T05:02:56.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.364+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c15c210 con 0x7f664c0cb810 2024-01-31T05:02:56.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.365+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c0677c0 con 0x7f664c0cb810 2024-01-31T05:02:56.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.366+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f66340113f0 con 0x7f664c0cb810 2024-01-31T05:02:56.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.367+0000 7f6649600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f662406aed0 0x7f662406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:56.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.368+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f66340895c0 con 0x7f664c0cb810 2024-01-31T05:02:56.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.368+0000 7f6651c00640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f662406aed0 0x7f662406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:56.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.368+0000 7f6651c00640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f662406aed0 0x7f662406d390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f663c036000 tx=0x7f663c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:56.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.372+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f66340541b0 con 0x7f664c0cb810 2024-01-31T05:02:56.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.607+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f664c0b32e0 con 0x7f664c0cb810 2024-01-31T05:02:56.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.608+0000 7f6649600640 1 -- 172.21.15.97:0/2339624911 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+82 (secure 0 0 0) 0x7f66340577d0 con 0x7f664c0cb810 2024-01-31T05:02:56.607 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-01-31T05:02:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.610+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f662406aed0 msgr2=0x7f662406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.610+0000 7f6654600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f662406aed0 0x7f662406d390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f663c036000 tx=0x7f663c03c040 comp rx=0 tx=0).stop 2024-01-31T05:02:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.610+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 msgr2=0x7f664c0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:56.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.610+0000 7f6654600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0abcf0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f663400c7c0 tx=0x7f663400cc80 comp rx=0 tx=0).stop 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 shutdown_connections 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f664c15b400 0x7f664c0ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f662406aed0 0x7f662406d390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f664c0cb810 0x7f664c0abcf0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 --2- 172.21.15.97:0/2339624911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f664c06db70 0x7f664c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 >> 172.21.15.97:0/2339624911 conn(0x7f664c0b0a10 msgr2=0x7f664c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 shutdown_connections 2024-01-31T05:02:56.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:56.611+0000 7f6654600640 1 -- 172.21.15.97:0/2339624911 wait complete. 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:02:56.629 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:02:56.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:56.630 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:02:56.680 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:02:56.680 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:02:56.680 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:02:56.680 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:56.681 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42347sec preferred_lft 42347sec 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether ca:84:a9:c5:b2:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:56.682 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c884:a9ff:fec5:b219/64 scope link 2024-01-31T05:02:56.683 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:02:56.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:56.684 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:02:56.770 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:02:56.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:56.774 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:02:56.826 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:02:56.826 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:02:56.826 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:02:56.826 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:02:56.826 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:02:56.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:56.827 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:02:56.827 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:02:56.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:02:56.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:02:56.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:02:56.827 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:02:57.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:57.005 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:02:57.054 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:02:57.054 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:02:57.055 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:02:57.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:57.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:02:57.149 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:02:57.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:02:57.149 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:57.201 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:02:57.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:02:57.201 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:57.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:02:57.345 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:02:57.400 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:02:57.400 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:02:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:57.403 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:57.403 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:57.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:57.404 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:02:57.454 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:02:57.454 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:02:57.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:02:57.530 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:02:57.573 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:02:57.574 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:02:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:02:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:02:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:02:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:57.575 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:02:57.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:57.576 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:02:57.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:02:57.646+0000 7f7ac3c35480 -1 init, newargv = 0x562a7e363bc0 newargc=15 2024-01-31T05:02:57.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[54037]: starting ceph client 2024-01-31T05:02:57.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[54037]: starting fuse 2024-01-31T05:02:57.663 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:02:57.663 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:02:57.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:02:57.663 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:02:57.663 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:02:57.663 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:02:57.663 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:02:57.664 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:02:57.665 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:02:57.850 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.54037.asok 2024-01-31T05:02:57.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:02:57.867 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54037.asok 2024-01-31T05:02:57.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:57.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54037.asok status 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "54037", 2024-01-31T05:02:58.104 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5865, 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 5865 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2020504050 2024-01-31T05:02:58.105 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2020504050 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.5865 192.168.0.1:0/2020504050", 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2020504050", 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 18, 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 43, 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:02:58.106 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:02:58.107 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:02:58.107 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:02:58.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:58.119 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:02:58.119 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:58.171 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:02:58.171 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:58.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:58.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:02:58.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:58.264 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-01-31T05:02:58.321 INFO:teuthology.orchestra.run.smithi097.stderr:setfattr: layoutfile: Permission denied 2024-01-31T05:02:58.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:02:58.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:02:58.322 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-01-31T05:02:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:setfattr: layoutdir: Permission denied 2024-01-31T05:02:58.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:02:58.376 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:02:58.376 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:02:58.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:02:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.677+0000 7f1ffb400640 1 Processor -- start 2024-01-31T05:02:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.677+0000 7f1ffb400640 1 -- start start 2024-01-31T05:02:58.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1ffb400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff406e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1ffb400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40aea50 0x7f1ff406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1ffb400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ff406f0c0 0x7f1ff40c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1ffb400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff40cf570 con 0x7f1ff40aea50 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1ffb400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff40cec20 con 0x7f1ff406f0c0 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1ffb400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff406f550 con 0x7f1ff406e260 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1febc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40aea50 0x7f1ff406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1febc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ff40aea50 0x7f1ff406eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49752/0 (socket says 172.21.15.97:49752) 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.679+0000 7f1febc00640 1 -- 172.21.15.97:0/3818701750 learned_addr learned my addr 172.21.15.97:0/3818701750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1ff9400640 1 --2- 172.21.15.97:0/3818701750 >> v2:172.21.15.188:3300/0 conn(0x7f1ff406f0c0 0x7f1ff40c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1febc00640 1 -- 172.21.15.97:0/3818701750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 msgr2=0x7f1ff406e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1ff8a00640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff406e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1febc00640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff406e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1febc00640 1 -- 172.21.15.97:0/3818701750 >> v2:172.21.15.188:3300/0 conn(0x7f1ff406f0c0 msgr2=0x7f1ff40c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1febc00640 1 --2- 172.21.15.97:0/3818701750 >> v2:172.21.15.188:3300/0 conn(0x7f1ff406f0c0 0x7f1ff40c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1febc00640 1 -- 172.21.15.97:0/3818701750 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40ce010 con 0x7f1ff40aea50 2024-01-31T05:02:58.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1ff9400640 1 --2- 172.21.15.97:0/3818701750 >> v2:172.21.15.188:3300/0 conn(0x7f1ff406f0c0 0x7f1ff40c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:58.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.680+0000 7f1ff8a00640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:58.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.681+0000 7f1febc00640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40aea50 0x7f1ff406eb80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40af290 tx=0x7f1fdc02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecbfeb08a5009a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.681+0000 7f1ff3e00640 1 -- 172.21.15.97:0/3818701750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0317f0 con 0x7f1ff40aea50 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.681+0000 7f1ff3e00640 1 -- 172.21.15.97:0/3818701750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc031950 con 0x7f1ff40aea50 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.681+0000 7f1ff3e00640 1 -- 172.21.15.97:0/3818701750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc031ce0 con 0x7f1ff40aea50 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 -- 172.21.15.97:0/3818701750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40aea50 msgr2=0x7f1ff406eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40aea50 0x7f1ff406eb80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40af290 tx=0x7f1fdc02b890 comp rx=0 tx=0).stop 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 -- 172.21.15.97:0/3818701750 shutdown_connections 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 --2- 172.21.15.97:0/3818701750 >> v2:172.21.15.188:3300/0 conn(0x7f1ff406f0c0 0x7f1ff40c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff40aea50 0x7f1ff406eb80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 --2- 172.21.15.97:0/3818701750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff406e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 -- 172.21.15.97:0/3818701750 >> 172.21.15.97:0/3818701750 conn(0x7f1ff40cf6e0 msgr2=0x7f1ff40cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 -- 172.21.15.97:0/3818701750 shutdown_connections 2024-01-31T05:02:58.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.682+0000 7f1ffb400640 1 -- 172.21.15.97:0/3818701750 wait complete. 2024-01-31T05:02:58.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.684+0000 7f1ffb400640 1 Processor -- start 2024-01-31T05:02:58.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.684+0000 7f1ffb400640 1 -- start start 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ffb400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff41471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ffb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 0x7f1ff40cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ffb400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff40aea50 0x7f1ff40c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ffb400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff41554b0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ffb400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff41569b0 con 0x7f1ff40aea50 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ffb400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4147730 con 0x7f1ff406e260 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1febc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 0x7f1ff40cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1febc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 0x7f1ff40cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49762/0 (socket says 172.21.15.97:49762) 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1febc00640 1 -- 172.21.15.97:0/2843784348 learned_addr learned my addr 172.21.15.97:0/2843784348 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ff8a00640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff41471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.685+0000 7f1ff9400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff40aea50 0x7f1ff40c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1febc00640 1 -- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 msgr2=0x7f1ff41471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1febc00640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff41471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1febc00640 1 -- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff40aea50 msgr2=0x7f1ff40c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1febc00640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff40aea50 0x7f1ff40c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1febc00640 1 -- 172.21.15.97:0/2843784348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc0094d0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1ff9400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff40aea50 0x7f1ff40c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1febc00640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 0x7f1ff40cb420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000c00 tx=0x7f1fdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc032990 con 0x7f1ff406f0c0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc032af0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc032ea0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.686+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff4154850 con 0x7f1ff406f0c0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.687+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff4154df0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.688+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff40738a0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.689+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1fdc04d3f0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.689+0000 7f1ff1600640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fcc06aed0 0x7f1fcc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.690+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f1fdc0aba90 con 0x7f1ff406f0c0 2024-01-31T05:02:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.690+0000 7f1ff8a00640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fcc06aed0 0x7f1fcc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.690+0000 7f1ff8a00640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fcc06aed0 0x7f1fcc06d390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400a990 tx=0x7f1fe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.694+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1fdc076690 con 0x7f1ff406f0c0 2024-01-31T05:02:58.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.888+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ff41555f0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.890+0000 7f1ff1600640 1 -- 172.21.15.97:0/2843784348 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f1ff41555f0 con 0x7f1ff406f0c0 2024-01-31T05:02:58.890 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:02:58.890 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:02:58.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.892+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fcc06aed0 msgr2=0x7f1fcc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.892+0000 7f1ffb400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fcc06aed0 0x7f1fcc06d390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400a990 tx=0x7f1fe4008040 comp rx=0 tx=0).stop 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 msgr2=0x7f1ff40cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 0x7f1ff40cb420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000c00 tx=0x7f1fdc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 shutdown_connections 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff40aea50 0x7f1ff40c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1fcc06aed0 0x7f1fcc06d390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff406f0c0 0x7f1ff40cb420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 --2- 172.21.15.97:0/2843784348 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff406e260 0x7f1ff41471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:58.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 >> 172.21.15.97:0/2843784348 conn(0x7f1ff40cf6e0 msgr2=0x7f1ff4157dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:58.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 shutdown_connections 2024-01-31T05:02:58.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:58.893+0000 7f1ffb400640 1 -- 172.21.15.97:0/2843784348 wait complete. 2024-01-31T05:02:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:02:58.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:02:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.123+0000 7f8e3e200640 1 Processor -- start 2024-01-31T05:02:59.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.124+0000 7f8e3e200640 1 -- start start 2024-01-31T05:02:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e3e200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8e3806fc70 0x7f8e3806a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e3e200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e380cac00 0x7f8e380cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e3e200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e380cc760 con 0x7f8e3815edf0 2024-01-31T05:02:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e3e200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3806b6b0 con 0x7f8e380cac00 2024-01-31T05:02:59.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e3e200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3806ae10 con 0x7f8e3806fc70 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e37e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e380cac00 0x7f8e380cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e37e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e380cac00 0x7f8e380cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49770/0 (socket says 172.21.15.97:49770) 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.125+0000 7f8e37e00640 1 -- 172.21.15.97:0/2637236719 learned_addr learned my addr 172.21.15.97:0/2637236719 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.126+0000 7f8e37400640 1 --2- 172.21.15.97:0/2637236719 >> v2:172.21.15.188:3300/0 conn(0x7f8e3806fc70 0x7f8e3806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.126+0000 7f8e37e00640 1 -- 172.21.15.97:0/2637236719 >> v2:172.21.15.188:3300/0 conn(0x7f8e3806fc70 msgr2=0x7f8e3806a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.126+0000 7f8e37e00640 1 --2- 172.21.15.97:0/2637236719 >> v2:172.21.15.188:3300/0 conn(0x7f8e3806fc70 0x7f8e3806a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.126+0000 7f8e37e00640 1 -- 172.21.15.97:0/2637236719 >> v1:172.21.15.188:6789/0 conn(0x7f8e3815edf0 legacy=0x7f8e3806eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:59.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.126+0000 7f8e37e00640 1 -- 172.21.15.97:0/2637236719 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3806b0f0 con 0x7f8e380cac00 2024-01-31T05:02:59.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.127+0000 7f8e37e00640 1 --2- 172.21.15.97:0/2637236719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380cb010 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e3806ce70 tx=0x7f8e3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0951648b5bc729 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:59.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.127+0000 7f8e36000640 1 -- 172.21.15.97:0/2637236719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30031870 con 0x7f8e380cac00 2024-01-31T05:02:59.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.127+0000 7f8e36000640 1 -- 172.21.15.97:0/2637236719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e300319d0 con 0x7f8e380cac00 2024-01-31T05:02:59.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.127+0000 7f8e36000640 1 -- 172.21.15.97:0/2637236719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30031d60 con 0x7f8e380cac00 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 -- 172.21.15.97:0/2637236719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 msgr2=0x7f8e380cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 --2- 172.21.15.97:0/2637236719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380cb010 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e3806ce70 tx=0x7f8e3002b910 comp rx=0 tx=0).stop 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 -- 172.21.15.97:0/2637236719 shutdown_connections 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 --2- 172.21.15.97:0/2637236719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380cb010 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 --2- 172.21.15.97:0/2637236719 >> v2:172.21.15.188:3300/0 conn(0x7f8e3806fc70 0x7f8e3806a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 -- 172.21.15.97:0/2637236719 >> 172.21.15.97:0/2637236719 conn(0x7f8e380cc970 msgr2=0x7f8e380ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 -- 172.21.15.97:0/2637236719 shutdown_connections 2024-01-31T05:02:59.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.128+0000 7f8e3e200640 1 -- 172.21.15.97:0/2637236719 wait complete. 2024-01-31T05:02:59.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.130+0000 7f8e3e200640 1 Processor -- start 2024-01-31T05:02:59.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.130+0000 7f8e3e200640 1 -- start start 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e3e200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e3806fc70 0x7f8e380c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e3e200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e3e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3815edf0 0x7f8e3815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e3e200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380c9ba0 con 0x7f8e380cac00 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e3e200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380c9e80 con 0x7f8e3806fc70 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e3e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e380ca160 con 0x7f8e3815edf0 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e36a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e36a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49786/0 (socket says 172.21.15.97:49786) 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e36a00640 1 -- 172.21.15.97:0/3168247170 learned_addr learned my addr 172.21.15.97:0/3168247170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.131+0000 7f8e37400640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e3806fc70 0x7f8e380c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e37e00640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3815edf0 0x7f8e3815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e36a00640 1 -- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3815edf0 msgr2=0x7f8e3815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e36a00640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3815edf0 0x7f8e3815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e36a00640 1 -- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e3806fc70 msgr2=0x7f8e380c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e36a00640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e3806fc70 0x7f8e380c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e36a00640 1 -- 172.21.15.97:0/3168247170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300094d0 con 0x7f8e380cac00 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e36a00640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380c79f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800ba50 tx=0x7f8e2800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2800cc10 con 0x7f8e380cac00 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2800cd70 con 0x7f8e380cac00 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28014680 con 0x7f8e380cac00 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.132+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e380b3680 con 0x7f8e380cac00 2024-01-31T05:02:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.133+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e380c7f30 con 0x7f8e380cac00 2024-01-31T05:02:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.134+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e380ca6a0 con 0x7f8e380cac00 2024-01-31T05:02:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.135+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8e2801d3b0 con 0x7f8e380cac00 2024-01-31T05:02:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.135+0000 7f8e2f400640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1006aed0 0x7f8e1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.135+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f8e28089960 con 0x7f8e380cac00 2024-01-31T05:02:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.136+0000 7f8e37400640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1006aed0 0x7f8e1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.136+0000 7f8e37400640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1006aed0 0x7f8e1006d390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e20006fd0 tx=0x7f8e20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:59.139 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.140+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8e28054560 con 0x7f8e380cac00 2024-01-31T05:02:59.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.333+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e38067830 con 0x7f8e380cac00 2024-01-31T05:02:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.334+0000 7f8e2f400640 1 -- 172.21.15.97:0/3168247170 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e280190a0 con 0x7f8e380cac00 2024-01-31T05:02:59.333 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:02:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.336+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1006aed0 msgr2=0x7f8e1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.336+0000 7f8e3e200640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1006aed0 0x7f8e1006d390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e20006fd0 tx=0x7f8e20008040 comp rx=0 tx=0).stop 2024-01-31T05:02:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.336+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 msgr2=0x7f8e380c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.336+0000 7f8e3e200640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380c79f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800ba50 tx=0x7f8e2800bf10 comp rx=0 tx=0).stop 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 shutdown_connections 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3815edf0 0x7f8e3815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e1006aed0 0x7f8e1006d390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e380cac00 0x7f8e380c79f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 --2- 172.21.15.97:0/3168247170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e3806fc70 0x7f8e380c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 >> 172.21.15.97:0/3168247170 conn(0x7f8e380cc970 msgr2=0x7f8e3806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 shutdown_connections 2024-01-31T05:02:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.337+0000 7f8e3e200640 1 -- 172.21.15.97:0/3168247170 wait complete. 2024-01-31T05:02:59.355 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:02:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.569+0000 7fa627c00640 1 Processor -- start 2024-01-31T05:02:59.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.569+0000 7fa627c00640 1 -- start start 2024-01-31T05:02:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa627c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa62015d590 0x7fa62015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa627c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 0x7fa6200cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa627c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6200d0460 con 0x7fa620161080 2024-01-31T05:02:59.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa627c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6200cded0 con 0x7fa62015d590 2024-01-31T05:02:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa627c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62015d970 con 0x7fa6200cfae0 2024-01-31T05:02:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa625c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa620161080 0x7fa620161510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35022/0 (socket says 172.21.15.97:35022) 2024-01-31T05:02:59.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa625c00640 1 -- 172.21.15.97:0/1638499058 learned_addr learned my addr 172.21.15.97:0/1638499058 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa61fe00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 0x7fa6200cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa625200640 1 --2- 172.21.15.97:0/1638499058 >> v2:172.21.15.188:3300/0 conn(0x7fa62015d590 0x7fa62015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.571+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824770855 0 0) 0x7fa6200d0460 con 0x7fa620161080 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa600003520 con 0x7fa620161080 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1425320838 0 0) 0x7fa600003520 con 0x7fa620161080 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 msgr2=0x7fa6200cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 0x7fa6200cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 >> v2:172.21.15.188:3300/0 conn(0x7fa62015d590 msgr2=0x7fa62015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 --2- 172.21.15.97:0/1638499058 >> v2:172.21.15.188:3300/0 conn(0x7fa62015d590 0x7fa62015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6200cf210 con 0x7fa620161080 2024-01-31T05:02:59.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa6180019f0 con 0x7fa620161080 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 >> v1:172.21.15.97:6789/0 conn(0x7fa620161080 legacy=0x7fa620161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 0x7fa600003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600003ef0 0x7fa6000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa600004d00 0x7fa600005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6200cded0 con 0x7fa600004d00 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62015d970 con 0x7fa600003ef0 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60000b9b0 con 0x7fa6200cfae0 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.572+0000 7fa625c00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa600004d00 0x7fa600005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa625200640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 0x7fa600003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa61fe00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600003ef0 0x7fa6000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa625c00640 1 -- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 msgr2=0x7fa600003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa625c00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cfae0 0x7fa600003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa625c00640 1 -- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600003ef0 msgr2=0x7fa6000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:02:59.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa625c00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600003ef0 0x7fa6000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.573+0000 7fa625c00640 1 -- 172.21.15.97:0/1638499058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa600003520 con 0x7fa600004d00 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.574+0000 7fa625c00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa600004d00 0x7fa600005180 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa61800a880 tx=0x7fa6000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc421066b9804112 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.574+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618034630 con 0x7fa600004d00 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.574+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61802f070 con 0x7fa600004d00 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.574+0000 7fa61f400640 1 -- 172.21.15.97:0/1638499058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618034a30 con 0x7fa600004d00 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa627c00640 1 -- 172.21.15.97:0/1638499058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa600004d00 msgr2=0x7fa600005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa627c00640 1 --2- 172.21.15.97:0/1638499058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa600004d00 0x7fa600005180 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa61800a880 tx=0x7fa6000057f0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa625c00640 1 -- 172.21.15.97:0/1638499058 reap_dead start 2024-01-31T05:02:59.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa627c00640 1 -- 172.21.15.97:0/1638499058 shutdown_connections 2024-01-31T05:02:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa627c00640 1 -- 172.21.15.97:0/1638499058 >> 172.21.15.97:0/1638499058 conn(0x7fa62009c560 msgr2=0x7fa6200d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa627c00640 1 -- 172.21.15.97:0/1638499058 shutdown_connections 2024-01-31T05:02:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.575+0000 7fa627c00640 1 -- 172.21.15.97:0/1638499058 wait complete. 2024-01-31T05:02:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.577+0000 7fa627c00640 1 Processor -- start 2024-01-31T05:02:59.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.577+0000 7fa627c00640 1 -- start start 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa627c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600004d00 0x7fa6200c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa627c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 0x7fa6200cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa627c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cb870 0x7fa6200cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa627c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62014b550 con 0x7fa6200cc700 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa627c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6200c9f00 con 0x7fa600004d00 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa627c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62014b7a0 con 0x7fa6200cb870 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa61fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 0x7fa6200cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa61fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 0x7fa6200cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49796/0 (socket says 172.21.15.97:49796) 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa61fe00640 1 -- 172.21.15.97:0/4038865086 learned_addr learned my addr 172.21.15.97:0/4038865086 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:02:59.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa625200640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600004d00 0x7fa6200c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.578+0000 7fa625c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cb870 0x7fa6200cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61fe00640 1 -- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cb870 msgr2=0x7fa6200cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61fe00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cb870 0x7fa6200cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61fe00640 1 -- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600004d00 msgr2=0x7fa6200c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61fe00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600004d00 0x7fa6200c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61fe00640 1 -- 172.21.15.97:0/4038865086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62014ba80 con 0x7fa6200cc700 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61fe00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 0x7fa6200cb330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa608011540 tx=0x7fa608011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608015930 con 0x7fa6200cc700 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608015a90 con 0x7fa6200cc700 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608015e60 con 0x7fa6200cc700 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62014bd60 con 0x7fa6200cc700 2024-01-31T05:02:59.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.579+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62009e590 con 0x7fa6200cc700 2024-01-31T05:02:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.581+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa620067830 con 0x7fa6200cc700 2024-01-31T05:02:59.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.581+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa60800f460 con 0x7fa6200cc700 2024-01-31T05:02:59.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.583+0000 7fa61cc00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5f806af00 0x7fa5f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:02:59.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.583+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa60808a140 con 0x7fa6200cc700 2024-01-31T05:02:59.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.583+0000 7fa625200640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5f806af00 0x7fa5f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:02:59.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.584+0000 7fa625200640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5f806af00 0x7fa5f806d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa610009a90 tx=0x7fa610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:02:59.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.587+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa608054ca0 con 0x7fa6200cc700 2024-01-31T05:02:59.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.779+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6200b5b60 con 0x7fa6200cc700 2024-01-31T05:02:59.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.780+0000 7fa61cc00640 1 -- 172.21.15.97:0/4038865086 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6080582c0 con 0x7fa6200cc700 2024-01-31T05:02:59.779 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:02:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.782+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5f806af00 msgr2=0x7fa5f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.782+0000 7fa627c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5f806af00 0x7fa5f806d3c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa610009a90 tx=0x7fa610008040 comp rx=0 tx=0).stop 2024-01-31T05:02:59.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.782+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 msgr2=0x7fa6200cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.782+0000 7fa627c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 0x7fa6200cb330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa608011540 tx=0x7fa608011a00 comp rx=0 tx=0).stop 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 shutdown_connections 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6200cb870 0x7fa6200cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5f806af00 0x7fa5f806d3c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6200cc700 0x7fa6200cb330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 --2- 172.21.15.97:0/4038865086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa600004d00 0x7fa6200c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 >> 172.21.15.97:0/4038865086 conn(0x7fa62009c560 msgr2=0x7fa62006c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 shutdown_connections 2024-01-31T05:02:59.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:02:59.783+0000 7fa627c00640 1 -- 172.21.15.97:0/4038865086 wait complete. 2024-01-31T05:02:59.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:00.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.012+0000 7ffa56400640 1 Processor -- start 2024-01-31T05:03:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.012+0000 7ffa56400640 1 -- start start 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.013+0000 7ffa56400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa5015a350 0x7ffa500abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa56400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa500cde10 0x7ffa500ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa56400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5015abc0 con 0x7ffa5015a730 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa56400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa500ce7e0 con 0x7ffa5015a350 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa56400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa500b2c90 con 0x7ffa500cde10 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa5015a350 0x7ffa500abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa5015a350 0x7ffa500abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49808/0 (socket says 172.21.15.97:49808) 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 -- 172.21.15.97:0/796426632 learned_addr learned my addr 172.21.15.97:0/796426632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 -- 172.21.15.97:0/796426632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa500cde10 msgr2=0x7ffa500ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4fe00640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa500cde10 0x7ffa500ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa500cde10 0x7ffa500ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4ea00640 1 -- 172.21.15.97:0/796426632 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405736433 0 0) 0x7ffa5015abc0 con 0x7ffa5015a730 2024-01-31T05:03:00.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 -- 172.21.15.97:0/796426632 >> v1:172.21.15.97:6789/0 conn(0x7ffa5015a730 legacy=0x7ffa5006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4f400640 1 -- 172.21.15.97:0/796426632 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5015aea0 con 0x7ffa5015a350 2024-01-31T05:03:00.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.014+0000 7ffa4fe00640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa500cde10 0x7ffa500ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:00.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.016+0000 7ffa4f400640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5015a350 0x7ffa500abcf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009880 tx=0x7ffa4002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ed4ccdc2e8165e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.016+0000 7ffa4ea00640 1 -- 172.21.15.97:0/796426632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40004230 con 0x7ffa5015a350 2024-01-31T05:03:00.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.016+0000 7ffa4ea00640 1 -- 172.21.15.97:0/796426632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa40004390 con 0x7ffa5015a350 2024-01-31T05:03:00.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.016+0000 7ffa4ea00640 1 -- 172.21.15.97:0/796426632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40003810 con 0x7ffa5015a350 2024-01-31T05:03:00.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.016+0000 7ffa56400640 1 -- 172.21.15.97:0/796426632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5015a350 msgr2=0x7ffa500abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.016+0000 7ffa56400640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5015a350 0x7ffa500abcf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009880 tx=0x7ffa4002b960 comp rx=0 tx=0).stop 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.017+0000 7ffa56400640 1 -- 172.21.15.97:0/796426632 shutdown_connections 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.017+0000 7ffa56400640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa500cde10 0x7ffa500ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.017+0000 7ffa56400640 1 --2- 172.21.15.97:0/796426632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5015a350 0x7ffa500abcf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.017+0000 7ffa56400640 1 -- 172.21.15.97:0/796426632 >> 172.21.15.97:0/796426632 conn(0x7ffa500b0a10 msgr2=0x7ffa500b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.017+0000 7ffa56400640 1 -- 172.21.15.97:0/796426632 shutdown_connections 2024-01-31T05:03:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.017+0000 7ffa56400640 1 -- 172.21.15.97:0/796426632 wait complete. 2024-01-31T05:03:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.018+0000 7ffa56400640 1 Processor -- start 2024-01-31T05:03:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.019+0000 7ffa56400640 1 -- start start 2024-01-31T05:03:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa56400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa500cde10 0x7ffa500c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa56400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5015a350 0x7ffa5015d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa56400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 0x7ffa5015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa56400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5015b950 con 0x7ffa50150ac0 2024-01-31T05:03:00.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa56400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5015d9d0 con 0x7ffa500cde10 2024-01-31T05:03:00.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa56400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa500ca5e0 con 0x7ffa5015a350 2024-01-31T05:03:00.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 0x7ffa5015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 0x7ffa5015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49818/0 (socket says 172.21.15.97:49818) 2024-01-31T05:03:00.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 -- 172.21.15.97:0/641218547 learned_addr learned my addr 172.21.15.97:0/641218547 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4f400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa500cde10 0x7ffa500c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 -- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5015a350 msgr2=0x7ffa5015d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5015a350 0x7ffa5015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 -- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa500cde10 msgr2=0x7ffa500c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa500cde10 0x7ffa500c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.020+0000 7ffa4fe00640 1 -- 172.21.15.97:0/641218547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40009460 con 0x7ffa50150ac0 2024-01-31T05:03:00.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.021+0000 7ffa4fe00640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 0x7ffa5015c050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800cf20 tx=0x7ffa4800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.021+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800d840 con 0x7ffa50150ac0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.021+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4800d9a0 con 0x7ffa50150ac0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.021+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800dd50 con 0x7ffa50150ac0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.021+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa500c9d40 con 0x7ffa50150ac0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.021+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa500ca300 con 0x7ffa50150ac0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.022+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa50067830 con 0x7ffa50150ac0 2024-01-31T05:03:00.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.023+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ffa4801e3b0 con 0x7ffa50150ac0 2024-01-31T05:03:00.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.024+0000 7ffa47400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa2806aed0 0x7ffa2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.025+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7ffa4808a4a0 con 0x7ffa50150ac0 2024-01-31T05:03:00.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.025+0000 7ffa4f400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa2806aed0 0x7ffa2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.025+0000 7ffa4f400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa2806aed0 0x7ffa2806d390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa40035000 tx=0x7ffa40008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.029+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ffa48055000 con 0x7ffa50150ac0 2024-01-31T05:03:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.222+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa500cc9c0 con 0x7ffa50150ac0 2024-01-31T05:03:00.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.223+0000 7ffa47400640 1 -- 172.21.15.97:0/641218547 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa48058620 con 0x7ffa50150ac0 2024-01-31T05:03:00.222 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa2806aed0 msgr2=0x7ffa2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa2806aed0 0x7ffa2806d390 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa40035000 tx=0x7ffa40008030 comp rx=0 tx=0).stop 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 msgr2=0x7ffa5015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 0x7ffa5015c050 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800cf20 tx=0x7ffa4800a7d0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 shutdown_connections 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa2806aed0 0x7ffa2806d390 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa50150ac0 0x7ffa5015c050 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5015a350 0x7ffa5015d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 --2- 172.21.15.97:0/641218547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa500cde10 0x7ffa500c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.226+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 >> 172.21.15.97:0/641218547 conn(0x7ffa500b0a10 msgr2=0x7ffa500cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.227+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 shutdown_connections 2024-01-31T05:03:00.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.227+0000 7ffa56400640 1 -- 172.21.15.97:0/641218547 wait complete. 2024-01-31T05:03:00.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:03:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.458+0000 7fe33b000640 1 Processor -- start 2024-01-31T05:03:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.458+0000 7fe33b000640 1 -- start start 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe33b000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe3340cbf20 0x7fe3340722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe33b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 0x7fe3340ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe33b000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33406cd20 con 0x7fe33415b470 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe33b000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3340ce970 con 0x7fe3340cbf20 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe33b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33415b850 con 0x7fe3340abcf0 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe333e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe33415b470 0x7fe3340cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35030/0 (socket says 172.21.15.97:35030) 2024-01-31T05:03:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.460+0000 7fe333e00640 1 -- 172.21.15.97:0/1082910974 learned_addr learned my addr 172.21.15.97:0/1082910974 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe32b200640 1 --2- 172.21.15.97:0/1082910974 >> v2:172.21.15.188:3300/0 conn(0x7fe3340cbf20 0x7fe3340722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe339000640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 0x7fe3340ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317809231 0 0) 0x7fe33406cd20 con 0x7fe33415b470 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe314003520 con 0x7fe33415b470 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454779129 0 0) 0x7fe314003520 con 0x7fe33415b470 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 msgr2=0x7fe3340ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 0x7fe3340ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 >> v2:172.21.15.188:3300/0 conn(0x7fe3340cbf20 msgr2=0x7fe3340722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 --2- 172.21.15.97:0/1082910974 >> v2:172.21.15.188:3300/0 conn(0x7fe3340cbf20 0x7fe3340722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340ceb60 con 0x7fe33415b470 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fe3240017f0 con 0x7fe33415b470 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.461+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 >> v1:172.21.15.97:6789/0 conn(0x7fe33415b470 legacy=0x7fe3340cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333400640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3140039a0 0x7fe314003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333400640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 0x7fe3140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333400640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe314004d00 0x7fe314005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33415b850 con 0x7fe3140039a0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31400b980 con 0x7fe314004d00 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31400bac0 con 0x7fe3340abcf0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333e00640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3140039a0 0x7fe314003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333e00640 1 -- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 msgr2=0x7fe3140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe32b200640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 0x7fe3140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333e00640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3340abcf0 0x7fe3140047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333e00640 1 -- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe314004d00 msgr2=0x7fe314005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe339000640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe314004d00 0x7fe314005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333e00640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe314004d00 0x7fe314005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe333e00640 1 -- 172.21.15.97:0/1082910974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe314003520 con 0x7fe3140039a0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.462+0000 7fe339000640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe314004d00 0x7fe314005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe333e00640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3140039a0 0x7fe314003dc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c110 tx=0x7fe3140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08f0cd0dcebbbc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324007270 con 0x7fe3140039a0 2024-01-31T05:03:00.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3240073d0 con 0x7fe3140039a0 2024-01-31T05:03:00.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe333400640 1 -- 172.21.15.97:0/1082910974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32402c7c0 con 0x7fe3140039a0 2024-01-31T05:03:00.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe33b000640 1 -- 172.21.15.97:0/1082910974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3140039a0 msgr2=0x7fe314003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe33b000640 1 --2- 172.21.15.97:0/1082910974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3140039a0 0x7fe314003dc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c110 tx=0x7fe3140057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe339000640 1 -- 172.21.15.97:0/1082910974 reap_dead start 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe33b000640 1 -- 172.21.15.97:0/1082910974 shutdown_connections 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.464+0000 7fe33b000640 1 -- 172.21.15.97:0/1082910974 >> 172.21.15.97:0/1082910974 conn(0x7fe3340b0a10 msgr2=0x7fe3340b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.465+0000 7fe33b000640 1 -- 172.21.15.97:0/1082910974 shutdown_connections 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.465+0000 7fe33b000640 1 -- 172.21.15.97:0/1082910974 wait complete. 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.466+0000 7fe33b000640 1 Processor -- start 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.466+0000 7fe33b000640 1 -- start start 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.467+0000 7fe33b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3140039a0 0x7fe3340c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.467+0000 7fe33b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe33415e100 0x7fe334152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.467+0000 7fe33b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 0x7fe3340afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.467+0000 7fe33b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340b0110 con 0x7fe3340af750 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.467+0000 7fe33b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33409c950 con 0x7fe33415e100 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.467+0000 7fe33b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3340caca0 con 0x7fe3140039a0 2024-01-31T05:03:00.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 0x7fe3340afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 0x7fe3340afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49842/0 (socket says 172.21.15.97:49842) 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 -- 172.21.15.97:0/4273047615 learned_addr learned my addr 172.21.15.97:0/4273047615 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe32b200640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe33415e100 0x7fe334152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 -- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3140039a0 msgr2=0x7fe3340c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3140039a0 0x7fe3340c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 -- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe33415e100 msgr2=0x7fe334152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe33415e100 0x7fe334152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe339000640 1 -- 172.21.15.97:0/4273047615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340ca6e0 con 0x7fe3340af750 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.468+0000 7fe32b200640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe33415e100 0x7fe334152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.469+0000 7fe339000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 0x7fe3340afbd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009fd0 tx=0x7fe32c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.469+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c017400 con 0x7fe3340af750 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.469+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c011040 con 0x7fe3340af750 2024-01-31T05:03:00.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.469+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c0177c0 con 0x7fe3340af750 2024-01-31T05:03:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.469+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3340ca9c0 con 0x7fe3340af750 2024-01-31T05:03:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.469+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3340b2190 con 0x7fe3340af750 2024-01-31T05:03:00.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.470+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3340b23b0 con 0x7fe3340af750 2024-01-31T05:03:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.471+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe32c0203b0 con 0x7fe3340af750 2024-01-31T05:03:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.472+0000 7fe330c00640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe30c06aed0 0x7fe30c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.472+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fe32c089cf0 con 0x7fe3340af750 2024-01-31T05:03:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.472+0000 7fe333e00640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe30c06aed0 0x7fe30c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.473+0000 7fe333e00640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe30c06aed0 0x7fe30c06d390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe324034000 tx=0x7fe324007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.476+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe32c0548e0 con 0x7fe3340af750 2024-01-31T05:03:00.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.669+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fe334067140 con 0x7fe3340af750 2024-01-31T05:03:00.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.670+0000 7fe330c00640 1 -- 172.21.15.97:0/4273047615 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7fe32c01c020 con 0x7fe3340af750 2024-01-31T05:03:00.669 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.672+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe30c06aed0 msgr2=0x7fe30c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.672+0000 7fe33b000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe30c06aed0 0x7fe30c06d390 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe324034000 tx=0x7fe324007000 comp rx=0 tx=0).stop 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.672+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 msgr2=0x7fe3340afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.672+0000 7fe33b000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 0x7fe3340afbd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009fd0 tx=0x7fe32c00f4a0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 shutdown_connections 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe30c06aed0 0x7fe30c06d390 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3340af750 0x7fe3340afbd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe33415e100 0x7fe334152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 --2- 172.21.15.97:0/4273047615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3140039a0 0x7fe3340c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 >> 172.21.15.97:0/4273047615 conn(0x7fe3340b0a10 msgr2=0x7fe3340cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 shutdown_connections 2024-01-31T05:03:00.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.673+0000 7fe33b000640 1 -- 172.21.15.97:0/4273047615 wait complete. 2024-01-31T05:03:00.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:03:00.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.895+0000 7faa6e400640 1 Processor -- start 2024-01-31T05:03:00.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.895+0000 7faa6e400640 1 -- start start 2024-01-31T05:03:00.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa6e400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa6815d690 0x7faa680ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa6e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 0x7faa681614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa6e400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6815db40 con 0x7faa6806d580 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa6e400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6815e7c0 con 0x7faa6815d690 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa6e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815de20 con 0x7faa68161020 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa67400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faa6806d580 0x7faa6806d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35044/0 (socket says 172.21.15.97:35044) 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa67400640 1 -- 172.21.15.97:0/935186225 learned_addr learned my addr 172.21.15.97:0/935186225 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888025082 0 0) 0x7faa6815db40 con 0x7faa6806d580 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa48003520 con 0x7faa6806d580 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.897+0000 7faa67e00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 0x7faa681614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66a00640 1 --2- 172.21.15.97:0/935186225 >> v2:172.21.15.188:3300/0 conn(0x7faa6815d690 0x7faa680ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525768310 0 0) 0x7faa48003520 con 0x7faa6806d580 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 msgr2=0x7faa681614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 0x7faa681614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 >> v2:172.21.15.188:3300/0 conn(0x7faa6815d690 msgr2=0x7faa680ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 --2- 172.21.15.97:0/935186225 >> v2:172.21.15.188:3300/0 conn(0x7faa6815d690 0x7faa680ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6815e100 con 0x7faa6806d580 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7faa580017f0 con 0x7faa6806d580 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 >> v1:172.21.15.97:6789/0 conn(0x7faa6806d580 legacy=0x7faa6806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 0x7faa48003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa48004d00 0x7faa48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815e7c0 con 0x7faa48003ff0 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815de20 con 0x7faa48004d00 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4800bae0 con 0x7faa68161020 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.898+0000 7faa66a00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa67400640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 0x7faa48003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa67e00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa48004d00 0x7faa48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa66a00640 1 -- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 msgr2=0x7faa48003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa66a00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa68161020 0x7faa48003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa66a00640 1 -- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa48004d00 msgr2=0x7faa48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa66a00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa48004d00 0x7faa48005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa66a00640 1 -- 172.21.15.97:0/935186225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48003520 con 0x7faa48003ff0 2024-01-31T05:03:00.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.899+0000 7faa67e00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa48004d00 0x7faa48005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa66a00640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa480047c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faa5000ba00 tx=0x7faa480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd70e31026dde75 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50032640 con 0x7faa48003ff0 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa500327a0 con 0x7faa48003ff0 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa66000640 1 -- 172.21.15.97:0/935186225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50032b30 con 0x7faa48003ff0 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa6e400640 1 -- 172.21.15.97:0/935186225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 msgr2=0x7faa480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa6e400640 1 --2- 172.21.15.97:0/935186225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa480047c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faa5000ba00 tx=0x7faa480057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.900+0000 7faa67e00640 1 -- 172.21.15.97:0/935186225 reap_dead start 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.901+0000 7faa6e400640 1 -- 172.21.15.97:0/935186225 shutdown_connections 2024-01-31T05:03:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.901+0000 7faa6e400640 1 -- 172.21.15.97:0/935186225 >> 172.21.15.97:0/935186225 conn(0x7faa680b0a10 msgr2=0x7faa680b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.901+0000 7faa6e400640 1 -- 172.21.15.97:0/935186225 shutdown_connections 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.901+0000 7faa6e400640 1 -- 172.21.15.97:0/935186225 wait complete. 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.902+0000 7faa6e400640 1 Processor -- start 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 -- start start 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa681f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa6806c290 0x7faa6806c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa6809c020 0x7faa6809c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6806cc50 con 0x7faa48003ff0 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa681f6850 con 0x7faa6809c020 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.903+0000 7faa6e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa681f73d0 con 0x7faa6806c290 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa681f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa681f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49870/0 (socket says 172.21.15.97:49870) 2024-01-31T05:03:00.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 -- 172.21.15.97:0/65740173 learned_addr learned my addr 172.21.15.97:0/65740173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 -- 172.21.15.97:0/65740173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa6806c290 msgr2=0x7faa6806c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa6806c290 0x7faa6806c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 -- 172.21.15.97:0/65740173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa6809c020 msgr2=0x7faa6809c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa6809c020 0x7faa6809c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 -- 172.21.15.97:0/65740173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6809c9e0 con 0x7faa48003ff0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.904+0000 7faa67400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa681f3b30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e730 tx=0x7faa5800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.905+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800ee40 con 0x7faa48003ff0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.905+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa580072e0 con 0x7faa48003ff0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.905+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580057a0 con 0x7faa48003ff0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.905+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa681f6aa0 con 0x7faa48003ff0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.905+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa681f7060 con 0x7faa48003ff0 2024-01-31T05:03:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.906+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa681f7220 con 0x7faa48003ff0 2024-01-31T05:03:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.907+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7faa58005900 con 0x7faa48003ff0 2024-01-31T05:03:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.908+0000 7faa5f400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa4006aed0 0x7faa4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:00.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.908+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7faa58089aa0 con 0x7faa48003ff0 2024-01-31T05:03:00.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.908+0000 7faa66a00640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa4006aed0 0x7faa4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:00.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.909+0000 7faa66a00640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa4006aed0 0x7faa4006d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa6815da70 tx=0x7faa50032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:00.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:00.913+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7faa58054600 con 0x7faa48003ff0 2024-01-31T05:03:01.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.117+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa68067830 con 0x7faa48003ff0 2024-01-31T05:03:01.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.118+0000 7faa5f400640 1 -- 172.21.15.97:0/65740173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa5801a310 con 0x7faa48003ff0 2024-01-31T05:03:01.117 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:03:01.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.120+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa4006aed0 msgr2=0x7faa4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa4006aed0 0x7faa4006d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa6815da70 tx=0x7faa50032000 comp rx=0 tx=0).stop 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 msgr2=0x7faa681f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa681f3b30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e730 tx=0x7faa5800ebf0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 shutdown_connections 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa6809c020 0x7faa6809c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa6806c290 0x7faa6806c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa4006aed0 0x7faa4006d390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 --2- 172.21.15.97:0/65740173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa48003ff0 0x7faa681f3b30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 >> 172.21.15.97:0/65740173 conn(0x7faa680b0a10 msgr2=0x7faa680b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 shutdown_connections 2024-01-31T05:03:01.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.121+0000 7faa6e400640 1 -- 172.21.15.97:0/65740173 wait complete. 2024-01-31T05:03:01.138 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:01.138 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:01.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:01.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.437+0000 7f19b5c00640 1 Processor -- start 2024-01-31T05:03:01.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.437+0000 7f19b5c00640 1 -- start start 2024-01-31T05:03:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.438+0000 7f19b5c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19b00b1d10 0x7f19b00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.438+0000 7f19b5c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b015ea70 con 0x7f19b015a600 2024-01-31T05:03:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.438+0000 7f19b5c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b00b2630 con 0x7f19b006e0a0 2024-01-31T05:03:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.438+0000 7f19b5c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b006c8b0 con 0x7f19b00b1d10 2024-01-31T05:03:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19a7e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f19b015a600 0x7f19b015dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35058/0 (socket says 172.21.15.97:35058) 2024-01-31T05:03:01.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19a7e00640 1 -- 172.21.15.97:0/4005503394 learned_addr learned my addr 172.21.15.97:0/4005503394 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034595768 0 0) 0x7f19b015ea70 con 0x7f19b015a600 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19aea00640 1 --2- 172.21.15.97:0/4005503394 >> v2:172.21.15.188:3300/0 conn(0x7f19b00b1d10 0x7f19b00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1990003520 con 0x7f19b015a600 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248149913 0 0) 0x7f1990003520 con 0x7f19b015a600 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 >> v2:172.21.15.188:3300/0 conn(0x7f19b00b1d10 msgr2=0x7f19b00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 --2- 172.21.15.97:0/4005503394 >> v2:172.21.15.188:3300/0 conn(0x7f19b00b1d10 0x7f19b00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 >> v1:172.21.15.188:6789/0 conn(0x7f19b006e0a0 legacy=0x7f19b015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b015b120 con 0x7f19b015a600 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f1998001670 con 0x7f19b015a600 2024-01-31T05:03:01.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 >> v1:172.21.15.97:6789/0 conn(0x7f19b015a600 legacy=0x7f19b015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.439+0000 7f19ae000640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19900039c0 0x7f1990003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19ae000640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900047c0 0x7f1990004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19ae000640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19900051d0 0x7f199000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b006c8b0 con 0x7f19900051d0 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199000c100 con 0x7f19900047c0 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199000c2c0 con 0x7f19900039c0 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19aea00640 1 --2- 172.21.15.97:0/4005503394 >> v2:172.21.15.188:3300/0 conn(0x7f19b00b1d10 0x7f19b00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19af400640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19900051d0 0x7f199000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19af400640 1 -- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19900039c0 msgr2=0x7f1990003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19af400640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19900039c0 0x7f1990003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19af400640 1 -- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900047c0 msgr2=0x7f1990004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19a7e00640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900047c0 0x7f1990004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19af400640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900047c0 0x7f1990004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19af400640 1 -- 172.21.15.97:0/4005503394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990003520 con 0x7f19900051d0 2024-01-31T05:03:01.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.440+0000 7f19a7e00640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900047c0 0x7f1990004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.441+0000 7f19af400640 1 -- 172.21.15.97:0/4005503394 reap_dead start 2024-01-31T05:03:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.441+0000 7f19af400640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19900051d0 0x7f199000b980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f19b006e4b0 tx=0x7f19900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ed81421fcde449 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8006d50 con 0x7f19900051d0 2024-01-31T05:03:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a802f4c0 con 0x7f19900051d0 2024-01-31T05:03:01.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19ae000640 1 -- 172.21.15.97:0/4005503394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a802f870 con 0x7f19900051d0 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 -- 172.21.15.97:0/4005503394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19900051d0 msgr2=0x7f199000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19900051d0 0x7f199000b980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f19b006e4b0 tx=0x7f19900057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 -- 172.21.15.97:0/4005503394 shutdown_connections 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 --2- 172.21.15.97:0/4005503394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19900051d0 0x7f199000b980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 -- 172.21.15.97:0/4005503394 >> 172.21.15.97:0/4005503394 conn(0x7f19b00b0a10 msgr2=0x7f19b00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 -- 172.21.15.97:0/4005503394 shutdown_connections 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.442+0000 7f19b5c00640 1 -- 172.21.15.97:0/4005503394 wait complete. 2024-01-31T05:03:01.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.444+0000 7f19b5c00640 1 Processor -- start 2024-01-31T05:03:01.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.444+0000 7f19b5c00640 1 -- start start 2024-01-31T05:03:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.445+0000 7f19b5c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900051d0 0x7f19b014cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.445+0000 7f19b5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19b01541a0 0x7f19b0154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.445+0000 7f19b5c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 0x7f19b0071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.445+0000 7f19b5c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00718b0 con 0x7f19b0070ef0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.445+0000 7f19b5c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0155db0 con 0x7f19900051d0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.445+0000 7f19b5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0154d10 con 0x7f19b01541a0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 0x7f19b0071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 0x7f19b0071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49888/0 (socket says 172.21.15.97:49888) 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 -- 172.21.15.97:0/868248464 learned_addr learned my addr 172.21.15.97:0/868248464 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 -- 172.21.15.97:0/868248464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19b01541a0 msgr2=0x7f19b0154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19b01541a0 0x7f19b0154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 -- 172.21.15.97:0/868248464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900051d0 msgr2=0x7f19b014cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900051d0 0x7f19b014cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 -- 172.21.15.97:0/868248464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00adb80 con 0x7f19b0070ef0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19af400640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 0x7f19b0071370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f19a802e120 tx=0x7f19a8003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8037630 con 0x7f19b0070ef0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a8037790 con 0x7f19b0070ef0 2024-01-31T05:03:01.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.446+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8037b00 con 0x7f19b0070ef0 2024-01-31T05:03:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.447+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b0154ff0 con 0x7f19b0070ef0 2024-01-31T05:03:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.447+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b01555b0 con 0x7f19b0070ef0 2024-01-31T05:03:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.448+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b00677c0 con 0x7f19b0070ef0 2024-01-31T05:03:01.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.449+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f19a8037c60 con 0x7f19b0070ef0 2024-01-31T05:03:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.449+0000 7f19a6a00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f198806aed0 0x7f198806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.450+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f19a8033070 con 0x7f19b0070ef0 2024-01-31T05:03:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.450+0000 7f19aea00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f198806aed0 0x7f198806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.450+0000 7f19aea00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f198806aed0 0x7f198806d390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f19a00096b0 tx=0x7f19a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:01.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.454+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f19a80759b0 con 0x7f19b0070ef0 2024-01-31T05:03:01.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.649+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19b006e740 con 0x7f19b0070ef0 2024-01-31T05:03:01.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.653+0000 7f19a6a00640 1 -- 172.21.15.97:0/868248464 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f19a8047090 con 0x7f19b0070ef0 2024-01-31T05:03:01.653 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:01.653 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f198806aed0 msgr2=0x7f198806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f198806aed0 0x7f198806d390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f19a00096b0 tx=0x7f19a0008040 comp rx=0 tx=0).stop 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 msgr2=0x7f19b0071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 0x7f19b0071370 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f19a802e120 tx=0x7f19a8003a70 comp rx=0 tx=0).stop 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 shutdown_connections 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f198806aed0 0x7f198806d390 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19b0070ef0 0x7f19b0071370 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19b01541a0 0x7f19b0154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 --2- 172.21.15.97:0/868248464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19900051d0 0x7f19b014cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 >> 172.21.15.97:0/868248464 conn(0x7f19b00b0a10 msgr2=0x7f19b00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:01.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 shutdown_connections 2024-01-31T05:03:01.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.656+0000 7f19b5c00640 1 -- 172.21.15.97:0/868248464 wait complete. 2024-01-31T05:03:01.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:01.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:01.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.888+0000 7fa182800640 1 Processor -- start 2024-01-31T05:03:01.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.889+0000 7fa182800640 1 -- start start 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.890+0000 7fa182800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa17c0ce020 0x7fa17c06f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.890+0000 7fa182800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa17c06ff20 0x7fa17c06f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.890+0000 7fa182800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0d1a20 con 0x7fa17c15e870 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.890+0000 7fa182800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c067040 con 0x7fa17c0ce020 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.890+0000 7fa182800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c06e6a0 con 0x7fa17c06ff20 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa17c0ce020 0x7fa17c06f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa17c0ce020 0x7fa17c06f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49892/0 (socket says 172.21.15.97:49892) 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 -- 172.21.15.97:0/4149727341 learned_addr learned my addr 172.21.15.97:0/4149727341 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa181e00640 1 --2- 172.21.15.97:0/4149727341 >> v2:172.21.15.188:3300/0 conn(0x7fa17c06ff20 0x7fa17c06f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 -- 172.21.15.97:0/4149727341 >> v2:172.21.15.188:3300/0 conn(0x7fa17c06ff20 msgr2=0x7fa17c06f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:01.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 --2- 172.21.15.97:0/4149727341 >> v2:172.21.15.188:3300/0 conn(0x7fa17c06ff20 0x7fa17c06f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 -- 172.21.15.97:0/4149727341 >> v1:172.21.15.188:6789/0 conn(0x7fa17c15e870 legacy=0x7fa17c0cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:01.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa180a00640 1 -- 172.21.15.97:0/4149727341 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0d1c70 con 0x7fa17c0ce020 2024-01-31T05:03:01.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.891+0000 7fa181e00640 1 --2- 172.21.15.97:0/4149727341 >> v2:172.21.15.188:3300/0 conn(0x7fa17c06ff20 0x7fa17c06f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.892+0000 7fa180a00640 1 --2- 172.21.15.97:0/4149727341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c0ce020 0x7fa17c06f0f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa17c06e0f0 tx=0x7fa16402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92b81fc4a1a939e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa17be00640 1 -- 172.21.15.97:0/4149727341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640317e0 con 0x7fa17c0ce020 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa17be00640 1 -- 172.21.15.97:0/4149727341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164031940 con 0x7fa17c0ce020 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa17be00640 1 -- 172.21.15.97:0/4149727341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164031cd0 con 0x7fa17c0ce020 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa182800640 1 -- 172.21.15.97:0/4149727341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c0ce020 msgr2=0x7fa17c06f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa182800640 1 --2- 172.21.15.97:0/4149727341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c0ce020 0x7fa17c06f0f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa17c06e0f0 tx=0x7fa16402b880 comp rx=0 tx=0).stop 2024-01-31T05:03:01.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa182800640 1 -- 172.21.15.97:0/4149727341 shutdown_connections 2024-01-31T05:03:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa182800640 1 --2- 172.21.15.97:0/4149727341 >> v2:172.21.15.188:3300/0 conn(0x7fa17c06ff20 0x7fa17c06f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa182800640 1 --2- 172.21.15.97:0/4149727341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c0ce020 0x7fa17c06f0f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.893+0000 7fa182800640 1 -- 172.21.15.97:0/4149727341 >> 172.21.15.97:0/4149727341 conn(0x7fa17c0d0f00 msgr2=0x7fa17c0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.894+0000 7fa182800640 1 -- 172.21.15.97:0/4149727341 shutdown_connections 2024-01-31T05:03:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.894+0000 7fa182800640 1 -- 172.21.15.97:0/4149727341 wait complete. 2024-01-31T05:03:01.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.895+0000 7fa182800640 1 Processor -- start 2024-01-31T05:03:01.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.895+0000 7fa182800640 1 -- start start 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.896+0000 7fa182800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 0x7fa17c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.896+0000 7fa182800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa17c0ce020 0x7fa17c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.896+0000 7fa182800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa17c15e870 0x7fa17c0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.896+0000 7fa182800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0b4410 con 0x7fa17c06ff20 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.896+0000 7fa182800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0b2e80 con 0x7fa17c15e870 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.896+0000 7fa182800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0b3160 con 0x7fa17c0ce020 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 0x7fa17c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 0x7fa17c14e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49896/0 (socket says 172.21.15.97:49896) 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 -- 172.21.15.97:0/490928210 learned_addr learned my addr 172.21.15.97:0/490928210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 -- 172.21.15.97:0/490928210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa17c0ce020 msgr2=0x7fa17c0b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa17c0ce020 0x7fa17c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 -- 172.21.15.97:0/490928210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa17c15e870 msgr2=0x7fa17c0c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa17c15e870 0x7fa17c0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 -- 172.21.15.97:0/490928210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640094d0 con 0x7fa17c06ff20 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.897+0000 7fa181400640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 0x7fa17c14e300 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00b3f0 tx=0x7fa16c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.898+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011830 con 0x7fa17c06ff20 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.898+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c011990 con 0x7fa17c06ff20 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.898+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011d40 con 0x7fa17c06ff20 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.898+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c0b4640 con 0x7fa17c06ff20 2024-01-31T05:03:01.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.898+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c14e840 con 0x7fa17c06ff20 2024-01-31T05:03:01.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.899+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c0b4b60 con 0x7fa17c06ff20 2024-01-31T05:03:01.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.900+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa16c01a400 con 0x7fa17c06ff20 2024-01-31T05:03:01.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.901+0000 7fa179600640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa15406aed0 0x7fa15406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:01.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.901+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa16c089720 con 0x7fa17c06ff20 2024-01-31T05:03:01.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.901+0000 7fa180a00640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa15406aed0 0x7fa15406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.902+0000 7fa180a00640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa15406aed0 0x7fa15406d390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa164036000 tx=0x7fa164030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:01.905+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa17c0b4b60 con 0x7fa17c06ff20 2024-01-31T05:03:02.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.098+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa17c0b4b60 con 0x7fa17c06ff20 2024-01-31T05:03:02.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.099+0000 7fa179600640 1 -- 172.21.15.97:0/490928210 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa16c0543b0 con 0x7fa17c06ff20 2024-01-31T05:03:02.099 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa15406aed0 msgr2=0x7fa15406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa15406aed0 0x7fa15406d390 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa164036000 tx=0x7fa164030000 comp rx=0 tx=0).stop 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 msgr2=0x7fa17c14e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 0x7fa17c14e300 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00b3f0 tx=0x7fa16c00b8b0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 shutdown_connections 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa17c15e870 0x7fa17c0c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa17c0ce020 0x7fa17c0b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa15406aed0 0x7fa15406d390 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 --2- 172.21.15.97:0/490928210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa17c06ff20 0x7fa17c14e300 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 >> 172.21.15.97:0/490928210 conn(0x7fa17c0d0f00 msgr2=0x7fa17c0af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 shutdown_connections 2024-01-31T05:03:02.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.102+0000 7fa182800640 1 -- 172.21.15.97:0/490928210 wait complete. 2024-01-31T05:03:02.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:03:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.326+0000 7f72a9400640 1 Processor -- start 2024-01-31T05:03:02.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.326+0000 7f72a9400640 1 -- start start 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a9400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72a415b470 0x7f72a40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a9400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72a40cbf20 0x7f72a40ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a9400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a406cd20 con 0x7f72a40abcf0 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a9400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a40ce970 con 0x7f72a40cbf20 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a9400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a415b850 con 0x7f72a415b470 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72a40cbf20 0x7f72a40ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72a40cbf20 0x7f72a40ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49910/0 (socket says 172.21.15.97:49910) 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.328+0000 7f72a3200640 1 -- 172.21.15.97:0/441879084 learned_addr learned my addr 172.21.15.97:0/441879084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:02.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.329+0000 7f72a2800640 1 --2- 172.21.15.97:0/441879084 >> v2:172.21.15.188:3300/0 conn(0x7f72a415b470 0x7f72a40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.329+0000 7f72a3200640 1 -- 172.21.15.97:0/441879084 >> v2:172.21.15.188:3300/0 conn(0x7f72a415b470 msgr2=0x7f72a40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:02.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.329+0000 7f72a3200640 1 --2- 172.21.15.97:0/441879084 >> v2:172.21.15.188:3300/0 conn(0x7f72a415b470 0x7f72a40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.329+0000 7f72a3200640 1 -- 172.21.15.97:0/441879084 >> v1:172.21.15.188:6789/0 conn(0x7f72a40abcf0 legacy=0x7f72a40cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:02.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.329+0000 7f72a3200640 1 -- 172.21.15.97:0/441879084 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40ceb60 con 0x7f72a40cbf20 2024-01-31T05:03:02.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.330+0000 7f72a3200640 1 --2- 172.21.15.97:0/441879084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40cbf20 0x7f72a40ce430 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f72a4067370 tx=0x7f729c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4cc2deffeb40d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.330+0000 7f72a1e00640 1 -- 172.21.15.97:0/441879084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c031870 con 0x7f72a40cbf20 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.330+0000 7f72a1e00640 1 -- 172.21.15.97:0/441879084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c0319d0 con 0x7f72a40cbf20 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.330+0000 7f72a1e00640 1 -- 172.21.15.97:0/441879084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c031d60 con 0x7f72a40cbf20 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 -- 172.21.15.97:0/441879084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40cbf20 msgr2=0x7f72a40ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 --2- 172.21.15.97:0/441879084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40cbf20 0x7f72a40ce430 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f72a4067370 tx=0x7f729c02b910 comp rx=0 tx=0).stop 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 -- 172.21.15.97:0/441879084 shutdown_connections 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 --2- 172.21.15.97:0/441879084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40cbf20 0x7f72a40ce430 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 --2- 172.21.15.97:0/441879084 >> v2:172.21.15.188:3300/0 conn(0x7f72a415b470 0x7f72a40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 -- 172.21.15.97:0/441879084 >> 172.21.15.97:0/441879084 conn(0x7f72a40b0a10 msgr2=0x7f72a40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:02.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 -- 172.21.15.97:0/441879084 shutdown_connections 2024-01-31T05:03:02.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.331+0000 7f72a9400640 1 -- 172.21.15.97:0/441879084 wait complete. 2024-01-31T05:03:02.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.333+0000 7f72a9400640 1 Processor -- start 2024-01-31T05:03:02.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.333+0000 7f72a9400640 1 -- start start 2024-01-31T05:03:02.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 0x7f72a4151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a9400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72a40cbf20 0x7f72a40999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a9400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72a415b470 0x7f72a409c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a9400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4152fb0 con 0x7f72a40abcf0 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a9400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40c8cb0 con 0x7f72a415b470 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a9400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40b3140 con 0x7f72a40cbf20 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 0x7f72a4151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a2800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 0x7f72a4151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49916/0 (socket says 172.21.15.97:49916) 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.334+0000 7f72a2800640 1 -- 172.21.15.97:0/4040406372 learned_addr learned my addr 172.21.15.97:0/4040406372 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a3200640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72a415b470 0x7f72a409c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f7299c00640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72a40cbf20 0x7f72a40999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a2800640 1 -- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72a40cbf20 msgr2=0x7f72a40999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a2800640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72a40cbf20 0x7f72a40999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a2800640 1 -- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72a415b470 msgr2=0x7f72a409c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a2800640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72a415b470 0x7f72a409c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a2800640 1 -- 172.21.15.97:0/4040406372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729c0094d0 con 0x7f72a40abcf0 2024-01-31T05:03:02.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f7299c00640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72a40cbf20 0x7f72a40999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:02.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f72a2800640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 0x7f72a4151e90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7294007ef0 tx=0x7f729400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:02.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294014620 con 0x7f72a40abcf0 2024-01-31T05:03:02.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.335+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400ee90 con 0x7f72a40abcf0 2024-01-31T05:03:02.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.336+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a40b3420 con 0x7f72a40abcf0 2024-01-31T05:03:02.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.336+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72940149c0 con 0x7f72a40abcf0 2024-01-31T05:03:02.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.336+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a40c8f00 con 0x7f72a40abcf0 2024-01-31T05:03:02.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.337+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a40c90c0 con 0x7f72a40abcf0 2024-01-31T05:03:02.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.338+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7294014b20 con 0x7f72a40abcf0 2024-01-31T05:03:02.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.338+0000 7f729b400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f727c06aed0 0x7f727c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.338+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f729408a300 con 0x7f72a40abcf0 2024-01-31T05:03:02.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.339+0000 7f7299c00640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f727c06aed0 0x7f727c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.339+0000 7f7299c00640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f727c06aed0 0x7f727c06d390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72a40cf730 tx=0x7f728c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:02.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.343+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7294054ef0 con 0x7f72a40abcf0 2024-01-31T05:03:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.531+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72a4067830 con 0x7f72a40abcf0 2024-01-31T05:03:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.532+0000 7f729b400640 1 -- 172.21.15.97:0/4040406372 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7294058510 con 0x7f72a40abcf0 2024-01-31T05:03:02.531 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f727c06aed0 msgr2=0x7f727c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f727c06aed0 0x7f727c06d390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f72a40cf730 tx=0x7f728c009040 comp rx=0 tx=0).stop 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 msgr2=0x7f72a4151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 0x7f72a4151e90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7294007ef0 tx=0x7f729400e4b0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 shutdown_connections 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72a415b470 0x7f72a409c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72a40cbf20 0x7f72a40999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f727c06aed0 0x7f727c06d390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 --2- 172.21.15.97:0/4040406372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72a40abcf0 0x7f72a4151e90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 >> 172.21.15.97:0/4040406372 conn(0x7f72a40b0a10 msgr2=0x7f72a40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.534+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 shutdown_connections 2024-01-31T05:03:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.535+0000 7f72a9400640 1 -- 172.21.15.97:0/4040406372 wait complete. 2024-01-31T05:03:02.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:02.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.764+0000 7f27a1400640 1 Processor -- start 2024-01-31T05:03:02.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.765+0000 7f27a1400640 1 -- start start 2024-01-31T05:03:02.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.766+0000 7f27a1400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 0x7f279c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.766+0000 7f27a1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 0x7f279c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.766+0000 7f27a1400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279c0d2190 con 0x7f279c15d450 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.766+0000 7f27a1400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279c0b3060 con 0x7f279c0d0040 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.766+0000 7f27a1400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0b2a30 con 0x7f279c0d39f0 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.767+0000 7f27a0a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 0x7f279c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.767+0000 7f279b400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 0x7f279c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.767+0000 7f279b400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 0x7f279c0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:40928/0 (socket says 172.21.15.97:40928) 2024-01-31T05:03:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.767+0000 7f27a0a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 0x7f279c0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:57170/0 (socket says 172.21.15.97:57170) 2024-01-31T05:03:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.767+0000 7f279b400640 1 -- 172.21.15.97:0/1014631365 learned_addr learned my addr 172.21.15.97:0/1014631365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.768+0000 7f27a0a00640 1 -- 172.21.15.97:0/1014631365 >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 msgr2=0x7f279c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.768+0000 7f27a0a00640 1 --2- 172.21.15.97:0/1014631365 >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 0x7f279c0b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.768+0000 7f27a0a00640 1 -- 172.21.15.97:0/1014631365 >> v1:172.21.15.188:6789/0 conn(0x7f279c15d450 legacy=0x7f279c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.768+0000 7f27a0a00640 1 -- 172.21.15.97:0/1014631365 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0d1c30 con 0x7f279c0d39f0 2024-01-31T05:03:02.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.768+0000 7f279b400640 1 --2- 172.21.15.97:0/1014631365 >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 0x7f279c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:03:02.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.768+0000 7f279aa00640 1 -- 172.21.15.97:0/1014631365 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108445707 0 0) 0x7f279c0d2190 con 0x7f279c15d450 2024-01-31T05:03:02.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.769+0000 7f27a0a00640 1 --2- 172.21.15.97:0/1014631365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 0x7f279c0b39c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2794009d90 tx=0x7f279402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c35724ba0f05cf8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:02.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f279aa00640 1 -- 172.21.15.97:0/1014631365 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279402cc60 con 0x7f279c0d39f0 2024-01-31T05:03:02.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f279aa00640 1 -- 172.21.15.97:0/1014631365 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279402cdc0 con 0x7f279c0d39f0 2024-01-31T05:03:02.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f279aa00640 1 -- 172.21.15.97:0/1014631365 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2794034660 con 0x7f279c0d39f0 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 -- 172.21.15.97:0/1014631365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 msgr2=0x7f279c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 --2- 172.21.15.97:0/1014631365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 0x7f279c0b39c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2794009d90 tx=0x7f279402beb0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 -- 172.21.15.97:0/1014631365 shutdown_connections 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 --2- 172.21.15.97:0/1014631365 >> v2:172.21.15.188:3300/0 conn(0x7f279c0d0040 0x7f279c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 --2- 172.21.15.97:0/1014631365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c0d39f0 0x7f279c0b39c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 -- 172.21.15.97:0/1014631365 >> 172.21.15.97:0/1014631365 conn(0x7f279c09c400 msgr2=0x7f279c0d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 -- 172.21.15.97:0/1014631365 shutdown_connections 2024-01-31T05:03:02.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.770+0000 7f27a1400640 1 -- 172.21.15.97:0/1014631365 wait complete. 2024-01-31T05:03:02.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.772+0000 7f27a1400640 1 Processor -- start 2024-01-31T05:03:02.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.772+0000 7f27a1400640 1 -- start start 2024-01-31T05:03:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f27a1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 0x7f279c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f27a1400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279c0d39f0 0x7f279c0b7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f27a1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c15d450 0x7f279c0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f27a1400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c155b60 con 0x7f279c0d0040 2024-01-31T05:03:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f27a1400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0b5f60 con 0x7f279c0d39f0 2024-01-31T05:03:02.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f27a1400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c0b6240 con 0x7f279c15d450 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.773+0000 7f279be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 0x7f279c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 0x7f279c155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49918/0 (socket says 172.21.15.97:49918) 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 -- 172.21.15.97:0/2514218254 learned_addr learned my addr 172.21.15.97:0/2514218254 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279b400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279c0d39f0 0x7f279c0b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f27a0a00640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c15d450 0x7f279c0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 -- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c15d450 msgr2=0x7f279c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c15d450 0x7f279c0cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 -- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279c0d39f0 msgr2=0x7f279c0b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279c0d39f0 0x7f279c0b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 -- 172.21.15.97:0/2514218254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840094d0 con 0x7f279c0d0040 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.774+0000 7f279be00640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 0x7f279c155620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c410 tx=0x7f278c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.775+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c0035f0 con 0x7f279c0d0040 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.775+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c003770 con 0x7f279c0d0040 2024-01-31T05:03:02.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.775+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c003b20 con 0x7f279c0d0040 2024-01-31T05:03:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.775+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2794009a70 con 0x7f279c0d0040 2024-01-31T05:03:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.775+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279c0b6800 con 0x7f279c0d0040 2024-01-31T05:03:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.776+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279c0676b0 con 0x7f279c0d0040 2024-01-31T05:03:02.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.777+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f278c003c80 con 0x7f279c0d0040 2024-01-31T05:03:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.778+0000 7f2793e00640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f277406af00 0x7f277406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.778+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f278c088fb0 con 0x7f279c0d0040 2024-01-31T05:03:02.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.778+0000 7f279b400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f277406af00 0x7f277406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:02.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.779+0000 7f279b400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f277406af00 0x7f277406d3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f279c0d31b0 tx=0x7f2784008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.782+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f278c0bc190 con 0x7f279c0d0040 2024-01-31T05:03:02.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.974+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f279c15cc00 con 0x7f279c0d0040 2024-01-31T05:03:02.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.975+0000 7f2793e00640 1 -- 172.21.15.97:0/2514218254 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f278c017070 con 0x7f279c0d0040 2024-01-31T05:03:02.975 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.977+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f277406af00 msgr2=0x7f277406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.977+0000 7f27a1400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f277406af00 0x7f277406d3c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f279c0d31b0 tx=0x7f2784008040 comp rx=0 tx=0).stop 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.977+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 msgr2=0x7f279c155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.977+0000 7f27a1400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 0x7f279c155620 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c410 tx=0x7f278c00c8d0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 shutdown_connections 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f279c15d450 0x7f279c0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f279c0d39f0 0x7f279c0b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f277406af00 0x7f277406d3c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 --2- 172.21.15.97:0/2514218254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f279c0d0040 0x7f279c155620 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 >> 172.21.15.97:0/2514218254 conn(0x7f279c09c400 msgr2=0x7f279c0b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:02.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 shutdown_connections 2024-01-31T05:03:02.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:02.978+0000 7f27a1400640 1 -- 172.21.15.97:0/2514218254 wait complete. 2024-01-31T05:03:02.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:03:03.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.194+0000 7ff45b000640 1 Processor -- start 2024-01-31T05:03:03.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.194+0000 7ff45b000640 1 -- start start 2024-01-31T05:03:03.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.195+0000 7ff45b000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff45415a350 0x7ff4540abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.195+0000 7ff45b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff45415a730 0x7ff45406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff45b000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45415abc0 con 0x7ff45406dfc0 2024-01-31T05:03:03.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff45b000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540b2c90 con 0x7ff45415a350 2024-01-31T05:03:03.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff45b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415aea0 con 0x7ff45415a730 2024-01-31T05:03:03.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff459000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff45406dfc0 0x7ff4540cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:35060/0 (socket says 172.21.15.97:35060) 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff459000640 1 -- 172.21.15.97:0/2514974296 learned_addr learned my addr 172.21.15.97:0/2514974296 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff453400640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff45415a730 0x7ff45406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff453e00640 1 --2- 172.21.15.97:0/2514974296 >> v2:172.21.15.188:3300/0 conn(0x7ff45415a350 0x7ff4540abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714036581 0 0) 0x7ff45415abc0 con 0x7ff45406dfc0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.196+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff434003520 con 0x7ff45406dfc0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337999422 0 0) 0x7ff434003520 con 0x7ff45406dfc0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff45415a730 msgr2=0x7ff45406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff45415a730 0x7ff45406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 >> v2:172.21.15.188:3300/0 conn(0x7ff45415a350 msgr2=0x7ff4540abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 --2- 172.21.15.97:0/2514974296 >> v2:172.21.15.188:3300/0 conn(0x7ff45415a350 0x7ff4540abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45415b180 con 0x7ff45406dfc0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff44c0019f0 con 0x7ff45406dfc0 2024-01-31T05:03:03.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 >> v1:172.21.15.97:6789/0 conn(0x7ff45406dfc0 legacy=0x7ff4540cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434003970 0x7ff434003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff434004320 0x7ff434004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff434004e70 0x7ff43400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540b2c90 con 0x7ff434004e70 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415aea0 con 0x7ff434004320 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff434005670 con 0x7ff434003970 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.197+0000 7ff459000640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff434004e70 0x7ff43400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff453e00640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434003970 0x7ff434003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff453400640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff434004320 0x7ff434004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff459000640 1 -- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434003970 msgr2=0x7ff434003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff459000640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434003970 0x7ff434003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff459000640 1 -- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff434004320 msgr2=0x7ff434004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff459000640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff434004320 0x7ff434004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.198+0000 7ff459000640 1 -- 172.21.15.97:0/2514974296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434003520 con 0x7ff434004e70 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff459000640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff434004e70 0x7ff43400db90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00a790 tx=0x7ff4340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dadf8eefab89f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c034630 con 0x7ff434004e70 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c02f070 con 0x7ff434004e70 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff452a00640 1 -- 172.21.15.97:0/2514974296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c034a30 con 0x7ff434004e70 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff45b000640 1 -- 172.21.15.97:0/2514974296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff434004e70 msgr2=0x7ff43400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff45b000640 1 --2- 172.21.15.97:0/2514974296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff434004e70 0x7ff43400db90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00a790 tx=0x7ff4340057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.199+0000 7ff459000640 1 -- 172.21.15.97:0/2514974296 reap_dead start 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.200+0000 7ff45b000640 1 -- 172.21.15.97:0/2514974296 shutdown_connections 2024-01-31T05:03:03.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.200+0000 7ff45b000640 1 -- 172.21.15.97:0/2514974296 >> 172.21.15.97:0/2514974296 conn(0x7ff4540b0a10 msgr2=0x7ff4540b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:03.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.200+0000 7ff45b000640 1 -- 172.21.15.97:0/2514974296 shutdown_connections 2024-01-31T05:03:03.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.200+0000 7ff45b000640 1 -- 172.21.15.97:0/2514974296 wait complete. 2024-01-31T05:03:03.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.201+0000 7ff45b000640 1 Processor -- start 2024-01-31T05:03:03.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 -- start start 2024-01-31T05:03:03.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434004e70 0x7ff4540c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 0x7ff4540ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454159380 0x7ff454159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454159d40 con 0x7ff454157770 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540c9980 con 0x7ff454159380 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.202+0000 7ff45b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4540c9c60 con 0x7ff434004e70 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 0x7ff4540ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 0x7ff4540ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49942/0 (socket says 172.21.15.97:49942) 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 -- 172.21.15.97:0/824190966 learned_addr learned my addr 172.21.15.97:0/824190966 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff459000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454159380 0x7ff454159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 -- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434004e70 msgr2=0x7ff4540c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434004e70 0x7ff4540c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 -- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454159380 msgr2=0x7ff454159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454159380 0x7ff454159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 -- 172.21.15.97:0/824190966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4540c9f40 con 0x7ff454157770 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.203+0000 7ff453400640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 0x7ff4540ad670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011560 tx=0x7ff43c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.204+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c004220 con 0x7ff454157770 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.204+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c004380 con 0x7ff454157770 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.204+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c015870 con 0x7ff454157770 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.204+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4540ca220 con 0x7ff454157770 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.204+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4540adbb0 con 0x7ff454157770 2024-01-31T05:03:03.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.205+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454067830 con 0x7ff454157770 2024-01-31T05:03:03.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.206+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff43c0159d0 con 0x7ff454157770 2024-01-31T05:03:03.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.207+0000 7ff44be00640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff42c06aed0 0x7ff42c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.207+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7ff43c089bb0 con 0x7ff454157770 2024-01-31T05:03:03.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.207+0000 7ff453e00640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff42c06aed0 0x7ff42c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.208+0000 7ff453e00640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff42c06aed0 0x7ff42c06d390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff444009980 tx=0x7ff444008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:03.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.211+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff43c0547a0 con 0x7ff454157770 2024-01-31T05:03:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.403+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7ff45415d9f0 con 0x7ff454157770 2024-01-31T05:03:03.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.404+0000 7ff44be00640 1 -- 172.21.15.97:0/824190966 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7ff43c0220a0 con 0x7ff454157770 2024-01-31T05:03:03.403 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:03.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff42c06aed0 msgr2=0x7ff42c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff42c06aed0 0x7ff42c06d390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff444009980 tx=0x7ff444008040 comp rx=0 tx=0).stop 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 msgr2=0x7ff4540ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 0x7ff4540ad670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff43c011560 tx=0x7ff43c011a20 comp rx=0 tx=0).stop 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 shutdown_connections 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454159380 0x7ff454159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff42c06aed0 0x7ff42c06d390 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff454157770 0x7ff4540ad670 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 --2- 172.21.15.97:0/824190966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff434004e70 0x7ff4540c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 >> 172.21.15.97:0/824190966 conn(0x7ff4540b0a10 msgr2=0x7ff4540cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 shutdown_connections 2024-01-31T05:03:03.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.408+0000 7ff45b000640 1 -- 172.21.15.97:0/824190966 wait complete. 2024-01-31T05:03:03.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:03:03.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.617+0000 7f5203c00640 1 Processor -- start 2024-01-31T05:03:03.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.617+0000 7f5203c00640 1 -- start start 2024-01-31T05:03:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.618+0000 7f5203c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 0x7f51fc0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.618+0000 7f5203c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc0cde10 0x7f51fc0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.618+0000 7f5203c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc15abc0 con 0x7f51fc15a350 2024-01-31T05:03:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.618+0000 7f5203c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc0ce7e0 con 0x7f51fc15a730 2024-01-31T05:03:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.618+0000 7f5203c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0b2c90 con 0x7f51fc0cde10 2024-01-31T05:03:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.619+0000 7f51f3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 0x7f51fc0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.619+0000 7f5201c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc0cde10 0x7f51fc0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.619+0000 7f51f3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 0x7f51fc0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:40966/0 (socket says 172.21.15.97:40966) 2024-01-31T05:03:03.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.619+0000 7f51f3e00640 1 -- 172.21.15.97:0/3216131840 learned_addr learned my addr 172.21.15.97:0/3216131840 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:03.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.620+0000 7f5201c00640 1 -- 172.21.15.97:0/3216131840 >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 msgr2=0x7f51fc0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:03.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.620+0000 7f5201c00640 1 --2- 172.21.15.97:0/3216131840 >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 0x7f51fc0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.620+0000 7f5201c00640 1 -- 172.21.15.97:0/3216131840 >> v1:172.21.15.188:6789/0 conn(0x7f51fc15a350 legacy=0x7f51fc06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.620+0000 7f5201c00640 1 -- 172.21.15.97:0/3216131840 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc15aea0 con 0x7f51fc0cde10 2024-01-31T05:03:03.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.620+0000 7f51f3e00640 1 --2- 172.21.15.97:0/3216131840 >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 0x7f51fc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:03:03.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.622+0000 7f5201c00640 1 --2- 172.21.15.97:0/3216131840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc0cde10 0x7f51fc0ce2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009d90 tx=0x7f51f402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9913fdcefaaab366 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:03.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.622+0000 7f51fbe00640 1 -- 172.21.15.97:0/3216131840 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f402cc50 con 0x7f51fc0cde10 2024-01-31T05:03:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.622+0000 7f51fbe00640 1 -- 172.21.15.97:0/3216131840 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f402cdb0 con 0x7f51fc0cde10 2024-01-31T05:03:03.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.622+0000 7f51fbe00640 1 -- 172.21.15.97:0/3216131840 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4034660 con 0x7f51fc0cde10 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 -- 172.21.15.97:0/3216131840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc0cde10 msgr2=0x7f51fc0ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 --2- 172.21.15.97:0/3216131840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc0cde10 0x7f51fc0ce2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009d90 tx=0x7f51f402bea0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 -- 172.21.15.97:0/3216131840 shutdown_connections 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 --2- 172.21.15.97:0/3216131840 >> v2:172.21.15.188:3300/0 conn(0x7f51fc15a730 0x7f51fc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 --2- 172.21.15.97:0/3216131840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc0cde10 0x7f51fc0ce2a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 -- 172.21.15.97:0/3216131840 >> 172.21.15.97:0/3216131840 conn(0x7f51fc0b0a10 msgr2=0x7f51fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 -- 172.21.15.97:0/3216131840 shutdown_connections 2024-01-31T05:03:03.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.623+0000 7f5203c00640 1 -- 172.21.15.97:0/3216131840 wait complete. 2024-01-31T05:03:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.625+0000 7f5203c00640 1 Processor -- start 2024-01-31T05:03:03.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.625+0000 7f5203c00640 1 -- start start 2024-01-31T05:03:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.625+0000 7f5203c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 0x7f51fc158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5203c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51fc15a350 0x7f51fc0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5203c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc071ae0 0x7f51fc0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5203c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc1593b0 con 0x7f51fc0cde10 2024-01-31T05:03:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5203c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc071f60 con 0x7f51fc15a350 2024-01-31T05:03:03.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5203c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0b06a0 con 0x7f51fc071ae0 2024-01-31T05:03:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 0x7f51fc158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 0x7f51fc158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49954/0 (socket says 172.21.15.97:49954) 2024-01-31T05:03:03.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 -- 172.21.15.97:0/2400116176 learned_addr learned my addr 172.21.15.97:0/2400116176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f51f3e00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51fc15a350 0x7f51fc0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 -- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc071ae0 msgr2=0x7f51fc0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc071ae0 0x7f51fc0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 -- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51fc15a350 msgr2=0x7f51fc0715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51fc15a350 0x7f51fc0715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.626+0000 7f5201200640 1 -- 172.21.15.97:0/2400116176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40094d0 con 0x7f51fc0cde10 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.627+0000 7f5201200640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 0x7f51fc158e70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c430 tx=0x7f51ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.627+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0052d0 con 0x7f51fc0cde10 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.627+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec005470 con 0x7f51fc0cde10 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.627+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec006a60 con 0x7f51fc0cde10 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.627+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f4009a70 con 0x7f51fc0cde10 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.627+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc0b03c0 con 0x7f51fc0cde10 2024-01-31T05:03:03.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.628+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc067830 con 0x7f51fc0cde10 2024-01-31T05:03:03.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.629+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f51ec006bc0 con 0x7f51fc0cde10 2024-01-31T05:03:03.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.630+0000 7f51f9600640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f51d406af00 0x7f51d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:03.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.630+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f51ec088d90 con 0x7f51fc0cde10 2024-01-31T05:03:03.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.630+0000 7f51f3e00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f51d406af00 0x7f51d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:03.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.631+0000 7f51f3e00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f51d406af00 0x7f51d406d3c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51fc1f6040 tx=0x7f51e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:03.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.634+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f51ec053900 con 0x7f51fc0cde10 2024-01-31T05:03:03.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.824+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51fc0736f0 con 0x7f51fc0cde10 2024-01-31T05:03:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.824+0000 7f51f9600640 1 -- 172.21.15.97:0/2400116176 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f51ec056f20 con 0x7f51fc0cde10 2024-01-31T05:03:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f51d406af00 msgr2=0x7f51d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f51d406af00 0x7f51d406d3c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f51fc1f6040 tx=0x7f51e4008040 comp rx=0 tx=0).stop 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 msgr2=0x7f51fc158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 0x7f51fc158e70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c430 tx=0x7f51ec00c8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 shutdown_connections 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51fc071ae0 0x7f51fc0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51fc15a350 0x7f51fc0715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f51d406af00 0x7f51d406d3c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 --2- 172.21.15.97:0/2400116176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc0cde10 0x7f51fc158e70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 >> 172.21.15.97:0/2400116176 conn(0x7f51fc0b0a10 msgr2=0x7f51fc0acef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 shutdown_connections 2024-01-31T05:03:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:03.827+0000 7f5203c00640 1 -- 172.21.15.97:0/2400116176 wait complete. 2024-01-31T05:03:03.844 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:03:03.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:03.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:03.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[54037]: fuse finished with error 0 and tester_r 0 2024-01-31T05:03:03.933 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:03:10.035 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:03:10.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:10.035 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:03:10.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-01-31T05:03:10.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.303+0000 7f2eb3200640 1 Processor -- start 2024-01-31T05:03:10.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.303+0000 7f2eb3200640 1 -- start start 2024-01-31T05:03:10.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2eb415b2b0 0x7f2eb40ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb40ce590 0x7f2eb406c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb3200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb40ac0b0 con 0x7f2eb415aed0 2024-01-31T05:03:10.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb3200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb415b760 con 0x7f2eb40ce590 2024-01-31T05:03:10.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb415ba40 con 0x7f2eb415b2b0 2024-01-31T05:03:10.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb2800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb40ce590 0x7f2eb406c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb2800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2eb40ce590 0x7f2eb406c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40242/0 (socket says 172.21.15.97:40242) 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.305+0000 7f2eb2800640 1 -- 172.21.15.97:0/3112688717 learned_addr learned my addr 172.21.15.97:0/3112688717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.306+0000 7f2eb1400640 1 --2- 172.21.15.97:0/3112688717 >> v2:172.21.15.188:3300/0 conn(0x7f2eb415b2b0 0x7f2eb40ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.306+0000 7f2eb2800640 1 -- 172.21.15.97:0/3112688717 >> v2:172.21.15.188:3300/0 conn(0x7f2eb415b2b0 msgr2=0x7f2eb40ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.306+0000 7f2eb2800640 1 --2- 172.21.15.97:0/3112688717 >> v2:172.21.15.188:3300/0 conn(0x7f2eb415b2b0 0x7f2eb40ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.306+0000 7f2eb2800640 1 -- 172.21.15.97:0/3112688717 >> v1:172.21.15.97:6789/0 conn(0x7f2eb415aed0 legacy=0x7f2eb40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.306+0000 7f2eb2800640 1 -- 172.21.15.97:0/3112688717 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb415bd20 con 0x7f2eb40ce590 2024-01-31T05:03:10.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.307+0000 7f2eb2800640 1 --2- 172.21.15.97:0/3112688717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb40ce590 0x7f2eb406c550 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2eb406b6b0 tx=0x7f2eac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81415df9cffe26bf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.307+0000 7f2eb0a00640 1 -- 172.21.15.97:0/3112688717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac02ca80 con 0x7f2eb40ce590 2024-01-31T05:03:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.307+0000 7f2eb0a00640 1 -- 172.21.15.97:0/3112688717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac02cbe0 con 0x7f2eb40ce590 2024-01-31T05:03:10.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.307+0000 7f2eb0a00640 1 -- 172.21.15.97:0/3112688717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0343f0 con 0x7f2eb40ce590 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 -- 172.21.15.97:0/3112688717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb40ce590 msgr2=0x7f2eb406c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 --2- 172.21.15.97:0/3112688717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb40ce590 0x7f2eb406c550 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2eb406b6b0 tx=0x7f2eac02bee0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 -- 172.21.15.97:0/3112688717 shutdown_connections 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 --2- 172.21.15.97:0/3112688717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb40ce590 0x7f2eb406c550 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 --2- 172.21.15.97:0/3112688717 >> v2:172.21.15.188:3300/0 conn(0x7f2eb415b2b0 0x7f2eb40ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 -- 172.21.15.97:0/3112688717 >> 172.21.15.97:0/3112688717 conn(0x7f2eb40b0910 msgr2=0x7f2eb40b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 -- 172.21.15.97:0/3112688717 shutdown_connections 2024-01-31T05:03:10.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.308+0000 7f2eb3200640 1 -- 172.21.15.97:0/3112688717 wait complete. 2024-01-31T05:03:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.310+0000 7f2eb3200640 1 Processor -- start 2024-01-31T05:03:10.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.310+0000 7f2eb3200640 1 -- start start 2024-01-31T05:03:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40ce590 0x7f2eb409bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 0x7f2eb414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb415b2b0 0x7f2eb40b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb409c7f0 con 0x7f2eb415aed0 2024-01-31T05:03:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb40b2850 con 0x7f2eb415b2b0 2024-01-31T05:03:10.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb40b2b30 con 0x7f2eb40ce590 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 0x7f2eb414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb1400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 0x7f2eb414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40256/0 (socket says 172.21.15.97:40256) 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.311+0000 7f2eb1400640 1 -- 172.21.15.97:0/2889603278 learned_addr learned my addr 172.21.15.97:0/2889603278 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb2800640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb415b2b0 0x7f2eb40b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1e00640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40ce590 0x7f2eb409bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1400640 1 -- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40ce590 msgr2=0x7f2eb409bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1400640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40ce590 0x7f2eb409bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1400640 1 -- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb415b2b0 msgr2=0x7f2eb40b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1400640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb415b2b0 0x7f2eb40b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1400640 1 -- 172.21.15.97:0/2889603278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac009b30 con 0x7f2eb415aed0 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1e00640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40ce590 0x7f2eb409bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb1400640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 0x7f2eb414f800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00d7d0 tx=0x7f2e9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:10.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00b6c0 con 0x7f2eb415aed0 2024-01-31T05:03:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c00b820 con 0x7f2eb415aed0 2024-01-31T05:03:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00bbb0 con 0x7f2eb415aed0 2024-01-31T05:03:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.312+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb40b2e10 con 0x7f2eb415aed0 2024-01-31T05:03:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.313+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb409c470 con 0x7f2eb415aed0 2024-01-31T05:03:10.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.314+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb40676b0 con 0x7f2eb415aed0 2024-01-31T05:03:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.315+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2e9c00bd10 con 0x7f2eb415aed0 2024-01-31T05:03:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.316+0000 7f2eaa000640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2e8c06aed0 0x7f2e8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.316+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f2e9c089f80 con 0x7f2eb415aed0 2024-01-31T05:03:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.316+0000 7f2eb1e00640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2e8c06aed0 0x7f2e8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.317+0000 7f2eb1e00640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2e8c06aed0 0x7f2e8c06d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4001680 tx=0x7f2ea4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:10.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.321+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2e9c054ae0 con 0x7f2eb415aed0 2024-01-31T05:03:10.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.596+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f2eb4073d40 con 0x7f2eb415aed0 2024-01-31T05:03:10.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.600+0000 7f2eaa000640 1 -- 172.21.15.97:0/2889603278 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v13) v1 ==== 211+0+168 (secure 0 0 0) 0x7f2e9c058100 con 0x7f2eb415aed0 2024-01-31T05:03:10.600 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:03:10.600 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:03:10.600 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow rwp" 2024-01-31T05:03:10.600 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:03:10.600 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-01-31T05:03:10.601 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:03:10.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.603+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2e8c06aed0 msgr2=0x7f2e8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.603+0000 7f2eb3200640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2e8c06aed0 0x7f2e8c06d390 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4001680 tx=0x7f2ea4009040 comp rx=0 tx=0).stop 2024-01-31T05:03:10.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.603+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 msgr2=0x7f2eb414f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.603+0000 7f2eb3200640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 0x7f2eb414f800 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00d7d0 tx=0x7f2e9c00dc90 comp rx=0 tx=0).stop 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 shutdown_connections 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb415b2b0 0x7f2eb40b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2e8c06aed0 0x7f2e8c06d390 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb415aed0 0x7f2eb414f800 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 --2- 172.21.15.97:0/2889603278 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40ce590 0x7f2eb409bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 >> 172.21.15.97:0/2889603278 conn(0x7f2eb40b0910 msgr2=0x7f2eb415f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 shutdown_connections 2024-01-31T05:03:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.604+0000 7f2eb3200640 1 -- 172.21.15.97:0/2889603278 wait complete. 2024-01-31T05:03:10.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:03:10.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.829+0000 7f6352a00640 1 Processor -- start 2024-01-31T05:03:10.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.829+0000 7f6352a00640 1 -- start start 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.830+0000 7f6352a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f634c0cb810 0x7f634c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.830+0000 7f6352a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c15b400 0x7f634c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.830+0000 7f6352a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634c06d1a0 con 0x7f634c06db70 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.830+0000 7f6352a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634c0cc960 con 0x7f634c0cb810 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.830+0000 7f6352a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c06c600 con 0x7f634c15b400 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f634c0cb810 0x7f634c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f634c0cb810 0x7f634c0cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40270/0 (socket says 172.21.15.97:40270) 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 -- 172.21.15.97:0/3845217658 learned_addr learned my addr 172.21.15.97:0/3845217658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634b400640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c15b400 0x7f634c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 -- 172.21.15.97:0/3845217658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c15b400 msgr2=0x7f634c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:10.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c15b400 0x7f634c06d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 -- 172.21.15.97:0/3845217658 >> v1:172.21.15.188:6789/0 conn(0x7f634c06db70 legacy=0x7f634c0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:10.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634be00640 1 -- 172.21.15.97:0/3845217658 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c0cc130 con 0x7f634c0cb810 2024-01-31T05:03:10.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.831+0000 7f634b400640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c15b400 0x7f634c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.832+0000 7f634be00640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c0cb810 0x7f634c0cbbf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f633c009870 tx=0x7f633c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd044f58f7dcdd83 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f634aa00640 1 -- 172.21.15.97:0/3845217658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c031640 con 0x7f634c0cb810 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f634aa00640 1 -- 172.21.15.97:0/3845217658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c0317a0 con 0x7f634c0cb810 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f634aa00640 1 -- 172.21.15.97:0/3845217658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c031b30 con 0x7f634c0cb810 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 -- 172.21.15.97:0/3845217658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c0cb810 msgr2=0x7f634c0cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c0cb810 0x7f634c0cbbf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f633c009870 tx=0x7f633c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 -- 172.21.15.97:0/3845217658 shutdown_connections 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c15b400 0x7f634c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 --2- 172.21.15.97:0/3845217658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c0cb810 0x7f634c0cbbf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 -- 172.21.15.97:0/3845217658 >> 172.21.15.97:0/3845217658 conn(0x7f634c0b0a10 msgr2=0x7f634c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.833+0000 7f6352a00640 1 -- 172.21.15.97:0/3845217658 shutdown_connections 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.834+0000 7f6352a00640 1 -- 172.21.15.97:0/3845217658 wait complete. 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.835+0000 7f6352a00640 1 Processor -- start 2024-01-31T05:03:10.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.835+0000 7f6352a00640 1 -- start start 2024-01-31T05:03:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f6352a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 0x7f634c15e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f6352a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c0cb810 0x7f634c15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f6352a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f634c15b400 0x7f634c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f6352a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c15c760 con 0x7f634c06db70 2024-01-31T05:03:10.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f6352a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c15d180 con 0x7f634c15b400 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f6352a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c15d460 con 0x7f634c0cb810 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.836+0000 7f634be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 0x7f634c15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 0x7f634c15e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40280/0 (socket says 172.21.15.97:40280) 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 -- 172.21.15.97:0/87427710 learned_addr learned my addr 172.21.15.97:0/87427710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634b400640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c0cb810 0x7f634c15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 -- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c0cb810 msgr2=0x7f634c15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c0cb810 0x7f634c15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 -- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f634c15b400 msgr2=0x7f634c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f634c15b400 0x7f634c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 -- 172.21.15.97:0/87427710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0094d0 con 0x7f634c06db70 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634b400640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c0cb810 0x7f634c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.837+0000 7f634be00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 0x7f634c15e220 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f633c009870 tx=0x7f633c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.838+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c03ac60 con 0x7f634c06db70 2024-01-31T05:03:10.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.838+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c03ade0 con 0x7f634c06db70 2024-01-31T05:03:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.838+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c15d740 con 0x7f634c06db70 2024-01-31T05:03:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.838+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c044630 con 0x7f634c06db70 2024-01-31T05:03:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.838+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c15dd00 con 0x7f634c06db70 2024-01-31T05:03:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.839+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c15def0 con 0x7f634c06db70 2024-01-31T05:03:10.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.840+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f633c02f050 con 0x7f634c06db70 2024-01-31T05:03:10.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.841+0000 7f6343e00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f632406aed0 0x7f632406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:10.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.841+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f633c0abb60 con 0x7f634c06db70 2024-01-31T05:03:10.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.841+0000 7f634b400640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f632406aed0 0x7f632406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:10.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.842+0000 7f634b400640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f632406aed0 0x7f632406d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6334006fd0 tx=0x7f6334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:10.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:10.845+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f633c0766c0 con 0x7f634c06db70 2024-01-31T05:03:11.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.070+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f634c0ad960 con 0x7f634c06db70 2024-01-31T05:03:11.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.071+0000 7f6343e00640 1 -- 172.21.15.97:0/87427710 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+82 (secure 0 0 0) 0x7f633c079ce0 con 0x7f634c06db70 2024-01-31T05:03:11.071 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-01-31T05:03:11.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.073+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f632406aed0 msgr2=0x7f632406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:11.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.073+0000 7f6352a00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f632406aed0 0x7f632406d390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6334006fd0 tx=0x7f6334008040 comp rx=0 tx=0).stop 2024-01-31T05:03:11.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.073+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 msgr2=0x7f634c15e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.073+0000 7f6352a00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 0x7f634c15e220 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f633c009870 tx=0x7f633c0046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 shutdown_connections 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f634c15b400 0x7f634c151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f634c0cb810 0x7f634c15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f632406aed0 0x7f632406d390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 --2- 172.21.15.97:0/87427710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f634c06db70 0x7f634c15e220 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 >> 172.21.15.97:0/87427710 conn(0x7f634c0b0a10 msgr2=0x7f634c0ac400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 shutdown_connections 2024-01-31T05:03:11.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:11.074+0000 7f6352a00640 1 -- 172.21.15.97:0/87427710 wait complete. 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:03:11.090 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:03:11.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:03:11.091 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:03:11.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.091 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:03:11.140 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:03:11.140 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:03:11.140 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:03:11.140 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42333sec preferred_lft 42333sec 2024-01-31T05:03:11.141 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether ca:84:a9:c5:b2:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c884:a9ff:fec5:b219/64 scope link 2024-01-31T05:03:11.142 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:11.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.143 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:03:11.220 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:03:11.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.224 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:03:11.275 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:03:11.275 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:03:11.275 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:03:11.275 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:03:11.275 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:03:11.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.276 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:03:11.276 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:03:11.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:03:11.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:03:11.276 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:03:11.276 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:03:11.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.453 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:03:11.501 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:03:11.501 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:03:11.502 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:03:11.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:03:11.594 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:03:11.594 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:03:11.594 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:11.647 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:03:11.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:03:11.648 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:11.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:03:11.789 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:03:11.842 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:03:11.842 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:03:11.842 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:03:11.842 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:03:11.842 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:03:11.843 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:11.844 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:11.845 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:11.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:11.845 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:03:11.895 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:03:11.895 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:03:11.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:03:11.973 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:03:12.012 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:03:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:03:12.014 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:03:12.015 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:03:12.015 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:12.015 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:12.015 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:12.015 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:12.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:12.016 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:03:12.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:03:12.076+0000 7f32cce49480 -1 init, newargv = 0x5629b87d9c20 newargc=15 2024-01-31T05:03:12.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[55101]: starting ceph client 2024-01-31T05:03:12.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[55101]: starting fuse 2024-01-31T05:03:12.102 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:03:12.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:03:12.103 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:03:12.104 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:03:12.105 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:12.105 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:03:12.105 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:03:12.303 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.55101.asok 2024-01-31T05:03:12.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:03:12.320 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55101.asok 2024-01-31T05:03:12.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:12.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55101.asok status 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:03:12.559 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "55101", 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5955, 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 5955 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:03:12.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 760392914 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 760392914 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.5955 192.168.0.1:0/760392914", 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/760392914", 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 18, 2024-01-31T05:03:12.561 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 43, 2024-01-31T05:03:12.562 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:03:12.562 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:03:12.562 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:03:12.562 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:03:12.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:12.573 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:03:12.573 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:12.623 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:03:12.623 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:12.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:12.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:12.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:12.706 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-01-31T05:03:12.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:12.763 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-01-31T05:03:12.824 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:12.824 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:12.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:13.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:03:13.034 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:03:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.081+0000 7f9197a00640 1 Processor -- start 2024-01-31T05:03:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.082+0000 7f9197a00640 1 -- start start 2024-01-31T05:03:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9197a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91900cfae0 0x7f91900cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9197a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91900d0de0 0x7f91900d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9197a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919015ec50 con 0x7f919015d590 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9197a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91900d0460 con 0x7f91900cfae0 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9197a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91900cded0 con 0x7f91900d0de0 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9195000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f919015d590 0x7f919015e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37892/0 (socket says 172.21.15.97:37892) 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9195000640 1 -- 172.21.15.97:0/866964221 learned_addr learned my addr 172.21.15.97:0/866964221 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.083+0000 7f9187e00640 1 --2- 172.21.15.97:0/866964221 >> v2:172.21.15.97:3300/0 conn(0x7f91900cfae0 0x7f91900cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f9195a00640 1 --2- 172.21.15.97:0/866964221 >> v2:172.21.15.188:3300/0 conn(0x7f91900d0de0 0x7f91900d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f918fe00640 1 -- 172.21.15.97:0/866964221 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770458953 0 0) 0x7f919015ec50 con 0x7f919015d590 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f918fe00640 1 -- 172.21.15.97:0/866964221 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9170003520 con 0x7f919015d590 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f9187e00640 1 -- 172.21.15.97:0/866964221 >> v2:172.21.15.188:3300/0 conn(0x7f91900d0de0 msgr2=0x7f91900d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f9187e00640 1 --2- 172.21.15.97:0/866964221 >> v2:172.21.15.188:3300/0 conn(0x7f91900d0de0 0x7f91900d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f9187e00640 1 -- 172.21.15.97:0/866964221 >> v1:172.21.15.97:6789/0 conn(0x7f919015d590 legacy=0x7f919015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.084+0000 7f9187e00640 1 -- 172.21.15.97:0/866964221 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919015d970 con 0x7f91900cfae0 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.085+0000 7f9187e00640 1 --2- 172.21.15.97:0/866964221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91900cfae0 0x7f91900cff20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f91900cdbd0 tx=0x7f917802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569357146c2808d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.085+0000 7f918fe00640 1 -- 172.21.15.97:0/866964221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178030610 con 0x7f91900cfae0 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f918fe00640 1 -- 172.21.15.97:0/866964221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9178030770 con 0x7f91900cfae0 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f918fe00640 1 -- 172.21.15.97:0/866964221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178030ae0 con 0x7f91900cfae0 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 -- 172.21.15.97:0/866964221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91900cfae0 msgr2=0x7f91900cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 --2- 172.21.15.97:0/866964221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91900cfae0 0x7f91900cff20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f91900cdbd0 tx=0x7f917802b960 comp rx=0 tx=0).stop 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 -- 172.21.15.97:0/866964221 shutdown_connections 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 --2- 172.21.15.97:0/866964221 >> v2:172.21.15.188:3300/0 conn(0x7f91900d0de0 0x7f91900d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 --2- 172.21.15.97:0/866964221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91900cfae0 0x7f91900cff20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 -- 172.21.15.97:0/866964221 >> 172.21.15.97:0/866964221 conn(0x7f919009c560 msgr2=0x7f91900d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 -- 172.21.15.97:0/866964221 shutdown_connections 2024-01-31T05:03:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.086+0000 7f9197a00640 1 -- 172.21.15.97:0/866964221 wait complete. 2024-01-31T05:03:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.088+0000 7f9197a00640 1 Processor -- start 2024-01-31T05:03:13.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.088+0000 7f9197a00640 1 -- start start 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9197a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91900cfae0 0x7f91901f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9197a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91900d0de0 0x7f91900a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9197a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 0x7f91900b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9197a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91900c7670 con 0x7f919015d590 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9197a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91900a0c90 con 0x7f91900d0de0 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9197a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91900c6d40 con 0x7f91900cfae0 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9195a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 0x7f91900b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9195a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 0x7f91900b7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40292/0 (socket says 172.21.15.97:40292) 2024-01-31T05:03:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.089+0000 7f9195a00640 1 -- 172.21.15.97:0/2065442225 learned_addr learned my addr 172.21.15.97:0/2065442225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9187e00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91900d0de0 0x7f91900a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195000640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91900cfae0 0x7f91901f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195a00640 1 -- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91900cfae0 msgr2=0x7f91901f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91900cfae0 0x7f91901f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195a00640 1 -- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91900d0de0 msgr2=0x7f91900a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91900d0de0 0x7f91900a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195a00640 1 -- 172.21.15.97:0/2065442225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178009400 con 0x7f919015d590 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9195a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 0x7f91900b7470 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f918800efc0 tx=0x7f918800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188014480 con 0x7f919015d590 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918800f040 con 0x7f919015d590 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.090+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91900c7020 con 0x7f919015d590 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.091+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188014820 con 0x7f919015d590 2024-01-31T05:03:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.091+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91901f2a00 con 0x7f919015d590 2024-01-31T05:03:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.092+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91900c77e0 con 0x7f919015d590 2024-01-31T05:03:13.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.093+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9188014980 con 0x7f919015d590 2024-01-31T05:03:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.094+0000 7f918d600640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f916806aed0 0x7f916806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.094+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f918808a4d0 con 0x7f919015d590 2024-01-31T05:03:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.095+0000 7f9195000640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f916806aed0 0x7f916806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.095+0000 7f9195000640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f916806aed0 0x7f916806d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9180005880 tx=0x7f9180009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.099+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f91880550c0 con 0x7f919015d590 2024-01-31T05:03:13.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.300+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91900b5e30 con 0x7f919015d590 2024-01-31T05:03:13.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.302+0000 7f918d600640 1 -- 172.21.15.97:0/2065442225 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f91880586e0 con 0x7f919015d590 2024-01-31T05:03:13.302 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:13.302 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f916806aed0 msgr2=0x7f916806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f916806aed0 0x7f916806d390 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9180005880 tx=0x7f9180009040 comp rx=0 tx=0).stop 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 msgr2=0x7f91900b7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 0x7f91900b7470 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f918800efc0 tx=0x7f918800c490 comp rx=0 tx=0).stop 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 shutdown_connections 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f916806aed0 0x7f916806d390 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f919015d590 0x7f91900b7470 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91900d0de0 0x7f91900a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 --2- 172.21.15.97:0/2065442225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91900cfae0 0x7f91901f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 >> 172.21.15.97:0/2065442225 conn(0x7f919009c560 msgr2=0x7f91900b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:13.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 shutdown_connections 2024-01-31T05:03:13.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.305+0000 7f9197a00640 1 -- 172.21.15.97:0/2065442225 wait complete. 2024-01-31T05:03:13.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:13.324 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.534+0000 7f8247800640 1 Processor -- start 2024-01-31T05:03:13.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.534+0000 7f8247800640 1 -- start start 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.536+0000 7f8247800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82401599a0 0x7f82400ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.536+0000 7f8247800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.536+0000 7f8247800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8240159d80 con 0x7f82400b2250 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.536+0000 7f8247800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82400cc5e0 con 0x7f82401599a0 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.536+0000 7f8247800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824015e210 con 0x7f82400adea0 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.536+0000 7f8237c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.537+0000 7f8237c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:38270/0 (socket says 172.21.15.97:38270) 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.537+0000 7f8237c00640 1 -- 172.21.15.97:0/3974758776 learned_addr learned my addr 172.21.15.97:0/3974758776 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:13.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.537+0000 7f8244e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82401599a0 0x7f82400ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.538+0000 7f8237c00640 1 -- 172.21.15.97:0/3974758776 >> v2:172.21.15.188:3300/0 conn(0x7f82401599a0 msgr2=0x7f82400ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.538+0000 7f8237c00640 1 --2- 172.21.15.97:0/3974758776 >> v2:172.21.15.188:3300/0 conn(0x7f82401599a0 0x7f82400ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.538+0000 7f8237c00640 1 -- 172.21.15.97:0/3974758776 >> v1:172.21.15.188:6789/0 conn(0x7f82400b2250 legacy=0x7f82400ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.538+0000 7f8237c00640 1 -- 172.21.15.97:0/3974758776 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400cc880 con 0x7f82400adea0 2024-01-31T05:03:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.539+0000 7f8237c00640 1 --2- 172.21.15.97:0/3974758776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400b1d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8228009870 tx=0x7f822802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aeae11b591777e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f823fe00640 1 -- 172.21.15.97:0/3974758776 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228031640 con 0x7f82400adea0 2024-01-31T05:03:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f823fe00640 1 -- 172.21.15.97:0/3974758776 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82280317a0 con 0x7f82400adea0 2024-01-31T05:03:13.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f823fe00640 1 -- 172.21.15.97:0/3974758776 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228031b30 con 0x7f82400adea0 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f8247800640 1 -- 172.21.15.97:0/3974758776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 msgr2=0x7f82400b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f8247800640 1 --2- 172.21.15.97:0/3974758776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400b1d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8228009870 tx=0x7f822802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f8247800640 1 -- 172.21.15.97:0/3974758776 shutdown_connections 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.540+0000 7f8247800640 1 --2- 172.21.15.97:0/3974758776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400b1d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.541+0000 7f8247800640 1 --2- 172.21.15.97:0/3974758776 >> v2:172.21.15.188:3300/0 conn(0x7f82401599a0 0x7f82400ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.541+0000 7f8247800640 1 -- 172.21.15.97:0/3974758776 >> 172.21.15.97:0/3974758776 conn(0x7f82400b0a10 msgr2=0x7f82400b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:13.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.541+0000 7f8247800640 1 -- 172.21.15.97:0/3974758776 shutdown_connections 2024-01-31T05:03:13.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.541+0000 7f8247800640 1 -- 172.21.15.97:0/3974758776 wait complete. 2024-01-31T05:03:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.542+0000 7f8247800640 1 Processor -- start 2024-01-31T05:03:13.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.542+0000 7f8247800640 1 -- start start 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.543+0000 7f8247800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.543+0000 7f8247800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 0x7f82400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.543+0000 7f8247800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82401599a0 0x7f8240152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.543+0000 7f8247800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240071e00 con 0x7f82400b2250 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.543+0000 7f8247800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824015aac0 con 0x7f82401599a0 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.543+0000 7f8247800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8240070b30 con 0x7f82400adea0 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 0x7f82400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 0x7f82400705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40306/0 (socket says 172.21.15.97:40306) 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 -- 172.21.15.97:0/2786958426 learned_addr learned my addr 172.21.15.97:0/2786958426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8244e00640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8245800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82401599a0 0x7f8240152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 -- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 msgr2=0x7f82400711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 -- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82401599a0 msgr2=0x7f8240152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82401599a0 0x7f8240152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.544+0000 7f8237c00640 1 -- 172.21.15.97:0/2786958426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280094d0 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.545+0000 7f8237c00640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 0x7f82400705f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8228000c00 tx=0x7f82280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.545+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822803a930 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.545+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822803aa90 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.545+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822803ae40 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.545+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8240071840 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.545+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824015ad60 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.546+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82400b16e0 con 0x7f82400b2250 2024-01-31T05:03:13.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.547+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f822802f050 con 0x7f82400b2250 2024-01-31T05:03:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.548+0000 7f823d600640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f821806aed0 0x7f821806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.548+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f82280ab830 con 0x7f82400b2250 2024-01-31T05:03:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.548+0000 7f8244e00640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f821806aed0 0x7f821806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.549+0000 7f8244e00640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f821806aed0 0x7f821806d390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8230009a20 tx=0x7f8230008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.552+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8228076390 con 0x7f82400b2250 2024-01-31T05:03:13.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.746+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8240067810 con 0x7f82400b2250 2024-01-31T05:03:13.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.747+0000 7f823d600640 1 -- 172.21.15.97:0/2786958426 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82280799b0 con 0x7f82400b2250 2024-01-31T05:03:13.747 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f821806aed0 msgr2=0x7f821806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f821806aed0 0x7f821806d390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8230009a20 tx=0x7f8230008040 comp rx=0 tx=0).stop 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 msgr2=0x7f82400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 0x7f82400705f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8228000c00 tx=0x7f82280046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 shutdown_connections 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82401599a0 0x7f8240152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f821806aed0 0x7f821806d390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82400b2250 0x7f82400705f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 --2- 172.21.15.97:0/2786958426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82400adea0 0x7f82400711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 >> 172.21.15.97:0/2786958426 conn(0x7f82400b0a10 msgr2=0x7f82400ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 shutdown_connections 2024-01-31T05:03:13.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.750+0000 7f8247800640 1 -- 172.21.15.97:0/2786958426 wait complete. 2024-01-31T05:03:13.770 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:03:13.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.983+0000 7fda42000640 1 Processor -- start 2024-01-31T05:03:13.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.983+0000 7fda42000640 1 -- start start 2024-01-31T05:03:13.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.984+0000 7fda42000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda3c0ce1e0 0x7fda3c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda42000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c161fc0 con 0x7fda3c161b10 2024-01-31T05:03:13.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda42000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c1622a0 con 0x7fda3c0cb7d0 2024-01-31T05:03:13.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda42000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c162c40 con 0x7fda3c0ce1e0 2024-01-31T05:03:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda3b400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fda3c0ce1e0 0x7fda3c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda3aa00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fda3c161b10 0x7fda3c0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37902/0 (socket says 172.21.15.97:37902) 2024-01-31T05:03:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda3aa00640 1 -- 172.21.15.97:0/1556294588 learned_addr learned my addr 172.21.15.97:0/1556294588 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda3b400640 1 -- 172.21.15.97:0/1556294588 >> v1:172.21.15.188:6789/0 conn(0x7fda3c0cb7d0 legacy=0x7fda3c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda3b400640 1 -- 172.21.15.97:0/1556294588 >> v1:172.21.15.97:6789/0 conn(0x7fda3c161b10 legacy=0x7fda3c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.986+0000 7fda3b400640 1 -- 172.21.15.97:0/1556294588 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c162580 con 0x7fda3c0ce1e0 2024-01-31T05:03:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.985+0000 7fda3a000640 1 -- 172.21.15.97:0/1556294588 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132324834 0 0) 0x7fda3c161fc0 con 0x7fda3c161b10 2024-01-31T05:03:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.987+0000 7fda3b400640 1 --2- 172.21.15.97:0/1556294588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c0b2fc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fda24009790 tx=0x7fda2402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585b19982872a30 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.987+0000 7fda3a000640 1 -- 172.21.15.97:0/1556294588 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24031780 con 0x7fda3c0ce1e0 2024-01-31T05:03:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.987+0000 7fda3a000640 1 -- 172.21.15.97:0/1556294588 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda240318e0 con 0x7fda3c0ce1e0 2024-01-31T05:03:13.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.987+0000 7fda3a000640 1 -- 172.21.15.97:0/1556294588 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24031c70 con 0x7fda3c0ce1e0 2024-01-31T05:03:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.987+0000 7fda42000640 1 -- 172.21.15.97:0/1556294588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0ce1e0 msgr2=0x7fda3c0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.987+0000 7fda42000640 1 --2- 172.21.15.97:0/1556294588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c0b2fc0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fda24009790 tx=0x7fda2402b820 comp rx=0 tx=0).stop 2024-01-31T05:03:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.988+0000 7fda42000640 1 -- 172.21.15.97:0/1556294588 shutdown_connections 2024-01-31T05:03:13.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.988+0000 7fda42000640 1 --2- 172.21.15.97:0/1556294588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c0b2fc0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.988+0000 7fda42000640 1 -- 172.21.15.97:0/1556294588 >> 172.21.15.97:0/1556294588 conn(0x7fda3c0b0a10 msgr2=0x7fda3c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.988+0000 7fda42000640 1 -- 172.21.15.97:0/1556294588 shutdown_connections 2024-01-31T05:03:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.988+0000 7fda42000640 1 -- 172.21.15.97:0/1556294588 wait complete. 2024-01-31T05:03:13.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.989+0000 7fda42000640 1 Processor -- start 2024-01-31T05:03:13.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.989+0000 7fda42000640 1 -- start start 2024-01-31T05:03:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.990+0000 7fda42000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda3c0cb7d0 0x7fda3c09c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.990+0000 7fda42000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.990+0000 7fda42000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 0x7fda3c1f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.990+0000 7fda42000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c154680 con 0x7fda3c161b10 2024-01-31T05:03:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.990+0000 7fda42000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c1fa9a0 con 0x7fda3c0ce1e0 2024-01-31T05:03:13.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.990+0000 7fda42000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c09ae70 con 0x7fda3c0cb7d0 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 0x7fda3c1f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 0x7fda3c1f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40326/0 (socket says 172.21.15.97:40326) 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 -- 172.21.15.97:0/182355407 learned_addr learned my addr 172.21.15.97:0/182355407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3b400640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda3c0cb7d0 0x7fda3c09c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3aa00640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 -- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda3c0cb7d0 msgr2=0x7fda3c09c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda3c0cb7d0 0x7fda3c09c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 -- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda3c0ce1e0 msgr2=0x7fda3c154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3be00640 1 -- 172.21.15.97:0/182355407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda240093a0 con 0x7fda3c161b10 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.991+0000 7fda3b400640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda3c0cb7d0 0x7fda3c09c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.992+0000 7fda3be00640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 0x7fda3c1f7380 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b970 tx=0x7fda3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.992+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400cb40 con 0x7fda3c161b10 2024-01-31T05:03:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.992+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3400cca0 con 0x7fda3c161b10 2024-01-31T05:03:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.992+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34014680 con 0x7fda3c161b10 2024-01-31T05:03:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.992+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c1f7bb0 con 0x7fda3c161b10 2024-01-31T05:03:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.992+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c1f8170 con 0x7fda3c161b10 2024-01-31T05:03:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.993+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c067830 con 0x7fda3c161b10 2024-01-31T05:03:13.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.994+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fda3401d3b0 con 0x7fda3c161b10 2024-01-31T05:03:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.995+0000 7fda33400640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fda1406aed0 0x7fda1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.996+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fda340896c0 con 0x7fda3c161b10 2024-01-31T05:03:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.996+0000 7fda3b400640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fda1406aed0 0x7fda1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:13.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:13.996+0000 7fda3b400640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fda1406aed0 0x7fda1406d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fda24036000 tx=0x7fda2403d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:13.999 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.000+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fda34054330 con 0x7fda3c161b10 2024-01-31T05:03:14.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.192+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda3c0cc6c0 con 0x7fda3c161b10 2024-01-31T05:03:14.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.193+0000 7fda33400640 1 -- 172.21.15.97:0/182355407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda34057950 con 0x7fda3c161b10 2024-01-31T05:03:14.193 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fda1406aed0 msgr2=0x7fda1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fda1406aed0 0x7fda1406d390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fda24036000 tx=0x7fda2403d040 comp rx=0 tx=0).stop 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 msgr2=0x7fda3c1f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 0x7fda3c1f7380 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fda3400b970 tx=0x7fda3400be30 comp rx=0 tx=0).stop 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 shutdown_connections 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fda1406aed0 0x7fda1406d390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda3c161b10 0x7fda3c1f7380 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda3c0ce1e0 0x7fda3c154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 --2- 172.21.15.97:0/182355407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fda3c0cb7d0 0x7fda3c09c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 >> 172.21.15.97:0/182355407 conn(0x7fda3c0b0a10 msgr2=0x7fda3c0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 shutdown_connections 2024-01-31T05:03:14.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.196+0000 7fda42000640 1 -- 172.21.15.97:0/182355407 wait complete. 2024-01-31T05:03:14.214 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.426+0000 7f5563400640 1 Processor -- start 2024-01-31T05:03:14.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.426+0000 7f5563400640 1 -- start start 2024-01-31T05:03:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.428+0000 7f5563400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f555c15d690 0x7f555c0ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.428+0000 7f5563400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f555c161020 0x7f555c1614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.428+0000 7f5563400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c15db40 con 0x7f555c06d580 2024-01-31T05:03:14.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.428+0000 7f5563400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c15e7c0 con 0x7f555c161020 2024-01-31T05:03:14.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.428+0000 7f5563400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c15de20 con 0x7f555c15d690 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f555c161020 0x7f555c1614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f555c161020 0x7f555c1614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40338/0 (socket says 172.21.15.97:40338) 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 -- 172.21.15.97:0/3055040323 learned_addr learned my addr 172.21.15.97:0/3055040323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f555be00640 1 --2- 172.21.15.97:0/3055040323 >> v2:172.21.15.188:3300/0 conn(0x7f555c15d690 0x7f555c0ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 -- 172.21.15.97:0/3055040323 >> v2:172.21.15.188:3300/0 conn(0x7f555c15d690 msgr2=0x7f555c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 --2- 172.21.15.97:0/3055040323 >> v2:172.21.15.188:3300/0 conn(0x7f555c15d690 0x7f555c0ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 -- 172.21.15.97:0/3055040323 >> v1:172.21.15.188:6789/0 conn(0x7f555c06d580 legacy=0x7f555c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:14.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.429+0000 7f5561400640 1 -- 172.21.15.97:0/3055040323 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c15e100 con 0x7f555c161020 2024-01-31T05:03:14.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.430+0000 7f5561400640 1 --2- 172.21.15.97:0/3055040323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c1614d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5554009e50 tx=0x7f555402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa7a3364efd76bf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:14.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.430+0000 7f555b400640 1 -- 172.21.15.97:0/3055040323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555402ca80 con 0x7f555c161020 2024-01-31T05:03:14.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.430+0000 7f555b400640 1 -- 172.21.15.97:0/3055040323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555402cbe0 con 0x7f555c161020 2024-01-31T05:03:14.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f555b400640 1 -- 172.21.15.97:0/3055040323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540343f0 con 0x7f555c161020 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 -- 172.21.15.97:0/3055040323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 msgr2=0x7f555c1614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 --2- 172.21.15.97:0/3055040323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c1614d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5554009e50 tx=0x7f555402bee0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 -- 172.21.15.97:0/3055040323 shutdown_connections 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 --2- 172.21.15.97:0/3055040323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c1614d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 --2- 172.21.15.97:0/3055040323 >> v2:172.21.15.188:3300/0 conn(0x7f555c15d690 0x7f555c0ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 -- 172.21.15.97:0/3055040323 >> 172.21.15.97:0/3055040323 conn(0x7f555c0b0a10 msgr2=0x7f555c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:14.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 -- 172.21.15.97:0/3055040323 shutdown_connections 2024-01-31T05:03:14.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.431+0000 7f5563400640 1 -- 172.21.15.97:0/3055040323 wait complete. 2024-01-31T05:03:14.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.433+0000 7f5563400640 1 Processor -- start 2024-01-31T05:03:14.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.433+0000 7f5563400640 1 -- start start 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5563400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f555c06d580 0x7f555c1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5563400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f555c15d690 0x7f555c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5563400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5563400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c151da0 con 0x7f555c161020 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5563400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c159b50 con 0x7f555c06d580 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5563400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c15ae00 con 0x7f555c15d690 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5561400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5561400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40342/0 (socket says 172.21.15.97:40342) 2024-01-31T05:03:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.434+0000 7f5561400640 1 -- 172.21.15.97:0/2673946890 learned_addr learned my addr 172.21.15.97:0/2673946890 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5560a00640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f555c06d580 0x7f555c1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f555be00640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f555c15d690 0x7f555c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5561400640 1 -- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f555c15d690 msgr2=0x7f555c0ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5561400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f555c15d690 0x7f555c0ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5561400640 1 -- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f555c06d580 msgr2=0x7f555c1489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5561400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f555c06d580 0x7f555c1489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5561400640 1 -- 172.21.15.97:0/2673946890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554009b30 con 0x7f555c161020 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.435+0000 7f5561400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c159610 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5554009e50 tx=0x7f5554004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.436+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554009510 con 0x7f555c161020 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.436+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555403e450 con 0x7f555c161020 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.436+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555403e7e0 con 0x7f555c161020 2024-01-31T05:03:14.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.436+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c15b0e0 con 0x7f555c161020 2024-01-31T05:03:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.436+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c15b6a0 con 0x7f555c161020 2024-01-31T05:03:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.437+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c067830 con 0x7f555c161020 2024-01-31T05:03:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.438+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f555402cd50 con 0x7f555c161020 2024-01-31T05:03:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.439+0000 7f5558c00640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f553406aed0 0x7f553406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.439+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f55540ac450 con 0x7f555c161020 2024-01-31T05:03:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.439+0000 7f5560a00640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f553406aed0 0x7f553406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.440+0000 7f5560a00640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f553406aed0 0x7f553406d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b7f0 tx=0x7f554c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:14.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.443+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5554077040 con 0x7f555c161020 2024-01-31T05:03:14.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.636+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f555c0cc6c0 con 0x7f555c161020 2024-01-31T05:03:14.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.637+0000 7f5558c00640 1 -- 172.21.15.97:0/2673946890 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f555407a660 con 0x7f555c161020 2024-01-31T05:03:14.636 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f553406aed0 msgr2=0x7f553406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f553406aed0 0x7f553406d390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b7f0 tx=0x7f554c008040 comp rx=0 tx=0).stop 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 msgr2=0x7f555c159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c159610 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f5554009e50 tx=0x7f5554004a40 comp rx=0 tx=0).stop 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 shutdown_connections 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f553406aed0 0x7f553406d390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c161020 0x7f555c159610 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f555c15d690 0x7f555c0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 --2- 172.21.15.97:0/2673946890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f555c06d580 0x7f555c1489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.639+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 >> 172.21.15.97:0/2673946890 conn(0x7f555c0b0a10 msgr2=0x7f555c15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:14.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.640+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 shutdown_connections 2024-01-31T05:03:14.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.640+0000 7f5563400640 1 -- 172.21.15.97:0/2673946890 wait complete. 2024-01-31T05:03:14.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:03:14.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.868+0000 7f8e9fa00640 1 Processor -- start 2024-01-31T05:03:14.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.868+0000 7f8e9fa00640 1 -- start start 2024-01-31T05:03:14.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e9fa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8e9815a730 0x7f8e980abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e9fa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9815abc0 con 0x7f8e9815a350 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e9fa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9815aea0 con 0x7f8e980cde10 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e9fa00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e980b2c90 con 0x7f8e9815a730 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e9d000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8e9815a350 0x7f8e9806da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37908/0 (socket says 172.21.15.97:37908) 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e9d000640 1 -- 172.21.15.97:0/3109213621 learned_addr learned my addr 172.21.15.97:0/3109213621 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e97e00640 1 --2- 172.21.15.97:0/3109213621 >> v2:172.21.15.97:3300/0 conn(0x7f8e9815a730 0x7f8e980abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.870+0000 7f8e97400640 1 -- 172.21.15.97:0/3109213621 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495329853 0 0) 0x7f8e9815abc0 con 0x7f8e9815a350 2024-01-31T05:03:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.871+0000 7f8e97400640 1 -- 172.21.15.97:0/3109213621 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e78003520 con 0x7f8e9815a350 2024-01-31T05:03:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.871+0000 7f8e97e00640 1 -- 172.21.15.97:0/3109213621 >> v1:172.21.15.188:6789/0 conn(0x7f8e980cde10 legacy=0x7f8e980cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.871+0000 7f8e97e00640 1 -- 172.21.15.97:0/3109213621 >> v1:172.21.15.97:6789/0 conn(0x7f8e9815a350 legacy=0x7f8e9806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.871+0000 7f8e97e00640 1 -- 172.21.15.97:0/3109213621 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9815b180 con 0x7f8e9815a730 2024-01-31T05:03:14.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.872+0000 7f8e97e00640 1 --2- 172.21.15.97:0/3109213621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a730 0x7f8e980abcf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9142cf9a2d8e310f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:14.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.872+0000 7f8e97400640 1 -- 172.21.15.97:0/3109213621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8002e070 con 0x7f8e9815a730 2024-01-31T05:03:14.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.872+0000 7f8e97400640 1 -- 172.21.15.97:0/3109213621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e800315a0 con 0x7f8e9815a730 2024-01-31T05:03:14.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.872+0000 7f8e97400640 1 -- 172.21.15.97:0/3109213621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031930 con 0x7f8e9815a730 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 -- 172.21.15.97:0/3109213621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a730 msgr2=0x7f8e980abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/3109213621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a730 0x7f8e980abcf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002b840 comp rx=0 tx=0).stop 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 -- 172.21.15.97:0/3109213621 shutdown_connections 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/3109213621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a730 0x7f8e980abcf0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 -- 172.21.15.97:0/3109213621 >> 172.21.15.97:0/3109213621 conn(0x7f8e980b0a10 msgr2=0x7f8e980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 -- 172.21.15.97:0/3109213621 shutdown_connections 2024-01-31T05:03:14.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.873+0000 7f8e9fa00640 1 -- 172.21.15.97:0/3109213621 wait complete. 2024-01-31T05:03:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.874+0000 7f8e9fa00640 1 Processor -- start 2024-01-31T05:03:14.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 -- start start 2024-01-31T05:03:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980cde10 0x7f8e9814a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 0x7f8e980709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e98159720 0x7f8e98070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9814e670 con 0x7f8e9815a350 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9814e8c0 con 0x7f8e980cde10 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.875+0000 7f8e9fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9814eba0 con 0x7f8e98159720 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 0x7f8e980709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 0x7f8e980709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40370/0 (socket says 172.21.15.97:40370) 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 -- 172.21.15.97:0/2771148161 learned_addr learned my addr 172.21.15.97:0/2771148161 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e9da00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e98159720 0x7f8e98070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 -- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e98159720 msgr2=0x7f8e98070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e9d000640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980cde10 0x7f8e9814a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e98159720 0x7f8e98070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 -- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980cde10 msgr2=0x7f8e9814a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980cde10 0x7f8e9814a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e97e00640 1 -- 172.21.15.97:0/2771148161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80009400 con 0x7f8e9815a350 2024-01-31T05:03:14.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e9da00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e98159720 0x7f8e98070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.876+0000 7f8e9d000640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980cde10 0x7f8e9814a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.877+0000 7f8e97e00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 0x7f8e980709c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e80034040 tx=0x7f8e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.877+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80032840 con 0x7f8e9815a350 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.877+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e80031f00 con 0x7f8e9815a350 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.877+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80032be0 con 0x7f8e9815a350 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.877+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9814ee80 con 0x7f8e9815a350 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.877+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e980cb160 con 0x7f8e9815a350 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.878+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9814f340 con 0x7f8e9815a350 2024-01-31T05:03:14.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.879+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8e80032d40 con 0x7f8e9815a350 2024-01-31T05:03:14.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.880+0000 7f8e94c00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e7006af00 0x7f8e7006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:14.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.880+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f8e800ab860 con 0x7f8e9815a350 2024-01-31T05:03:14.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.880+0000 7f8e9d000640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e7006af00 0x7f8e7006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:14.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.881+0000 7f8e9d000640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e7006af00 0x7f8e7006d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8e88007fd0 tx=0x7f8e88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:14.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:14.885+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8e80076450 con 0x7f8e9815a350 2024-01-31T05:03:15.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.077+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f8e9814f340 con 0x7f8e9815a350 2024-01-31T05:03:15.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.078+0000 7f8e94c00640 1 -- 172.21.15.97:0/2771148161 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7f8e80079a70 con 0x7f8e9815a350 2024-01-31T05:03:15.077 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:15.080 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e7006af00 msgr2=0x7f8e7006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e7006af00 0x7f8e7006d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8e88007fd0 tx=0x7f8e88009040 comp rx=0 tx=0).stop 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 msgr2=0x7f8e980709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 0x7f8e980709c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e80034040 tx=0x7f8e800046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 shutdown_connections 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e98159720 0x7f8e98070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8e7006af00 0x7f8e7006d3c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e9815a350 0x7f8e980709c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 --2- 172.21.15.97:0/2771148161 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980cde10 0x7f8e9814a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.080+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 >> 172.21.15.97:0/2771148161 conn(0x7f8e980b0a10 msgr2=0x7f8e980cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.081+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 shutdown_connections 2024-01-31T05:03:15.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.081+0000 7f8e9fa00640 1 -- 172.21.15.97:0/2771148161 wait complete. 2024-01-31T05:03:15.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:03:15.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.301+0000 7feedfc00640 1 Processor -- start 2024-01-31T05:03:15.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.301+0000 7feedfc00640 1 -- start start 2024-01-31T05:03:15.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feedfc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feed81663a0 0x7feed8168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feedfc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed80ceb40 con 0x7feed806d580 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feedfc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed815db40 con 0x7feed815d690 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feedfc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed815e7c0 con 0x7feed81663a0 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feeddc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feed81663a0 0x7feed8168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feeddc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feed81663a0 0x7feed8168790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60482/0 (socket says 172.21.15.97:60482) 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.303+0000 7feeddc00640 1 -- 172.21.15.97:0/343010038 learned_addr learned my addr 172.21.15.97:0/343010038 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.304+0000 7feeddc00640 1 -- 172.21.15.97:0/343010038 >> v1:172.21.15.188:6789/0 conn(0x7feed815d690 legacy=0x7feed80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.304+0000 7feeddc00640 1 -- 172.21.15.97:0/343010038 >> v1:172.21.15.97:6789/0 conn(0x7feed806d580 legacy=0x7feed806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.304+0000 7feeddc00640 1 -- 172.21.15.97:0/343010038 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed815de20 con 0x7feed81663a0 2024-01-31T05:03:15.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.305+0000 7feeddc00640 1 --2- 172.21.15.97:0/343010038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed81663a0 0x7feed8168790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feed0009e50 tx=0x7feed002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e7ca6f119b0f0c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:15.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.305+0000 7feed7400640 1 -- 172.21.15.97:0/343010038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed002cb10 con 0x7feed81663a0 2024-01-31T05:03:15.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.305+0000 7feed7400640 1 -- 172.21.15.97:0/343010038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed002cc70 con 0x7feed81663a0 2024-01-31T05:03:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.305+0000 7feed7400640 1 -- 172.21.15.97:0/343010038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed00343f0 con 0x7feed81663a0 2024-01-31T05:03:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 -- 172.21.15.97:0/343010038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed81663a0 msgr2=0x7feed8168790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 --2- 172.21.15.97:0/343010038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed81663a0 0x7feed8168790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feed0009e50 tx=0x7feed002bf70 comp rx=0 tx=0).stop 2024-01-31T05:03:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 -- 172.21.15.97:0/343010038 shutdown_connections 2024-01-31T05:03:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 --2- 172.21.15.97:0/343010038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed81663a0 0x7feed8168790 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 -- 172.21.15.97:0/343010038 >> 172.21.15.97:0/343010038 conn(0x7feed80b0a10 msgr2=0x7feed80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:15.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 -- 172.21.15.97:0/343010038 shutdown_connections 2024-01-31T05:03:15.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.306+0000 7feedfc00640 1 -- 172.21.15.97:0/343010038 wait complete. 2024-01-31T05:03:15.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.307+0000 7feedfc00640 1 Processor -- start 2024-01-31T05:03:15.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.307+0000 7feedfc00640 1 -- start start 2024-01-31T05:03:15.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.308+0000 7feedfc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feed806d580 0x7feed8140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.308+0000 7feedfc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 0x7feed815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.308+0000 7feedfc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feed81663a0 0x7feed8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.308+0000 7feedfc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8072670 con 0x7feed815d690 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.308+0000 7feedfc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8140dc0 con 0x7feed806d580 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.308+0000 7feedfc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed8158170 con 0x7feed81663a0 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 0x7feed815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 0x7feed815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60486/0 (socket says 172.21.15.97:60486) 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 -- 172.21.15.97:0/1504626012 learned_addr learned my addr 172.21.15.97:0/1504626012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feedd200640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feed806d580 0x7feed8140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 -- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feed81663a0 msgr2=0x7feed8158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feed81663a0 0x7feed8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 -- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feed806d580 msgr2=0x7feed8140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feed806d580 0x7feed8140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 -- 172.21.15.97:0/1504626012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed0009b30 con 0x7feed815d690 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.309+0000 7feed7e00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 0x7feed815ebd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feec000c400 tx=0x7feec000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.310+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0011790 con 0x7feed815d690 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.310+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0011930 con 0x7feed815d690 2024-01-31T05:03:15.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.310+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed8158450 con 0x7feed815d690 2024-01-31T05:03:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.310+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0011ce0 con 0x7feed815d690 2024-01-31T05:03:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.310+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8158a10 con 0x7feed815d690 2024-01-31T05:03:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.311+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed8067830 con 0x7feed815d690 2024-01-31T05:03:15.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.312+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7feec001c460 con 0x7feed815d690 2024-01-31T05:03:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.313+0000 7feed4c00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feeb006af00 0x7feeb006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.313+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7feec0089880 con 0x7feed815d690 2024-01-31T05:03:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.314+0000 7feedd200640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feeb006af00 0x7feeb006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.314+0000 7feedd200640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feeb006af00 0x7feeb006d3c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feec8001590 tx=0x7feec8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.318+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7feec0054480 con 0x7feed815d690 2024-01-31T05:03:15.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.525+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feed80b3e10 con 0x7feed815d690 2024-01-31T05:03:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.525+0000 7feed4c00640 1 -- 172.21.15.97:0/1504626012 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7feec0057aa0 con 0x7feed815d690 2024-01-31T05:03:15.525 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:03:15.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feeb006af00 msgr2=0x7feeb006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feeb006af00 0x7feeb006d3c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7feec8001590 tx=0x7feec8009040 comp rx=0 tx=0).stop 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 msgr2=0x7feed815ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 0x7feed815ebd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feec000c400 tx=0x7feec000c8c0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 shutdown_connections 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feed81663a0 0x7feed8158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7feeb006af00 0x7feeb006d3c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed815d690 0x7feed815ebd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 --2- 172.21.15.97:0/1504626012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feed806d580 0x7feed8140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 >> 172.21.15.97:0/1504626012 conn(0x7feed80b0a10 msgr2=0x7feed806d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 shutdown_connections 2024-01-31T05:03:15.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.528+0000 7feedfc00640 1 -- 172.21.15.97:0/1504626012 wait complete. 2024-01-31T05:03:15.547 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:15.547 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:15.598 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.847+0000 7fbfa9800640 1 Processor -- start 2024-01-31T05:03:15.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.847+0000 7fbfa9800640 1 -- start start 2024-01-31T05:03:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbfa4071e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa9800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa415a5c0 con 0x7fbfa40c9f40 2024-01-31T05:03:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa9800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40cd8d0 con 0x7fbfa40abcf0 2024-01-31T05:03:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40cd2c0 con 0x7fbfa40c9420 2024-01-31T05:03:15.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbfa40c9f40 0x7fbfa40ca320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49528/0 (socket says 172.21.15.97:49528) 2024-01-31T05:03:15.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa3e00640 1 -- 172.21.15.97:0/2781333118 learned_addr learned my addr 172.21.15.97:0/2781333118 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa3400640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbfa4071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570368150 0 0) 0x7fbfa415a5c0 con 0x7fbfa40c9f40 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.849+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf84005730 con 0x7fbfa40c9f40 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 399390965 0 0) 0x7fbf84005730 con 0x7fbfa40c9f40 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 msgr2=0x7fbfa4071e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbfa4071e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 >> v1:172.21.15.188:6789/0 conn(0x7fbfa40abcf0 legacy=0x7fbfa40cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa406df80 con 0x7fbfa40c9f40 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fbf94001670 con 0x7fbfa40c9f40 2024-01-31T05:03:15.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 >> v1:172.21.15.97:6789/0 conn(0x7fbfa40c9f40 legacy=0x7fbfa40ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbf84005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf840069d0 0x7fbf84006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbf84007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40cd2c0 con 0x7fbf84005bb0 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8400dc40 con 0x7fbf840069d0 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8400de00 con 0x7fbfa40c9420 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbf84005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.850+0000 7fbfa3400640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf840069d0 0x7fbf84006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa8e00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbf84007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa3e00640 1 -- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 msgr2=0x7fbf84007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbf84007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa3e00640 1 -- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf840069d0 msgr2=0x7fbf84006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf840069d0 0x7fbf84006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa3e00640 1 -- 172.21.15.97:0/2781333118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf84005730 con 0x7fbf84005bb0 2024-01-31T05:03:15.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa3400640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf840069d0 0x7fbf84006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:15.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.851+0000 7fbfa8e00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40c9420 0x7fbf84007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:15.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbf84005fd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbf94007fd0 tx=0x7fbf84007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5be99b1f1c13aa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94005ad0 con 0x7fbf84005bb0 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94005c30 con 0x7fbf84005bb0 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa2a00640 1 -- 172.21.15.97:0/2781333118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9402c850 con 0x7fbf84005bb0 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa9800640 1 -- 172.21.15.97:0/2781333118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 msgr2=0x7fbf84005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa9800640 1 --2- 172.21.15.97:0/2781333118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbf84005fd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbf94007fd0 tx=0x7fbf84007a00 comp rx=0 tx=0).stop 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa8e00640 1 -- 172.21.15.97:0/2781333118 reap_dead start 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa9800640 1 -- 172.21.15.97:0/2781333118 shutdown_connections 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.852+0000 7fbfa9800640 1 -- 172.21.15.97:0/2781333118 >> 172.21.15.97:0/2781333118 conn(0x7fbfa40cec40 msgr2=0x7fbfa40cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:15.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.853+0000 7fbfa9800640 1 -- 172.21.15.97:0/2781333118 shutdown_connections 2024-01-31T05:03:15.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.853+0000 7fbfa9800640 1 -- 172.21.15.97:0/2781333118 wait complete. 2024-01-31T05:03:15.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.854+0000 7fbfa9800640 1 Processor -- start 2024-01-31T05:03:15.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.854+0000 7fbfa9800640 1 -- start start 2024-01-31T05:03:15.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.855+0000 7fbfa9800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbfa41ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.855+0000 7fbfa9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40999c0 0x7fbfa40b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.855+0000 7fbfa9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa409aef0 0x7fbfa409b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.855+0000 7fbfa9800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa409aa60 con 0x7fbf84005bb0 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.855+0000 7fbfa9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa414eb50 con 0x7fbfa40999c0 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.855+0000 7fbfa9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4099e50 con 0x7fbfa409aef0 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbfa41ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbfa41ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60500/0 (socket says 172.21.15.97:60500) 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 -- 172.21.15.97:0/910590061 learned_addr learned my addr 172.21.15.97:0/910590061 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 -- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa409aef0 msgr2=0x7fbfa409b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa409aef0 0x7fbfa409b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 -- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40999c0 msgr2=0x7fbfa40b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3400640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40999c0 0x7fbfa40b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40999c0 0x7fbfa40b2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 -- 172.21.15.97:0/910590061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa409a130 con 0x7fbf84005bb0 2024-01-31T05:03:15.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.856+0000 7fbfa3e00640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbfa41ee470 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf94000c00 tx=0x7fbf9402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.857+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9403c040 con 0x7fbf84005bb0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.857+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9402cc60 con 0x7fbf84005bb0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.857+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94007070 con 0x7fbf84005bb0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.857+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa409a410 con 0x7fbf84005bb0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.857+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa40b3150 con 0x7fbf84005bb0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.858+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa4072dd0 con 0x7fbf84005bb0 2024-01-31T05:03:15.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.859+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fbf94032070 con 0x7fbf84005bb0 2024-01-31T05:03:15.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.860+0000 7fbf9be00640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf7c06aed0 0x7fbf7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:15.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.860+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fbf940aab80 con 0x7fbf84005bb0 2024-01-31T05:03:15.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.860+0000 7fbfa3400640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf7c06aed0 0x7fbf7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:15.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.861+0000 7fbfa3400640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf7c06aed0 0x7fbf7c06d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40b3730 tx=0x7fbf8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:15.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:15.865+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fbf94075650 con 0x7fbf84005bb0 2024-01-31T05:03:16.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.048+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfa40676b0 con 0x7fbf84005bb0 2024-01-31T05:03:16.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.050+0000 7fbf9be00640 1 -- 172.21.15.97:0/910590061 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fbf940aae60 con 0x7fbf84005bb0 2024-01-31T05:03:16.050 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:16.050 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf7c06aed0 msgr2=0x7fbf7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf7c06aed0 0x7fbf7c06d390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40b3730 tx=0x7fbf8c008040 comp rx=0 tx=0).stop 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 msgr2=0x7fbfa41ee470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbfa41ee470 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbf94000c00 tx=0x7fbf9402e070 comp rx=0 tx=0).stop 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 shutdown_connections 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa409aef0 0x7fbfa409b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40999c0 0x7fbfa40b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbf7c06aed0 0x7fbf7c06d390 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 --2- 172.21.15.97:0/910590061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf84005bb0 0x7fbfa41ee470 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 >> 172.21.15.97:0/910590061 conn(0x7fbfa40cec40 msgr2=0x7fbfa4071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 shutdown_connections 2024-01-31T05:03:16.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.053+0000 7fbfa9800640 1 -- 172.21.15.97:0/910590061 wait complete. 2024-01-31T05:03:16.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:16.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.278+0000 7fa383200640 1 Processor -- start 2024-01-31T05:03:16.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.278+0000 7fa383200640 1 -- start start 2024-01-31T05:03:16.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa383200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 0x7fa37c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa383200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa37c0ce680 0x7fa37c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa383200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37c0b3f60 con 0x7fa37c0ae360 2024-01-31T05:03:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa383200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37c0b41c0 con 0x7fa37c0ce680 2024-01-31T05:03:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa383200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c15c540 con 0x7fa37c15b080 2024-01-31T05:03:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa381400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa37c0ae360 0x7fa37c0cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49530/0 (socket says 172.21.15.97:49530) 2024-01-31T05:03:16.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa381400640 1 -- 172.21.15.97:0/3112562245 learned_addr learned my addr 172.21.15.97:0/3112562245 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa381e00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 0x7fa37c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.280+0000 7fa382800640 1 --2- 172.21.15.97:0/3112562245 >> v2:172.21.15.188:3300/0 conn(0x7fa37c0ce680 0x7fa37c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929101540 0 0) 0x7fa37c0b3f60 con 0x7fa37c0ae360 2024-01-31T05:03:16.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa35c003520 con 0x7fa37c0ae360 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3687211185 0 0) 0x7fa35c003520 con 0x7fa37c0ae360 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 msgr2=0x7fa37c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 0x7fa37c0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 >> v2:172.21.15.188:3300/0 conn(0x7fa37c0ce680 msgr2=0x7fa37c0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 --2- 172.21.15.97:0/3112562245 >> v2:172.21.15.188:3300/0 conn(0x7fa37c0ce680 0x7fa37c0b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c15b780 con 0x7fa37c0ae360 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa364001670 con 0x7fa37c0ae360 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 >> v1:172.21.15.97:6789/0 conn(0x7fa37c0ae360 legacy=0x7fa37c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35c0039e0 0x7fa35c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35c0047c0 0x7fa35c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 0x7fa35c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c0b41c0 con 0x7fa35c0039e0 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c15c540 con 0x7fa35c0047c0 2024-01-31T05:03:16.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.281+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c00b9b0 con 0x7fa37c15b080 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa381e00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35c0039e0 0x7fa35c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa382800640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 0x7fa35c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa381e00640 1 -- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 msgr2=0x7fa35c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa381e00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa37c15b080 0x7fa35c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa381e00640 1 -- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35c0047c0 msgr2=0x7fa35c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa381e00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa35c0047c0 0x7fa35c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.282+0000 7fa381e00640 1 -- 172.21.15.97:0/3112562245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c003520 con 0x7fa35c0039e0 2024-01-31T05:03:16.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.283+0000 7fa381e00640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35c0039e0 0x7fa35c003e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009ac0 tx=0x7fa35c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1ee47ef9ddd8e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.283+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c02c6b0 con 0x7fa35c0039e0 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.283+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c02c810 con 0x7fa35c0039e0 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.283+0000 7fa380a00640 1 -- 172.21.15.97:0/3112562245 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c02cba0 con 0x7fa35c0039e0 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa383200640 1 -- 172.21.15.97:0/3112562245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35c0039e0 msgr2=0x7fa35c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa383200640 1 --2- 172.21.15.97:0/3112562245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa35c0039e0 0x7fa35c003e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009ac0 tx=0x7fa35c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa382800640 1 -- 172.21.15.97:0/3112562245 reap_dead start 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa383200640 1 -- 172.21.15.97:0/3112562245 shutdown_connections 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa383200640 1 -- 172.21.15.97:0/3112562245 >> 172.21.15.97:0/3112562245 conn(0x7fa37c06ad40 msgr2=0x7fa37c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa383200640 1 -- 172.21.15.97:0/3112562245 shutdown_connections 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.284+0000 7fa383200640 1 -- 172.21.15.97:0/3112562245 wait complete. 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.286+0000 7fa383200640 1 Processor -- start 2024-01-31T05:03:16.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.286+0000 7fa383200640 1 -- start start 2024-01-31T05:03:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa383200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35c0039e0 0x7fa37c1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa383200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 0x7fa37c152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa383200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa37c09b130 0x7fa37c09b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa383200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c1f0a30 con 0x7fa37c158420 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa383200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c15a1a0 con 0x7fa37c09b130 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa383200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c1f4d00 con 0x7fa35c0039e0 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa381400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 0x7fa37c152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa381400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 0x7fa37c152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60514/0 (socket says 172.21.15.97:60514) 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.287+0000 7fa381400640 1 -- 172.21.15.97:0/969994786 learned_addr learned my addr 172.21.15.97:0/969994786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381e00640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35c0039e0 0x7fa37c1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa382800640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa37c09b130 0x7fa37c09b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381400640 1 -- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35c0039e0 msgr2=0x7fa37c1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381400640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35c0039e0 0x7fa37c1f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381400640 1 -- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa37c09b130 msgr2=0x7fa37c09b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381400640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa37c09b130 0x7fa37c09b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381400640 1 -- 172.21.15.97:0/969994786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c1588a0 con 0x7fa37c158420 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa382800640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa37c09b130 0x7fa37c09b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:16.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa381400640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 0x7fa37c152ab0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c700 tx=0x7fa36400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3640072d0 con 0x7fa37c158420 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364007430 con 0x7fa37c158420 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3640047e0 con 0x7fa37c158420 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.288+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c1f4460 con 0x7fa37c158420 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.289+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c1f4a20 con 0x7fa37c158420 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.290+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c1f0790 con 0x7fa37c158420 2024-01-31T05:03:16.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.291+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa364004940 con 0x7fa37c158420 2024-01-31T05:03:16.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.291+0000 7fa37a000640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa35406aed0 0x7fa35406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.292+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa364088d40 con 0x7fa37c158420 2024-01-31T05:03:16.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.292+0000 7fa381e00640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa35406aed0 0x7fa35406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.292+0000 7fa381e00640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa35406aed0 0x7fa35406d390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa36c036000 tx=0x7fa36c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:16.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.296+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa37c1f0790 con 0x7fa37c158420 2024-01-31T05:03:16.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.488+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa37c1f0b70 con 0x7fa37c158420 2024-01-31T05:03:16.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.489+0000 7fa37a000640 1 -- 172.21.15.97:0/969994786 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa37c1f0b70 con 0x7fa37c158420 2024-01-31T05:03:16.489 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa35406aed0 msgr2=0x7fa35406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa35406aed0 0x7fa35406d390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa36c036000 tx=0x7fa36c030000 comp rx=0 tx=0).stop 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 msgr2=0x7fa37c152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 0x7fa37c152ab0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c700 tx=0x7fa36400cbc0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 shutdown_connections 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa37c09b130 0x7fa37c09b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa35406aed0 0x7fa35406d390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa37c158420 0x7fa37c152ab0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 --2- 172.21.15.97:0/969994786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35c0039e0 0x7fa37c1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 >> 172.21.15.97:0/969994786 conn(0x7fa37c06ad40 msgr2=0x7fa37c0b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.491+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 shutdown_connections 2024-01-31T05:03:16.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.492+0000 7fa383200640 1 -- 172.21.15.97:0/969994786 wait complete. 2024-01-31T05:03:16.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:03:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.722+0000 7f1891e00640 1 Processor -- start 2024-01-31T05:03:16.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.722+0000 7f1891e00640 1 -- start start 2024-01-31T05:03:16.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.724+0000 7f1891e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f188c15a730 0x7f188c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.724+0000 7f1891e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c15abc0 con 0x7f188c15a350 2024-01-31T05:03:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.724+0000 7f1891e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c15aea0 con 0x7f188c0cde10 2024-01-31T05:03:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.724+0000 7f1891e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c0b2c90 con 0x7f188c15a730 2024-01-31T05:03:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.724+0000 7f188aa00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f188c15a350 0x7f188c06da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49532/0 (socket says 172.21.15.97:49532) 2024-01-31T05:03:16.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.724+0000 7f188aa00640 1 -- 172.21.15.97:0/2202297777 learned_addr learned my addr 172.21.15.97:0/2202297777 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:16.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f1881e00640 1 --2- 172.21.15.97:0/2202297777 >> v2:172.21.15.188:3300/0 conn(0x7f188c15a730 0x7f188c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257986979 0 0) 0x7f188c15abc0 con 0x7f188c15a350 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f186c005730 con 0x7f188c15a350 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565222753 0 0) 0x7f186c005730 con 0x7f188c15a350 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 >> v2:172.21.15.188:3300/0 conn(0x7f188c15a730 msgr2=0x7f188c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 --2- 172.21.15.97:0/2202297777 >> v2:172.21.15.188:3300/0 conn(0x7f188c15a730 0x7f188c0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 >> v1:172.21.15.188:6789/0 conn(0x7f188c0cde10 legacy=0x7f188c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c15b180 con 0x7f188c15a350 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f187c001b70 con 0x7f188c15a350 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 >> v1:172.21.15.97:6789/0 conn(0x7f188c15a350 legacy=0x7f188c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f186c005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0069d0 0x7f186c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c0074b0 0x7f186c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0b2c90 con 0x7f186c005b80 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c00e360 con 0x7f186c0074b0 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.725+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c00e520 con 0x7f186c0069d0 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188aa00640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f186c005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188aa00640 1 -- 172.21.15.97:0/2202297777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0069d0 msgr2=0x7f186c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188aa00640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186c0069d0 0x7f186c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188aa00640 1 -- 172.21.15.97:0/2202297777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c0074b0 msgr2=0x7f186c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188aa00640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f186c0074b0 0x7f186c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188aa00640 1 -- 172.21.15.97:0/2202297777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c005730 con 0x7f186c005b80 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.726+0000 7f188b400640 1 -- 172.21.15.97:0/2202297777 reap_dead start 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.727+0000 7f188aa00640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f186c005f80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c510 tx=0x7f186c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f217d6842e00b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.727+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c005b20 con 0x7f186c005b80 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.727+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c005c80 con 0x7f186c005b80 2024-01-31T05:03:16.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.727+0000 7f188a000640 1 -- 172.21.15.97:0/2202297777 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0072b0 con 0x7f186c005b80 2024-01-31T05:03:16.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 -- 172.21.15.97:0/2202297777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 msgr2=0x7f186c005f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f186c005f80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c510 tx=0x7f186c007a00 comp rx=0 tx=0).stop 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 -- 172.21.15.97:0/2202297777 shutdown_connections 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 --2- 172.21.15.97:0/2202297777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f186c005f80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 -- 172.21.15.97:0/2202297777 >> 172.21.15.97:0/2202297777 conn(0x7f188c0b0a10 msgr2=0x7f188c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 -- 172.21.15.97:0/2202297777 shutdown_connections 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.728+0000 7f1891e00640 1 -- 172.21.15.97:0/2202297777 wait complete. 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.730+0000 7f1891e00640 1 Processor -- start 2024-01-31T05:03:16.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.730+0000 7f1891e00640 1 -- start start 2024-01-31T05:03:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1891e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f188c0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1891e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f188c09c1d0 0x7f188c09c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1891e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f188c157450 0x7f188c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1891e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1f2ff0 con 0x7f186c005b80 2024-01-31T05:03:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1891e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1f2370 con 0x7f188c157450 2024-01-31T05:03:16.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1891e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c1f2650 con 0x7f188c09c1d0 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f188c0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f188c0ac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60526/0 (socket says 172.21.15.97:60526) 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 -- 172.21.15.97:0/150867568 learned_addr learned my addr 172.21.15.97:0/150867568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1881e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f188c09c1d0 0x7f188c09c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 -- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f188c09c1d0 msgr2=0x7f188c09c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f188c09c1d0 0x7f188c09c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 -- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f188c157450 msgr2=0x7f188c1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f188c157450 0x7f188c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f188aa00640 1 -- 172.21.15.97:0/150867568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c1f2930 con 0x7f186c005b80 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.731+0000 7f1881e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f188c09c1d0 0x7f188c09c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.732+0000 7f188aa00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f188c0ac730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f187c001e30 tx=0x7f187c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.732+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c03c040 con 0x7f186c005b80 2024-01-31T05:03:16.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.732+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c035670 con 0x7f186c005b80 2024-01-31T05:03:16.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.732+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c005b20 con 0x7f186c005b80 2024-01-31T05:03:16.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.732+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c1f2c10 con 0x7f186c005b80 2024-01-31T05:03:16.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.732+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c157e20 con 0x7f186c005b80 2024-01-31T05:03:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.733+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c067830 con 0x7f186c005b80 2024-01-31T05:03:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.734+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f187c032070 con 0x7f186c005b80 2024-01-31T05:03:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.735+0000 7f1883e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f186406aed0 0x7f186406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.735+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f187c0ab610 con 0x7f186c005b80 2024-01-31T05:03:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.735+0000 7f1881e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f186406aed0 0x7f186406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:16.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.736+0000 7f1881e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f186406aed0 0x7f186406d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1874006fd0 tx=0x7f1874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.739+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f187c076210 con 0x7f186c005b80 2024-01-31T05:03:16.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.934+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f188c0cce00 con 0x7f186c005b80 2024-01-31T05:03:16.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.935+0000 7f1883e00640 1 -- 172.21.15.97:0/150867568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f187c0012f0 con 0x7f186c005b80 2024-01-31T05:03:16.935 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f186406aed0 msgr2=0x7f186406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f186406aed0 0x7f186406d390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1874006fd0 tx=0x7f1874008040 comp rx=0 tx=0).stop 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 msgr2=0x7f188c0ac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f188c0ac730 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f187c001e30 tx=0x7f187c02e070 comp rx=0 tx=0).stop 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 shutdown_connections 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f188c157450 0x7f188c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f188c09c1d0 0x7f188c09c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f186406aed0 0x7f186406d390 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 --2- 172.21.15.97:0/150867568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c005b80 0x7f188c0ac730 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:16.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 >> 172.21.15.97:0/150867568 conn(0x7f188c0b0a10 msgr2=0x7f188c0ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 shutdown_connections 2024-01-31T05:03:16.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:16.938+0000 7f1891e00640 1 -- 172.21.15.97:0/150867568 wait complete. 2024-01-31T05:03:16.957 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:17.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.173+0000 7f9d9cc00640 1 Processor -- start 2024-01-31T05:03:17.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.173+0000 7f9d9cc00640 1 -- start start 2024-01-31T05:03:17.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d9cc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d9815a350 0x7f9d980abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d9cc00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9815abc0 con 0x7f9d9815a730 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d9cc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9815aea0 con 0x7f9d980cde10 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d9cc00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d980b2c90 con 0x7f9d9815a350 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d95e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d9815a350 0x7f9d980abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d95e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9d9815a350 0x7f9d980abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60532/0 (socket says 172.21.15.97:60532) 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d95e00640 1 -- 172.21.15.97:0/4095444753 learned_addr learned my addr 172.21.15.97:0/4095444753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.175+0000 7f9d8fe00640 1 --1- 172.21.15.97:0/4095444753 >> v1:172.21.15.97:6789/0 conn(0x7f9d9815a730 0x7f9d9806da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49544/0 (socket says 172.21.15.97:49544) 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.176+0000 7f9d95e00640 1 -- 172.21.15.97:0/4095444753 >> v1:172.21.15.188:6789/0 conn(0x7f9d980cde10 legacy=0x7f9d980cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:17.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.176+0000 7f9d95e00640 1 -- 172.21.15.97:0/4095444753 >> v1:172.21.15.97:6789/0 conn(0x7f9d9815a730 legacy=0x7f9d9806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.176+0000 7f9d95e00640 1 -- 172.21.15.97:0/4095444753 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9815b180 con 0x7f9d9815a350 2024-01-31T05:03:17.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.177+0000 7f9d95e00640 1 --2- 172.21.15.97:0/4095444753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9815a350 0x7f9d980abcf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009850 tx=0x7f9d8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed1cf2045a3e229 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:17.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.177+0000 7f9d95400640 1 -- 172.21.15.97:0/4095444753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88004230 con 0x7f9d9815a350 2024-01-31T05:03:17.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.177+0000 7f9d95400640 1 -- 172.21.15.97:0/4095444753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d88004390 con 0x7f9d9815a350 2024-01-31T05:03:17.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.177+0000 7f9d95400640 1 -- 172.21.15.97:0/4095444753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88003810 con 0x7f9d9815a350 2024-01-31T05:03:17.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 -- 172.21.15.97:0/4095444753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9815a350 msgr2=0x7f9d980abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/4095444753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9815a350 0x7f9d980abcf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009850 tx=0x7f9d8802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 -- 172.21.15.97:0/4095444753 shutdown_connections 2024-01-31T05:03:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/4095444753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d9815a350 0x7f9d980abcf0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 -- 172.21.15.97:0/4095444753 >> 172.21.15.97:0/4095444753 conn(0x7f9d980b0a10 msgr2=0x7f9d980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 -- 172.21.15.97:0/4095444753 shutdown_connections 2024-01-31T05:03:17.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.178+0000 7f9d9cc00640 1 -- 172.21.15.97:0/4095444753 wait complete. 2024-01-31T05:03:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.179+0000 7f9d9cc00640 1 Processor -- start 2024-01-31T05:03:17.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.180+0000 7f9d9cc00640 1 -- start start 2024-01-31T05:03:17.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d9cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d980cde10 0x7f9d980c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d9cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9815a350 0x7f9d980afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d9cc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 0x7f9d98150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d9cc00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980c8c50 con 0x7f9d980c90e0 2024-01-31T05:03:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d9cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980ca010 con 0x7f9d980cde10 2024-01-31T05:03:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d9cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d980ca2f0 con 0x7f9d9815a350 2024-01-31T05:03:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 0x7f9d98150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 0x7f9d98150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60536/0 (socket says 172.21.15.97:60536) 2024-01-31T05:03:17.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 -- 172.21.15.97:0/118668682 learned_addr learned my addr 172.21.15.97:0/118668682 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:17.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d95e00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d980cde10 0x7f9d980c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 -- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9815a350 msgr2=0x7f9d980afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9815a350 0x7f9d980afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 -- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d980cde10 msgr2=0x7f9d980c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d980cde10 0x7f9d980c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.181+0000 7f9d96800640 1 -- 172.21.15.97:0/118668682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009460 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.182+0000 7f9d96800640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 0x7f9d98150520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c780 tx=0x7f9d9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.182+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90003d80 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.182+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000ce90 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.182+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d980c8600 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.182+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900052b0 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.182+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d980c9d00 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.183+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d980c8e00 con 0x7f9d980c90e0 2024-01-31T05:03:17.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.184+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9d90005410 con 0x7f9d980c90e0 2024-01-31T05:03:17.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.185+0000 7f9d8e000640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9d7006aed0 0x7f9d7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.185+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f9d900891d0 con 0x7f9d980c90e0 2024-01-31T05:03:17.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.185+0000 7f9d95e00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9d7006aed0 0x7f9d7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.186+0000 7f9d95e00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9d7006aed0 0x7f9d7006d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9d88035000 tx=0x7f9d88008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:17.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.190+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9d90053d30 con 0x7f9d980c90e0 2024-01-31T05:03:17.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.381+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d98067830 con 0x7f9d980c90e0 2024-01-31T05:03:17.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.382+0000 7f9d8e000640 1 -- 172.21.15.97:0/118668682 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d900170a0 con 0x7f9d980c90e0 2024-01-31T05:03:17.382 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9d7006aed0 msgr2=0x7f9d7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9d7006aed0 0x7f9d7006d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9d88035000 tx=0x7f9d88008030 comp rx=0 tx=0).stop 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 msgr2=0x7f9d98150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 0x7f9d98150520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c780 tx=0x7f9d9000cc40 comp rx=0 tx=0).stop 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 shutdown_connections 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9d7006aed0 0x7f9d7006d390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d980c90e0 0x7f9d98150520 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d9815a350 0x7f9d980afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 --2- 172.21.15.97:0/118668682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d980cde10 0x7f9d980c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 >> 172.21.15.97:0/118668682 conn(0x7f9d980b0a10 msgr2=0x7f9d980ac530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 shutdown_connections 2024-01-31T05:03:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.385+0000 7f9d9cc00640 1 -- 172.21.15.97:0/118668682 wait complete. 2024-01-31T05:03:17.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:03:17.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.619+0000 7f8b22800640 1 Processor -- start 2024-01-31T05:03:17.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.619+0000 7f8b22800640 1 -- start start 2024-01-31T05:03:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b22800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b1c15a350 0x7f8b1c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b22800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c15a730 0x7f8b1c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b22800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b1c15abc0 con 0x7f8b1c06dfc0 2024-01-31T05:03:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b22800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c0b2c90 con 0x7f8b1c15a350 2024-01-31T05:03:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b22800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15aea0 con 0x7f8b1c15a730 2024-01-31T05:03:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b1b400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b1c15a350 0x7f8b1c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b1b400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b1c15a350 0x7f8b1c0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:57258/0 (socket says 172.21.15.97:57258) 2024-01-31T05:03:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b1b400640 1 -- 172.21.15.97:0/458484300 learned_addr learned my addr 172.21.15.97:0/458484300 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.621+0000 7f8b1aa00640 1 --2- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c15a730 0x7f8b1c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.622+0000 7f8b1b400640 1 -- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c15a730 msgr2=0x7f8b1c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.622+0000 7f8b1b400640 1 --2- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c15a730 0x7f8b1c06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.622+0000 7f8b1b400640 1 -- 172.21.15.97:0/458484300 >> v1:172.21.15.188:6789/0 conn(0x7f8b1c06dfc0 legacy=0x7f8b1c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.622+0000 7f8b1b400640 1 -- 172.21.15.97:0/458484300 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c15b180 con 0x7f8b1c15a350 2024-01-31T05:03:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.624+0000 7f8b1b400640 1 --2- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c0abcf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009850 tx=0x7f8b0c02db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0506a191d8d3edd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.624+0000 7f8b1a000640 1 -- 172.21.15.97:0/458484300 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c004230 con 0x7f8b1c15a350 2024-01-31T05:03:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.624+0000 7f8b1a000640 1 -- 172.21.15.97:0/458484300 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c004390 con 0x7f8b1c15a350 2024-01-31T05:03:17.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.624+0000 7f8b1a000640 1 -- 172.21.15.97:0/458484300 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c02e920 con 0x7f8b1c15a350 2024-01-31T05:03:17.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 -- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 msgr2=0x7f8b1c0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 --2- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c0abcf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009850 tx=0x7f8b0c02db70 comp rx=0 tx=0).stop 2024-01-31T05:03:17.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 -- 172.21.15.97:0/458484300 shutdown_connections 2024-01-31T05:03:17.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 --2- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c15a730 0x7f8b1c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 --2- 172.21.15.97:0/458484300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c0abcf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 -- 172.21.15.97:0/458484300 >> 172.21.15.97:0/458484300 conn(0x7f8b1c0b0a10 msgr2=0x7f8b1c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:17.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 -- 172.21.15.97:0/458484300 shutdown_connections 2024-01-31T05:03:17.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.625+0000 7f8b22800640 1 -- 172.21.15.97:0/458484300 wait complete. 2024-01-31T05:03:17.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.627+0000 7f8b22800640 1 Processor -- start 2024-01-31T05:03:17.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.627+0000 7f8b22800640 1 -- start start 2024-01-31T05:03:17.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.627+0000 7f8b22800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c06dfc0 0x7f8b1c1f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b22800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b22800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 0x7f8b1c0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b22800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c09c420 con 0x7f8b1c15d750 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b22800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c15b9d0 con 0x7f8b1c15a350 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b22800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c1f7fe0 con 0x7f8b1c06dfc0 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 0x7f8b1c0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 0x7f8b1c0999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60550/0 (socket says 172.21.15.97:60550) 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 -- 172.21.15.97:0/3054615914 learned_addr learned my addr 172.21.15.97:0/3054615914 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1b400640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c06dfc0 0x7f8b1c1f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1aa00640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 -- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c06dfc0 msgr2=0x7f8b1c1f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c06dfc0 0x7f8b1c1f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 -- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 msgr2=0x7f8b1c157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.628+0000 7f8b1be00640 1 -- 172.21.15.97:0/3054615914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c009460 con 0x7f8b1c15d750 2024-01-31T05:03:17.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.629+0000 7f8b1be00640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 0x7f8b1c0999c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c710 tx=0x7f8b1400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.629+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14006d50 con 0x7f8b1c15d750 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.629+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400ce20 con 0x7f8b1c15d750 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.629+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c1f82c0 con 0x7f8b1c15d750 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.629+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14003bb0 con 0x7f8b1c15d750 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.629+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c15bc70 con 0x7f8b1c15d750 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.630+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c15be90 con 0x7f8b1c15d750 2024-01-31T05:03:17.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.631+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8b14003d10 con 0x7f8b1c15d750 2024-01-31T05:03:17.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.632+0000 7f8b13400640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8af406af00 0x7f8af406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.632+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f8b14089430 con 0x7f8b1c15d750 2024-01-31T05:03:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.632+0000 7f8b1b400640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8af406af00 0x7f8af406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:17.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.633+0000 7f8b1b400640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8af406af00 0x7f8af406d3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c037000 tx=0x7f8b0c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:17.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.637+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8b14057b50 con 0x7f8b1c15d750 2024-01-31T05:03:17.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.829+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f8b1c067810 con 0x7f8b1c15d750 2024-01-31T05:03:17.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.831+0000 7f8b13400640 1 -- 172.21.15.97:0/3054615914 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7f8b140576e0 con 0x7f8b1c15d750 2024-01-31T05:03:17.831 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.833+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8af406af00 msgr2=0x7f8af406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.833+0000 7f8b22800640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8af406af00 0x7f8af406d3c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c037000 tx=0x7f8b0c032000 comp rx=0 tx=0).stop 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 msgr2=0x7f8b1c0999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 0x7f8b1c0999c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400c710 tx=0x7f8b1400cbd0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 shutdown_connections 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8af406af00 0x7f8af406d3c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b1c15d750 0x7f8b1c0999c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1c15a350 0x7f8b1c157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 --2- 172.21.15.97:0/3054615914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b1c06dfc0 0x7f8b1c1f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 >> 172.21.15.97:0/3054615914 conn(0x7f8b1c0b0a10 msgr2=0x7f8b1c0ac2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:17.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 shutdown_connections 2024-01-31T05:03:17.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:17.834+0000 7f8b22800640 1 -- 172.21.15.97:0/3054615914 wait complete. 2024-01-31T05:03:17.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:03:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.062+0000 7f711d200640 1 Processor -- start 2024-01-31T05:03:18.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.063+0000 7f711d200640 1 -- start start 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f711d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711806d580 0x7f711806d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f711d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7118161020 0x7f71181614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f711d200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180ceb40 con 0x7f711815d690 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f711d200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711815e7c0 con 0x7f7118161020 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f711d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711806dea0 con 0x7f711806d580 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f710fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f711815d690 0x7f71180ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49548/0 (socket says 172.21.15.97:49548) 2024-01-31T05:03:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.064+0000 7f710fe00640 1 -- 172.21.15.97:0/3435991631 learned_addr learned my addr 172.21.15.97:0/3435991631 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7116800640 1 --2- 172.21.15.97:0/3435991631 >> v2:172.21.15.188:3300/0 conn(0x7f7118161020 0x7f71181614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335654815 0 0) 0x7f71180ceb40 con 0x7f711815d690 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115e00640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711806d580 0x7f711806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70f8003520 con 0x7f711815d690 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3094621636 0 0) 0x7f70f8003520 con 0x7f711815d690 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711806d580 msgr2=0x7f711806d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711806d580 0x7f711806d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 >> v2:172.21.15.188:3300/0 conn(0x7f7118161020 msgr2=0x7f71181614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 --2- 172.21.15.97:0/3435991631 >> v2:172.21.15.188:3300/0 conn(0x7f7118161020 0x7f71181614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711815db40 con 0x7f711815d690 2024-01-31T05:03:18.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f7100001670 con 0x7f711815d690 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 >> v1:172.21.15.97:6789/0 conn(0x7f711815d690 legacy=0x7f71180ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f70f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70f80047c0 0x7f70f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70f8005180 0x7f70f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711815e7c0 con 0x7f70f8003970 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711806dea0 con 0x7f70f8005180 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.065+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f800bf40 con 0x7f70f80047c0 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711806d580 0x7f711806d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f70f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7116800640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70f8005180 0x7f70f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 -- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70f80047c0 msgr2=0x7f70f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70f80047c0 0x7f70f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 -- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70f8005180 msgr2=0x7f70f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:18.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70f8005180 0x7f70f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.066+0000 7f7115e00640 1 -- 172.21.15.97:0/3435991631 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f8003520 con 0x7f70f8003970 2024-01-31T05:03:18.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.067+0000 7f7115e00640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f70f8003d90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7118072e60 tx=0x7f70f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4e112d956175f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:18.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.067+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108004230 con 0x7f70f8003970 2024-01-31T05:03:18.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.067+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710802e050 con 0x7f70f8003970 2024-01-31T05:03:18.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.067+0000 7f7115400640 1 -- 172.21.15.97:0/3435991631 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108003610 con 0x7f70f8003970 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f711d200640 1 -- 172.21.15.97:0/3435991631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 msgr2=0x7f70f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f711d200640 1 --2- 172.21.15.97:0/3435991631 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f70f8003d90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7118072e60 tx=0x7f70f80057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f7116800640 1 -- 172.21.15.97:0/3435991631 reap_dead start 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f711d200640 1 -- 172.21.15.97:0/3435991631 shutdown_connections 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f711d200640 1 -- 172.21.15.97:0/3435991631 >> 172.21.15.97:0/3435991631 conn(0x7f71180b0a10 msgr2=0x7f71180b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f711d200640 1 -- 172.21.15.97:0/3435991631 shutdown_connections 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.068+0000 7f711d200640 1 -- 172.21.15.97:0/3435991631 wait complete. 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.070+0000 7f711d200640 1 Processor -- start 2024-01-31T05:03:18.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.070+0000 7f711d200640 1 -- start start 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f711d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f71181566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f711d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71180713b0 0x7f7118071830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f711d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711809a520 0x7f711809a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f711d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711815b9e0 con 0x7f70f8003970 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f711d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118071d70 con 0x7f71180713b0 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f711d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7118156bf0 con 0x7f711809a520 2024-01-31T05:03:18.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f7115e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f71181566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f7115e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f71181566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60572/0 (socket says 172.21.15.97:60572) 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.071+0000 7f7115e00640 1 -- 172.21.15.97:0/2047988727 learned_addr learned my addr 172.21.15.97:0/2047988727 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f7115e00640 1 -- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711809a520 msgr2=0x7f711809a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f7115e00640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711809a520 0x7f711809a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f7115e00640 1 -- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71180713b0 msgr2=0x7f7118071830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f710fe00640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71180713b0 0x7f7118071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f7115e00640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71180713b0 0x7f7118071830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f7115e00640 1 -- 172.21.15.97:0/2047988727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711809b1c0 con 0x7f70f8003970 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f7115e00640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f71181566b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7108009760 tx=0x7f7108004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108005b90 con 0x7f70f8003970 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108005cf0 con 0x7f70f8003970 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108006260 con 0x7f70f8003970 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711809b4a0 con 0x7f70f8003970 2024-01-31T05:03:18.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.072+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711809ba60 con 0x7f70f8003970 2024-01-31T05:03:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.074+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118067830 con 0x7f70f8003970 2024-01-31T05:03:18.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.075+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f71080063c0 con 0x7f70f8003970 2024-01-31T05:03:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.075+0000 7f710e000640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f70f006aed0 0x7f70f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.076+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f71080ab270 con 0x7f70f8003970 2024-01-31T05:03:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.076+0000 7f710fe00640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f70f006aed0 0x7f70f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.076+0000 7f710fe00640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f70f006aed0 0x7f70f006d390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7100000c00 tx=0x7f7100009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:18.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.080+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7108075e80 con 0x7f70f8003970 2024-01-31T05:03:18.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.286+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f711809cec0 con 0x7f70f8003970 2024-01-31T05:03:18.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.287+0000 7f710e000640 1 -- 172.21.15.97:0/2047988727 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7f710800f040 con 0x7f70f8003970 2024-01-31T05:03:18.286 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:03:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.289+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f70f006aed0 msgr2=0x7f70f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.289+0000 7f711d200640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f70f006aed0 0x7f70f006d390 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7100000c00 tx=0x7f7100009040 comp rx=0 tx=0).stop 2024-01-31T05:03:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 msgr2=0x7f71181566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f71181566b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7108009760 tx=0x7f7108004730 comp rx=0 tx=0).stop 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 shutdown_connections 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f711809a520 0x7f711809a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71180713b0 0x7f7118071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f70f006aed0 0x7f70f006d390 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 --2- 172.21.15.97:0/2047988727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70f8003970 0x7f71181566b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 >> 172.21.15.97:0/2047988727 conn(0x7f71180b0a10 msgr2=0x7f71180aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 shutdown_connections 2024-01-31T05:03:18.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.290+0000 7f711d200640 1 -- 172.21.15.97:0/2047988727 wait complete. 2024-01-31T05:03:18.307 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:03:18.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:18.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:18.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[55101]: fuse finished with error 0 and tester_r 0 2024-01-31T05:03:18.401 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:03:18.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:03:18.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:18.476 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:03:18.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-01-31T05:03:18.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.754+0000 7f6c28c00640 1 Processor -- start 2024-01-31T05:03:18.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.754+0000 7f6c28c00640 1 -- start start 2024-01-31T05:03:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c28c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c240cfae0 0x7f6c240cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c28c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c240d0de0 0x7f6c240d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c28c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2415ec50 con 0x7f6c2415d590 2024-01-31T05:03:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c28c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240d0460 con 0x7f6c240d0de0 2024-01-31T05:03:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c28c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cded0 con 0x7f6c240cfae0 2024-01-31T05:03:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c22800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c240d0de0 0x7f6c240d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c22800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c240d0de0 0x7f6c240d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60576/0 (socket says 172.21.15.97:60576) 2024-01-31T05:03:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c22800640 1 -- 172.21.15.97:0/2499239688 learned_addr learned my addr 172.21.15.97:0/2499239688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.756+0000 7f6c21400640 1 --2- 172.21.15.97:0/2499239688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c240cfae0 0x7f6c240cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.757+0000 7f6c22800640 1 -- 172.21.15.97:0/2499239688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c240cfae0 msgr2=0x7f6c240cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.757+0000 7f6c22800640 1 --2- 172.21.15.97:0/2499239688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c240cfae0 0x7f6c240cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.757+0000 7f6c22800640 1 -- 172.21.15.97:0/2499239688 >> v1:172.21.15.188:6789/0 conn(0x7f6c2415d590 legacy=0x7f6c2415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.757+0000 7f6c22800640 1 -- 172.21.15.97:0/2499239688 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2415d970 con 0x7f6c240d0de0 2024-01-31T05:03:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.758+0000 7f6c22800640 1 --2- 172.21.15.97:0/2499239688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c240d1270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa19a78a4c3516b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:18.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.758+0000 7f6c20a00640 1 -- 172.21.15.97:0/2499239688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c02f840 con 0x7f6c240d0de0 2024-01-31T05:03:18.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.758+0000 7f6c20a00640 1 -- 172.21.15.97:0/2499239688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c02f9a0 con 0x7f6c240d0de0 2024-01-31T05:03:18.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.758+0000 7f6c20a00640 1 -- 172.21.15.97:0/2499239688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c02fd30 con 0x7f6c240d0de0 2024-01-31T05:03:18.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 -- 172.21.15.97:0/2499239688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 msgr2=0x7f6c240d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 --2- 172.21.15.97:0/2499239688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c240d1270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c02ba20 comp rx=0 tx=0).stop 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 -- 172.21.15.97:0/2499239688 shutdown_connections 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 --2- 172.21.15.97:0/2499239688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c240d1270 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 --2- 172.21.15.97:0/2499239688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c240cfae0 0x7f6c240cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 -- 172.21.15.97:0/2499239688 >> 172.21.15.97:0/2499239688 conn(0x7f6c2409c560 msgr2=0x7f6c240d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 -- 172.21.15.97:0/2499239688 shutdown_connections 2024-01-31T05:03:18.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.759+0000 7f6c28c00640 1 -- 172.21.15.97:0/2499239688 wait complete. 2024-01-31T05:03:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.760+0000 7f6c28c00640 1 Processor -- start 2024-01-31T05:03:18.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.761+0000 7f6c28c00640 1 -- start start 2024-01-31T05:03:18.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.761+0000 7f6c28c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c240cfae0 0x7f6c240c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c28c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c2409ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c28c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415d590 0x7f6c241f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c28c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240c8df0 con 0x7f6c240d0de0 2024-01-31T05:03:18.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c28c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240ccf40 con 0x7f6c240cfae0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c28c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cd1e0 con 0x7f6c2415d590 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c2409ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c2409ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60586/0 (socket says 172.21.15.97:60586) 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 -- 172.21.15.97:0/917057877 learned_addr learned my addr 172.21.15.97:0/917057877 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21e00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c240cfae0 0x7f6c240c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 -- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415d590 msgr2=0x7f6c241f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415d590 0x7f6c241f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 -- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c240cfae0 msgr2=0x7f6c240c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c240cfae0 0x7f6c240c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.762+0000 7f6c21400640 1 -- 172.21.15.97:0/917057877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0094d0 con 0x7f6c240d0de0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.763+0000 7f6c21400640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c2409ced0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c007ec0 tx=0x7f6c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.763+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c014620 con 0x7f6c240d0de0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.763+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0147a0 con 0x7f6c240d0de0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.763+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c014b50 con 0x7f6c240d0de0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.763+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c240cd4c0 con 0x7f6c240d0de0 2024-01-31T05:03:18.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.763+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c240c8a80 con 0x7f6c240d0de0 2024-01-31T05:03:18.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.764+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c240c8f60 con 0x7f6c240d0de0 2024-01-31T05:03:18.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.765+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6c0c014cb0 con 0x7f6c240d0de0 2024-01-31T05:03:18.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.766+0000 7f6c1a000640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6bfc06aed0 0x7f6bfc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:18.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.767+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f6c0c08a3b0 con 0x7f6c240d0de0 2024-01-31T05:03:18.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.767+0000 7f6c21e00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6bfc06aed0 0x7f6bfc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:18.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.767+0000 7f6c21e00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6bfc06aed0 0x7f6bfc06d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6c14001500 tx=0x7f6c14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:18.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:18.771+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6c0c054fb0 con 0x7f6c240d0de0 2024-01-31T05:03:19.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.032+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6c24067830 con 0x7f6c240d0de0 2024-01-31T05:03:19.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.037+0000 7f6c1a000640 1 -- 172.21.15.97:0/917057877 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v14) v1 ==== 184+0+141 (secure 0 0 0) 0x7f6c0c0585d0 con 0x7f6c240d0de0 2024-01-31T05:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-01-31T05:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:03:19.037 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw pool=cephfs_data" 2024-01-31T05:03:19.038 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:03:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.040+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6bfc06aed0 msgr2=0x7f6bfc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:19.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.040+0000 7f6c28c00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6bfc06aed0 0x7f6bfc06d390 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6c14001500 tx=0x7f6c14009040 comp rx=0 tx=0).stop 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.040+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 msgr2=0x7f6c2409ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.040+0000 7f6c28c00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c2409ced0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c007ec0 tx=0x7f6c0c00e4b0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 shutdown_connections 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c2415d590 0x7f6c241f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6bfc06aed0 0x7f6bfc06d390 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c240d0de0 0x7f6c2409ced0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 --2- 172.21.15.97:0/917057877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c240cfae0 0x7f6c240c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 >> 172.21.15.97:0/917057877 conn(0x7f6c2409c560 msgr2=0x7f6c2406ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 shutdown_connections 2024-01-31T05:03:19.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.041+0000 7f6c28c00640 1 -- 172.21.15.97:0/917057877 wait complete. 2024-01-31T05:03:19.057 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-31T05:03:19.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:03:19.160 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:03:19.277 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-31T05:03:19.277 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:19.277 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:19.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:03:19.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:03:19.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:19.333 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:03:19.386 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-01-31T05:03:19.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:03:19.387 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:03:19.387 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:19.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:03:19.444 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:03:19.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:19.444 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-31T05:03:19.499 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-01-31T05:03:19.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:03:19.500 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:19.500 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:19.558 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:03:19.558 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:03:19.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:19.558 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-31T05:03:19.611 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-31T05:03:19.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:03:19.612 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:03:19.613 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:19.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:03:19.669 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:03:19.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:19.669 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-31T05:03:19.723 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-31T05:03:19.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:03:19.725 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.943+0000 7fd6ed800640 1 Processor -- start 2024-01-31T05:03:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.943+0000 7fd6ed800640 1 -- start start 2024-01-31T05:03:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6ed800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6ed800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd6e80ce430 0x7fd6e8160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6ed800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd6e8161500 0x7fd6e80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6ed800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e8161990 con 0x7fd6e80ce430 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6ed800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e815d900 con 0x7fd6e8161500 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6ed800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e80abcf0 con 0x7fd6e815d520 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6e6000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd6e80ce430 0x7fd6e8160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6e6000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd6e80ce430 0x7fd6e8160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60596/0 (socket says 172.21.15.97:60596) 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.945+0000 7fd6e6000640 1 -- 172.21.15.97:0/3021754411 learned_addr learned my addr 172.21.15.97:0/3021754411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6000640 1 -- 172.21.15.97:0/3021754411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 msgr2=0x7fd6e80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e7400640 1 --2- 172.21.15.97:0/3021754411 >> v2:172.21.15.188:3300/0 conn(0x7fd6e8161500 0x7fd6e80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6a00640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6000640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6000640 1 -- 172.21.15.97:0/3021754411 >> v2:172.21.15.188:3300/0 conn(0x7fd6e8161500 msgr2=0x7fd6e80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6000640 1 --2- 172.21.15.97:0/3021754411 >> v2:172.21.15.188:3300/0 conn(0x7fd6e8161500 0x7fd6e80b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6000640 1 -- 172.21.15.97:0/3021754411 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e806ca40 con 0x7fd6e80ce430 2024-01-31T05:03:19.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.946+0000 7fd6e6a00640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.947+0000 7fd6e6000640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e80ce430 0x7fd6e8160fc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd6e806e650 tx=0x7fd6d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddf6aadb62eb040 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.947+0000 7fd6e5600640 1 -- 172.21.15.97:0/3021754411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0031640 con 0x7fd6e80ce430 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.947+0000 7fd6e5600640 1 -- 172.21.15.97:0/3021754411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d00317a0 con 0x7fd6e80ce430 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.947+0000 7fd6e5600640 1 -- 172.21.15.97:0/3021754411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0031b30 con 0x7fd6e80ce430 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 -- 172.21.15.97:0/3021754411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e80ce430 msgr2=0x7fd6e8160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e80ce430 0x7fd6e8160fc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd6e806e650 tx=0x7fd6d002b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 -- 172.21.15.97:0/3021754411 shutdown_connections 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 --2- 172.21.15.97:0/3021754411 >> v2:172.21.15.188:3300/0 conn(0x7fd6e8161500 0x7fd6e80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e80ce430 0x7fd6e8160fc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 --2- 172.21.15.97:0/3021754411 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 -- 172.21.15.97:0/3021754411 >> 172.21.15.97:0/3021754411 conn(0x7fd6e80b0a10 msgr2=0x7fd6e80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 -- 172.21.15.97:0/3021754411 shutdown_connections 2024-01-31T05:03:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.948+0000 7fd6ed800640 1 -- 172.21.15.97:0/3021754411 wait complete. 2024-01-31T05:03:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.950+0000 7fd6ed800640 1 Processor -- start 2024-01-31T05:03:19.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.950+0000 7fd6ed800640 1 -- start start 2024-01-31T05:03:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.951+0000 7fd6ed800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e80ce430 0x7fd6e814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.951+0000 7fd6ed800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.951+0000 7fd6ed800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 0x7fd6e8160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.951+0000 7fd6ed800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e814bd90 con 0x7fd6e8161500 2024-01-31T05:03:19.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.951+0000 7fd6ed800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e815f010 con 0x7fd6e80ce430 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.951+0000 7fd6ed800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8099f00 con 0x7fd6e815d520 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 0x7fd6e8160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 0x7fd6e8160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60608/0 (socket says 172.21.15.97:60608) 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 -- 172.21.15.97:0/4260259729 learned_addr learned my addr 172.21.15.97:0/4260259729 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e6a00640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e80ce430 0x7fd6e814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e6000640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 -- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 msgr2=0x7fd6e80999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 -- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e80ce430 msgr2=0x7fd6e814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e80ce430 0x7fd6e814b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 -- 172.21.15.97:0/4260259729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00094d0 con 0x7fd6e8161500 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.952+0000 7fd6e7400640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 0x7fd6e8160300 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000cc60 tx=0x7fd6e000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:19.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.953+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000ce70 con 0x7fd6e8161500 2024-01-31T05:03:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.953+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e0003990 con 0x7fd6e8161500 2024-01-31T05:03:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.953+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0003d20 con 0x7fd6e8161500 2024-01-31T05:03:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.953+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e815f260 con 0x7fd6e8161500 2024-01-31T05:03:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.953+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e815f820 con 0x7fd6e8161500 2024-01-31T05:03:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.954+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e80cc2e0 con 0x7fd6e8161500 2024-01-31T05:03:19.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.955+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd6e0003e80 con 0x7fd6e8161500 2024-01-31T05:03:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.956+0000 7fd6dea00640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd6c006aed0 0x7fd6c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.956+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fd6e0089e10 con 0x7fd6e8161500 2024-01-31T05:03:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.956+0000 7fd6e6a00640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd6c006aed0 0x7fd6c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.957+0000 7fd6e6a00640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd6c006aed0 0x7fd6c006d390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8006b10 tx=0x7fd6d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:19.961+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd6e008d030 con 0x7fd6e8161500 2024-01-31T05:03:20.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.195+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e80b3ee0 con 0x7fd6e8161500 2024-01-31T05:03:20.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.196+0000 7fd6dea00640 1 -- 172.21.15.97:0/4260259729 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fd6e001d030 con 0x7fd6e8161500 2024-01-31T05:03:20.196 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:20.196 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:42.346423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:03:20.196 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd6c006aed0 msgr2=0x7fd6c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd6c006aed0 0x7fd6c006d390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8006b10 tx=0x7fd6d8008040 comp rx=0 tx=0).stop 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 msgr2=0x7fd6e8160300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 0x7fd6e8160300 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000cc60 tx=0x7fd6e000a5c0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 shutdown_connections 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd6c006aed0 0x7fd6c006d390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6e8161500 0x7fd6e8160300 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6e815d520 0x7fd6e80999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 --2- 172.21.15.97:0/4260259729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6e80ce430 0x7fd6e814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 >> 172.21.15.97:0/4260259729 conn(0x7fd6e80b0a10 msgr2=0x7fd6e80aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:20.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 shutdown_connections 2024-01-31T05:03:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.199+0000 7fd6ed800640 1 -- 172.21.15.97:0/4260259729 wait complete. 2024-01-31T05:03:20.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.425+0000 7ff49e600640 1 Processor -- start 2024-01-31T05:03:20.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.425+0000 7ff49e600640 1 -- start start 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff49e600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff498159a50 0x7ff4980caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff49e600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff49815db50 0x7ff49806c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff49e600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980cb650 con 0x7ff4980b1d80 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff49e600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980cb440 con 0x7ff498159a50 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff49e600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4980cddb0 con 0x7ff49815db50 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff497400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff498159a50 0x7ff4980caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff496a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff4980b1d80 0x7ff49815d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49554/0 (socket says 172.21.15.97:49554) 2024-01-31T05:03:20.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.427+0000 7ff496a00640 1 -- 172.21.15.97:0/87205279 learned_addr learned my addr 172.21.15.97:0/87205279 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff496000640 1 -- 172.21.15.97:0/87205279 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036983090 0 0) 0x7ff4980cb650 con 0x7ff4980b1d80 2024-01-31T05:03:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff497e00640 1 --2- 172.21.15.97:0/87205279 >> v2:172.21.15.188:3300/0 conn(0x7ff49815db50 0x7ff49806c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff496000640 1 -- 172.21.15.97:0/87205279 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff478005730 con 0x7ff4980b1d80 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff497400640 1 -- 172.21.15.97:0/87205279 >> v2:172.21.15.188:3300/0 conn(0x7ff49815db50 msgr2=0x7ff49806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff497400640 1 --2- 172.21.15.97:0/87205279 >> v2:172.21.15.188:3300/0 conn(0x7ff49815db50 0x7ff49806c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff497400640 1 -- 172.21.15.97:0/87205279 >> v1:172.21.15.97:6789/0 conn(0x7ff4980b1d80 legacy=0x7ff49815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff497400640 1 -- 172.21.15.97:0/87205279 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980ce090 con 0x7ff498159a50 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.428+0000 7ff497e00640 1 --2- 172.21.15.97:0/87205279 >> v2:172.21.15.188:3300/0 conn(0x7ff49815db50 0x7ff49806c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:20.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.429+0000 7ff497400640 1 --2- 172.21.15.97:0/87205279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff498159a50 0x7ff4980caf00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff4980ce9f0 tx=0x7ff48802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e91066f2c3862be server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.429+0000 7ff496000640 1 -- 172.21.15.97:0/87205279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488031830 con 0x7ff498159a50 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff496000640 1 -- 172.21.15.97:0/87205279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff488031990 con 0x7ff498159a50 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff496000640 1 -- 172.21.15.97:0/87205279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488031d20 con 0x7ff498159a50 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 -- 172.21.15.97:0/87205279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff498159a50 msgr2=0x7ff4980caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 --2- 172.21.15.97:0/87205279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff498159a50 0x7ff4980caf00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff4980ce9f0 tx=0x7ff48802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 -- 172.21.15.97:0/87205279 shutdown_connections 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 --2- 172.21.15.97:0/87205279 >> v2:172.21.15.188:3300/0 conn(0x7ff49815db50 0x7ff49806c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 --2- 172.21.15.97:0/87205279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff498159a50 0x7ff4980caf00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 -- 172.21.15.97:0/87205279 >> 172.21.15.97:0/87205279 conn(0x7ff4980b0a10 msgr2=0x7ff4980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 -- 172.21.15.97:0/87205279 shutdown_connections 2024-01-31T05:03:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.430+0000 7ff49e600640 1 -- 172.21.15.97:0/87205279 wait complete. 2024-01-31T05:03:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.432+0000 7ff49e600640 1 Processor -- start 2024-01-31T05:03:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.432+0000 7ff49e600640 1 -- start start 2024-01-31T05:03:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff49e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4980b1d80 0x7ff4980acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff49e600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff498159a50 0x7ff49809b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff49e600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 0x7ff49815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff49e600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498099d30 con 0x7ff49815db50 2024-01-31T05:03:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff49e600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49809a010 con 0x7ff498159a50 2024-01-31T05:03:20.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff49e600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49809a2f0 con 0x7ff4980b1d80 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff497e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 0x7ff49815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff497e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 0x7ff49815c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60626/0 (socket says 172.21.15.97:60626) 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.433+0000 7ff497e00640 1 -- 172.21.15.97:0/912019386 learned_addr learned my addr 172.21.15.97:0/912019386 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497400640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4980b1d80 0x7ff4980acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff496a00640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff498159a50 0x7ff49809b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497e00640 1 -- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4980b1d80 msgr2=0x7ff4980acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497e00640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4980b1d80 0x7ff4980acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497e00640 1 -- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff498159a50 msgr2=0x7ff49809b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497e00640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff498159a50 0x7ff49809b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497e00640 1 -- 172.21.15.97:0/912019386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009400 con 0x7ff49815db50 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff496a00640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff498159a50 0x7ff49809b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff497e00640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 0x7ff49815c3f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c7c0 tx=0x7ff49000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490004230 con 0x7ff49815db50 2024-01-31T05:03:20.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490004390 con 0x7ff49815db50 2024-01-31T05:03:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490007860 con 0x7ff49815db50 2024-01-31T05:03:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.434+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49809a5d0 con 0x7ff49815db50 2024-01-31T05:03:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.435+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4980999c0 con 0x7ff49815db50 2024-01-31T05:03:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.436+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980677c0 con 0x7ff49815db50 2024-01-31T05:03:20.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.437+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff4900079c0 con 0x7ff49815db50 2024-01-31T05:03:20.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.438+0000 7ff48f400640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff47006af00 0x7ff47006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.438+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7ff490089be0 con 0x7ff49815db50 2024-01-31T05:03:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.438+0000 7ff497400640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff47006af00 0x7ff47006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.439+0000 7ff497400640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff47006af00 0x7ff47006d3c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff488036000 tx=0x7ff48803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:20.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.442+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff490054800 con 0x7ff49815db50 2024-01-31T05:03:20.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.635+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff49809ca80 con 0x7ff49815db50 2024-01-31T05:03:20.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.638+0000 7ff48f400640 1 -- 172.21.15.97:0/912019386 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7ff490015090 con 0x7ff49815db50 2024-01-31T05:03:20.638 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:20.638 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.640+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff47006af00 msgr2=0x7ff47006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.640+0000 7ff49e600640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff47006af00 0x7ff47006d3c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff488036000 tx=0x7ff48803c040 comp rx=0 tx=0).stop 2024-01-31T05:03:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.640+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 msgr2=0x7ff49815c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.640+0000 7ff49e600640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 0x7ff49815c3f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c7c0 tx=0x7ff49000cc80 comp rx=0 tx=0).stop 2024-01-31T05:03:20.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 shutdown_connections 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff47006af00 0x7ff47006d3c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49815db50 0x7ff49815c3f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff498159a50 0x7ff49809b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 --2- 172.21.15.97:0/912019386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4980b1d80 0x7ff4980acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 >> 172.21.15.97:0/912019386 conn(0x7ff4980b0a10 msgr2=0x7ff49806c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 shutdown_connections 2024-01-31T05:03:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.641+0000 7ff49e600640 1 -- 172.21.15.97:0/912019386 wait complete. 2024-01-31T05:03:20.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:02:29.547540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:02:30.723512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-01-31T05:02:41.538409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:20.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:20.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.870+0000 7f1618c00640 1 Processor -- start 2024-01-31T05:03:20.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.870+0000 7f1618c00640 1 -- start start 2024-01-31T05:03:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1618c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 0x7f16140722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1618c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16140abcf0 0x7f16140ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1618c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161406cd20 con 0x7f16140cb810 2024-01-31T05:03:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1618c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16140ce970 con 0x7f16140abcf0 2024-01-31T05:03:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1618c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161415b850 con 0x7f161415b470 2024-01-31T05:03:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1612800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f16140cb810 0x7f16140ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49562/0 (socket says 172.21.15.97:49562) 2024-01-31T05:03:20.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1612800640 1 -- 172.21.15.97:0/1665050970 learned_addr learned my addr 172.21.15.97:0/1665050970 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1611400640 1 --2- 172.21.15.97:0/1665050970 >> v2:172.21.15.188:3300/0 conn(0x7f16140abcf0 0x7f16140ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1611e00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 0x7f16140722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.872+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051889099 0 0) 0x7f161406cd20 con 0x7f16140cb810 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f4003520 con 0x7f16140cb810 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1570447848 0 0) 0x7f15f4003520 con 0x7f16140cb810 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 msgr2=0x7f16140722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 0x7f16140722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 >> v2:172.21.15.188:3300/0 conn(0x7f16140abcf0 msgr2=0x7f16140ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:20.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 --2- 172.21.15.97:0/1665050970 >> v2:172.21.15.188:3300/0 conn(0x7f16140abcf0 0x7f16140ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140ceb60 con 0x7f16140cb810 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f160c0019f0 con 0x7f16140cb810 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 >> v1:172.21.15.97:6789/0 conn(0x7f16140cb810 legacy=0x7f16140ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f15f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f40047c0 0x7f15f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 0x7f15f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161415b850 con 0x7f15f40039a0 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f400b980 con 0x7f15f40047c0 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f400bb40 con 0x7f161415b470 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1611400640 1 --2- 172.21.15.97:0/1665050970 >> v2:172.21.15.188:3300/0 conn(0x7f16140abcf0 0x7f16140ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.873+0000 7f1611e00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f15f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1612800640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 0x7f15f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1611e00640 1 -- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 msgr2=0x7f15f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:20.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1611e00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f161415b470 0x7f15f4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1611e00640 1 -- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f40047c0 msgr2=0x7f15f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:20.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1611400640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f40047c0 0x7f15f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1611e00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15f40047c0 0x7f15f4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.874+0000 7f1611e00640 1 -- 172.21.15.97:0/1665050970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4003520 con 0x7f15f40039a0 2024-01-31T05:03:20.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.875+0000 7f1611e00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f15f4003dc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1604009ab0 tx=0x7f15f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c81e8b537a38a74 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.875+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604005990 con 0x7f15f40039a0 2024-01-31T05:03:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.875+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1604005af0 con 0x7f15f40039a0 2024-01-31T05:03:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.875+0000 7f1610a00640 1 -- 172.21.15.97:0/1665050970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604005ea0 con 0x7f15f40039a0 2024-01-31T05:03:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1618c00640 1 -- 172.21.15.97:0/1665050970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 msgr2=0x7f15f4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1618c00640 1 --2- 172.21.15.97:0/1665050970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f15f4003dc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1604009ab0 tx=0x7f15f40057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1612800640 1 -- 172.21.15.97:0/1665050970 reap_dead start 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1618c00640 1 -- 172.21.15.97:0/1665050970 shutdown_connections 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1618c00640 1 -- 172.21.15.97:0/1665050970 >> 172.21.15.97:0/1665050970 conn(0x7f16140b0a10 msgr2=0x7f16140b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1618c00640 1 -- 172.21.15.97:0/1665050970 shutdown_connections 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.876+0000 7f1618c00640 1 -- 172.21.15.97:0/1665050970 wait complete. 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.878+0000 7f1618c00640 1 Processor -- start 2024-01-31T05:03:20.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.878+0000 7f1618c00640 1 -- start start 2024-01-31T05:03:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1618c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f161413dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1618c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161415de00 0x7f161415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1618c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614150b70 0x7f1614150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1618c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1614152060 con 0x7f15f40039a0 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1618c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161415e7c0 con 0x7f161415de00 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1618c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1614151530 con 0x7f1614150b70 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f161413dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f161413dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60646/0 (socket says 172.21.15.97:60646) 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 -- 172.21.15.97:0/3940919003 learned_addr learned my addr 172.21.15.97:0/3940919003 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1612800640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614150b70 0x7f1614150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 -- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614150b70 msgr2=0x7f1614150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614150b70 0x7f1614150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 -- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161415de00 msgr2=0x7f161415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.879+0000 7f1611e00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161415de00 0x7f161415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f1611e00640 1 -- 172.21.15.97:0/3940919003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16141517c0 con 0x7f15f40039a0 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f1612800640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614150b70 0x7f1614150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f1611e00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f161413dd50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f160402c320 tx=0x7f1604004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604006030 con 0x7f15f40039a0 2024-01-31T05:03:20.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16040061b0 con 0x7f15f40039a0 2024-01-31T05:03:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604006560 con 0x7f15f40039a0 2024-01-31T05:03:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1614151aa0 con 0x7f15f40039a0 2024-01-31T05:03:20.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.880+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161413e7a0 con 0x7f15f40039a0 2024-01-31T05:03:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.882+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161413e600 con 0x7f15f40039a0 2024-01-31T05:03:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.882+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f16040066c0 con 0x7f15f40039a0 2024-01-31T05:03:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.883+0000 7f160a000640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f15ec06aed0 0x7f15ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.883+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f16040abdc0 con 0x7f15f40039a0 2024-01-31T05:03:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.883+0000 7f1611400640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f15ec06aed0 0x7f15ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:20.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.884+0000 7f1611400640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f15ec06aed0 0x7f15ec06d390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15fc006e80 tx=0x7f15fc006e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:20.888+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f161413e600 con 0x7f15f40039a0 2024-01-31T05:03:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.082+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f161413e600 con 0x7f15f40039a0 2024-01-31T05:03:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.083+0000 7f160a000640 1 -- 172.21.15.97:0/3940919003 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f161413e600 con 0x7f15f40039a0 2024-01-31T05:03:21.082 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f15ec06aed0 msgr2=0x7f15ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f15ec06aed0 0x7f15ec06d390 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15fc006e80 tx=0x7f15fc006e10 comp rx=0 tx=0).stop 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 msgr2=0x7f161413dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f161413dd50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f160402c320 tx=0x7f1604004730 comp rx=0 tx=0).stop 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 shutdown_connections 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614150b70 0x7f1614150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f161415de00 0x7f161415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f15ec06aed0 0x7f15ec06d390 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 --2- 172.21.15.97:0/3940919003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15f40039a0 0x7f161413dd50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 >> 172.21.15.97:0/3940919003 conn(0x7f16140b0a10 msgr2=0x7f161406d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 shutdown_connections 2024-01-31T05:03:21.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.085+0000 7f1618c00640 1 -- 172.21.15.97:0/3940919003 wait complete. 2024-01-31T05:03:21.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:03:21.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.317+0000 7fa995800640 1 Processor -- start 2024-01-31T05:03:21.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.318+0000 7fa995800640 1 -- start start 2024-01-31T05:03:21.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.319+0000 7fa995800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 0x7fa9900cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.319+0000 7fa995800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99015ec50 con 0x7fa9900d0de0 2024-01-31T05:03:21.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.319+0000 7fa995800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9900d0460 con 0x7fa99015d590 2024-01-31T05:03:21.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.319+0000 7fa995800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cded0 con 0x7fa9900cfae0 2024-01-31T05:03:21.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98f400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa9900d0de0 0x7fa9900d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49574/0 (socket says 172.21.15.97:49574) 2024-01-31T05:03:21.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98f400640 1 -- 172.21.15.97:0/1443377260 learned_addr learned my addr 172.21.15.97:0/1443377260 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98e000640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 0x7fa9900cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170290992 0 0) 0x7fa99015ec50 con 0x7fa9900d0de0 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa970003520 con 0x7fa9900d0de0 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3720732189 0 0) 0x7fa970003520 con 0x7fa9900d0de0 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 msgr2=0x7fa9900cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 0x7fa9900cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 >> v1:172.21.15.188:6789/0 conn(0x7fa99015d590 legacy=0x7fa99015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99015d970 con 0x7fa9900d0de0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa988001670 con 0x7fa9900d0de0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.320+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 >> v1:172.21.15.97:6789/0 conn(0x7fa9900d0de0 legacy=0x7fa9900d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98d600640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9700039a0 0x7fa970003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98d600640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9700047c0 0x7fa970004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98d600640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 0x7fa970005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900cded0 con 0x7fa9700047c0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97000bac0 con 0x7fa9700039a0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97000bc80 con 0x7fa9900cfae0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98e000640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9700047c0 0x7fa970004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98ea00640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9700039a0 0x7fa970003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98e000640 1 -- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 msgr2=0x7fa970005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98e000640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9900cfae0 0x7fa970005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98e000640 1 -- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9700039a0 msgr2=0x7fa970003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98e000640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9700039a0 0x7fa970003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.321+0000 7fa98e000640 1 -- 172.21.15.97:0/1443377260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970003520 con 0x7fa9700047c0 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.322+0000 7fa98e000640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9700047c0 0x7fa970004c40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa980009f90 tx=0x7fa9700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8d0b6a2b9ee96e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98002cbf0 con 0x7fa9700047c0 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000f040 con 0x7fa9700047c0 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa98d600640 1 -- 172.21.15.97:0/1443377260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980034440 con 0x7fa9700047c0 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa995800640 1 -- 172.21.15.97:0/1443377260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9700047c0 msgr2=0x7fa970004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa995800640 1 --2- 172.21.15.97:0/1443377260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9700047c0 0x7fa970004c40 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa980009f90 tx=0x7fa9700057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa98f400640 1 -- 172.21.15.97:0/1443377260 reap_dead start 2024-01-31T05:03:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa995800640 1 -- 172.21.15.97:0/1443377260 shutdown_connections 2024-01-31T05:03:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.323+0000 7fa995800640 1 -- 172.21.15.97:0/1443377260 >> 172.21.15.97:0/1443377260 conn(0x7fa99009c560 msgr2=0x7fa9900d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.324+0000 7fa995800640 1 -- 172.21.15.97:0/1443377260 shutdown_connections 2024-01-31T05:03:21.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.324+0000 7fa995800640 1 -- 172.21.15.97:0/1443377260 wait complete. 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.325+0000 7fa995800640 1 Processor -- start 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.325+0000 7fa995800640 1 -- start start 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.326+0000 7fa995800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9700047c0 0x7fa990151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.326+0000 7fa995800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9900ca3c0 0x7fa9900b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.326+0000 7fa995800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 0x7fa99015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.326+0000 7fa995800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015a9f0 con 0x7fa99015a020 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.326+0000 7fa995800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900b66d0 con 0x7fa9900ca3c0 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.326+0000 7fa995800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900b69b0 con 0x7fa9700047c0 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 0x7fa99015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 0x7fa99015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60662/0 (socket says 172.21.15.97:60662) 2024-01-31T05:03:21.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 -- 172.21.15.97:0/123807407 learned_addr learned my addr 172.21.15.97:0/123807407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98ea00640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9700047c0 0x7fa990151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98e000640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9900ca3c0 0x7fa9900b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 -- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9700047c0 msgr2=0x7fa990151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9700047c0 0x7fa990151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 -- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9900ca3c0 msgr2=0x7fa9900b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9900ca3c0 0x7fa9900b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98f400640 1 -- 172.21.15.97:0/123807407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900b6c90 con 0x7fa99015a020 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.327+0000 7fa98e000640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9900ca3c0 0x7fa9900b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:21.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.328+0000 7fa98f400640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 0x7fa99015a4b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa9900d2f90 tx=0x7fa98800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.328+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988007440 con 0x7fa99015a020 2024-01-31T05:03:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.328+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9880075c0 con 0x7fa99015a020 2024-01-31T05:03:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.328+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988004a20 con 0x7fa99015a020 2024-01-31T05:03:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.328+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9900b7410 con 0x7fa99015a020 2024-01-31T05:03:21.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.328+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9900b6310 con 0x7fa99015a020 2024-01-31T05:03:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.329+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9900b7930 con 0x7fa99015a020 2024-01-31T05:03:21.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.330+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa988004b80 con 0x7fa99015a020 2024-01-31T05:03:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.331+0000 7fa986a00640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa96806aed0 0x7fa96806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.331+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa988088f60 con 0x7fa99015a020 2024-01-31T05:03:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.331+0000 7fa98ea00640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa96806aed0 0x7fa96806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.332+0000 7fa98ea00640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa96806aed0 0x7fa96806d390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa978001710 tx=0x7fa978009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:21.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.336+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa9900b7930 con 0x7fa99015a020 2024-01-31T05:03:21.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.528+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9900b7930 con 0x7fa99015a020 2024-01-31T05:03:21.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.529+0000 7fa986a00640 1 -- 172.21.15.97:0/123807407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9900b7930 con 0x7fa99015a020 2024-01-31T05:03:21.528 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa96806aed0 msgr2=0x7fa96806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa96806aed0 0x7fa96806d390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa978001710 tx=0x7fa978009040 comp rx=0 tx=0).stop 2024-01-31T05:03:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 msgr2=0x7fa99015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 0x7fa99015a4b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa9900d2f90 tx=0x7fa98800c930 comp rx=0 tx=0).stop 2024-01-31T05:03:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 shutdown_connections 2024-01-31T05:03:21.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa96806aed0 0x7fa96806d390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa99015a020 0x7fa99015a4b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9900ca3c0 0x7fa9900b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 --2- 172.21.15.97:0/123807407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9700047c0 0x7fa990151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 >> 172.21.15.97:0/123807407 conn(0x7fa99009c560 msgr2=0x7fa9900b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.531+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 shutdown_connections 2024-01-31T05:03:21.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.532+0000 7fa995800640 1 -- 172.21.15.97:0/123807407 wait complete. 2024-01-31T05:03:21.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:21.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.762+0000 7f108f200640 1 Processor -- start 2024-01-31T05:03:21.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.762+0000 7f108f200640 1 -- start start 2024-01-31T05:03:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.763+0000 7f108f200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f108815a350 0x7f10880abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.763+0000 7f108f200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f108815a730 0x7f108806da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f108f200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108815abc0 con 0x7f108806dfc0 2024-01-31T05:03:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f108f200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10880b2c90 con 0x7f108815a730 2024-01-31T05:03:21.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f108f200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108815aea0 con 0x7f108815a350 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f108d200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f108806dfc0 0x7f10880cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49586/0 (socket says 172.21.15.97:49586) 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f108d200640 1 -- 172.21.15.97:0/1814373532 learned_addr learned my addr 172.21.15.97:0/1814373532 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f107fe00640 1 --2- 172.21.15.97:0/1814373532 >> v2:172.21.15.97:3300/0 conn(0x7f108815a730 0x7f108806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f1087e00640 1 --2- 172.21.15.97:0/1814373532 >> v2:172.21.15.188:3300/0 conn(0x7f108815a350 0x7f10880abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f107fe00640 1 -- 172.21.15.97:0/1814373532 >> v2:172.21.15.188:3300/0 conn(0x7f108815a350 msgr2=0x7f10880abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f107fe00640 1 --2- 172.21.15.97:0/1814373532 >> v2:172.21.15.188:3300/0 conn(0x7f108815a350 0x7f10880abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f107fe00640 1 -- 172.21.15.97:0/1814373532 >> v1:172.21.15.97:6789/0 conn(0x7f108806dfc0 legacy=0x7f10880cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.764+0000 7f1087400640 1 -- 172.21.15.97:0/1814373532 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683466137 0 0) 0x7f108815abc0 con 0x7f108806dfc0 2024-01-31T05:03:21.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.765+0000 7f107fe00640 1 -- 172.21.15.97:0/1814373532 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108815b180 con 0x7f108815a730 2024-01-31T05:03:21.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.766+0000 7f107fe00640 1 --2- 172.21.15.97:0/1814373532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108815a730 0x7f108806da80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f10700097d0 tx=0x7f107002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9d8b84d9f12a7d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:21.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.766+0000 7f1087400640 1 -- 172.21.15.97:0/1814373532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107002e070 con 0x7f108815a730 2024-01-31T05:03:21.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.766+0000 7f1087400640 1 -- 172.21.15.97:0/1814373532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10700315a0 con 0x7f108815a730 2024-01-31T05:03:21.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.766+0000 7f1087400640 1 -- 172.21.15.97:0/1814373532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070031930 con 0x7f108815a730 2024-01-31T05:03:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.766+0000 7f108f200640 1 -- 172.21.15.97:0/1814373532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108815a730 msgr2=0x7f108806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.766+0000 7f108f200640 1 --2- 172.21.15.97:0/1814373532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108815a730 0x7f108806da80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f10700097d0 tx=0x7f107002b840 comp rx=0 tx=0).stop 2024-01-31T05:03:21.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.767+0000 7f108f200640 1 -- 172.21.15.97:0/1814373532 shutdown_connections 2024-01-31T05:03:21.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.767+0000 7f108f200640 1 --2- 172.21.15.97:0/1814373532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108815a730 0x7f108806da80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.767+0000 7f108f200640 1 --2- 172.21.15.97:0/1814373532 >> v2:172.21.15.188:3300/0 conn(0x7f108815a350 0x7f10880abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.767+0000 7f108f200640 1 -- 172.21.15.97:0/1814373532 >> 172.21.15.97:0/1814373532 conn(0x7f10880b0a10 msgr2=0x7f10880b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:21.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.767+0000 7f108f200640 1 -- 172.21.15.97:0/1814373532 shutdown_connections 2024-01-31T05:03:21.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.767+0000 7f108f200640 1 -- 172.21.15.97:0/1814373532 wait complete. 2024-01-31T05:03:21.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.769+0000 7f108f200640 1 Processor -- start 2024-01-31T05:03:21.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.769+0000 7f108f200640 1 -- start start 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f108f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 0x7f1088158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f108f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108815a350 0x7f10880ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f108f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10880aec50 0x7f10881f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f108f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108809c840 con 0x7f108806dfc0 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f108f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880af6c0 con 0x7f10880aec50 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f108f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880af100 con 0x7f108815a350 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f1087e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 0x7f1088158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f1087e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 0x7f1088158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60668/0 (socket says 172.21.15.97:60668) 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f1087e00640 1 -- 172.21.15.97:0/3640776132 learned_addr learned my addr 172.21.15.97:0/3640776132 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.770+0000 7f107fe00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108815a350 0x7f10880ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f108d200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10880aec50 0x7f10881f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1087e00640 1 -- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108815a350 msgr2=0x7f10880ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1087e00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108815a350 0x7f10880ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1087e00640 1 -- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10880aec50 msgr2=0x7f10881f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1087e00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10880aec50 0x7f10881f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1087e00640 1 -- 172.21.15.97:0/3640776132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070009400 con 0x7f108806dfc0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f108d200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10880aec50 0x7f10881f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1087e00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 0x7f1088158ce0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f107800d800 tx=0x7f107800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107800b6c0 con 0x7f108806dfc0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107800b820 con 0x7f108806dfc0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.771+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107800bbd0 con 0x7f108806dfc0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.772+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10880af3e0 con 0x7f108806dfc0 2024-01-31T05:03:21.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.772+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1088099d40 con 0x7f108806dfc0 2024-01-31T05:03:21.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.773+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1088067830 con 0x7f108806dfc0 2024-01-31T05:03:21.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.774+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f107800bd30 con 0x7f108806dfc0 2024-01-31T05:03:21.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.775+0000 7f1084c00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f106006af00 0x7f106006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:21.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.775+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f1078089db0 con 0x7f108806dfc0 2024-01-31T05:03:21.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.776+0000 7f107fe00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f106006af00 0x7f106006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:21.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.776+0000 7f107fe00640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f106006af00 0x7f106006d3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1070036000 tx=0x7f107003d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:21.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.780+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f10780549b0 con 0x7f108806dfc0 2024-01-31T05:03:21.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.965+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f108809ba10 con 0x7f108806dfc0 2024-01-31T05:03:21.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.966+0000 7f1084c00640 1 -- 172.21.15.97:0/3640776132 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1078057fd0 con 0x7f108806dfc0 2024-01-31T05:03:21.966 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f106006af00 msgr2=0x7f106006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f106006af00 0x7f106006d3c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1070036000 tx=0x7f107003d040 comp rx=0 tx=0).stop 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 msgr2=0x7f1088158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 0x7f1088158ce0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f107800d800 tx=0x7f107800dcc0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 shutdown_connections 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10880aec50 0x7f10881f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f108815a350 0x7f10880ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f106006af00 0x7f106006d3c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 --2- 172.21.15.97:0/3640776132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f108806dfc0 0x7f1088158ce0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:21.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 >> 172.21.15.97:0/3640776132 conn(0x7f10880b0a10 msgr2=0x7f10881535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:21.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 shutdown_connections 2024-01-31T05:03:21.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:21.969+0000 7f108f200640 1 -- 172.21.15.97:0/3640776132 wait complete. 2024-01-31T05:03:21.986 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-01-31T05:03:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.196+0000 7ffabda00640 1 Processor -- start 2024-01-31T05:03:22.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.196+0000 7ffabda00640 1 -- start start 2024-01-31T05:03:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffabda00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffab80cbf20 0x7ffab80722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffabda00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffab80abcf0 0x7ffab80ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffabda00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab806cd20 con 0x7ffab815b470 2024-01-31T05:03:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffabda00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab80ce970 con 0x7ffab80cbf20 2024-01-31T05:03:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffabda00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab815b850 con 0x7ffab80abcf0 2024-01-31T05:03:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffab6000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffab80cbf20 0x7ffab80722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffab6000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffab80cbf20 0x7ffab80722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60684/0 (socket says 172.21.15.97:60684) 2024-01-31T05:03:22.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffab6000640 1 -- 172.21.15.97:0/470887249 learned_addr learned my addr 172.21.15.97:0/470887249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.198+0000 7ffab7400640 1 --2- 172.21.15.97:0/470887249 >> v2:172.21.15.188:3300/0 conn(0x7ffab80abcf0 0x7ffab80ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.199+0000 7ffab6000640 1 -- 172.21.15.97:0/470887249 >> v2:172.21.15.188:3300/0 conn(0x7ffab80abcf0 msgr2=0x7ffab80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.199+0000 7ffab6000640 1 --2- 172.21.15.97:0/470887249 >> v2:172.21.15.188:3300/0 conn(0x7ffab80abcf0 0x7ffab80ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.199+0000 7ffab6000640 1 -- 172.21.15.97:0/470887249 >> v1:172.21.15.188:6789/0 conn(0x7ffab815b470 legacy=0x7ffab80cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:22.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.199+0000 7ffab6000640 1 -- 172.21.15.97:0/470887249 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80ceb60 con 0x7ffab80cbf20 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.200+0000 7ffab6000640 1 --2- 172.21.15.97:0/470887249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80722c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffab80711b0 tx=0x7ffaa002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78e35a2749105a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.200+0000 7ffab5600640 1 -- 172.21.15.97:0/470887249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0031870 con 0x7ffab80cbf20 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.200+0000 7ffab5600640 1 -- 172.21.15.97:0/470887249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa00319d0 con 0x7ffab80cbf20 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.200+0000 7ffab5600640 1 -- 172.21.15.97:0/470887249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0031d60 con 0x7ffab80cbf20 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 -- 172.21.15.97:0/470887249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 msgr2=0x7ffab80722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 --2- 172.21.15.97:0/470887249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80722c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ffab80711b0 tx=0x7ffaa002b910 comp rx=0 tx=0).stop 2024-01-31T05:03:22.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 -- 172.21.15.97:0/470887249 shutdown_connections 2024-01-31T05:03:22.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 --2- 172.21.15.97:0/470887249 >> v2:172.21.15.188:3300/0 conn(0x7ffab80abcf0 0x7ffab80ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 --2- 172.21.15.97:0/470887249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80722c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 -- 172.21.15.97:0/470887249 >> 172.21.15.97:0/470887249 conn(0x7ffab80b0a10 msgr2=0x7ffab80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:22.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 -- 172.21.15.97:0/470887249 shutdown_connections 2024-01-31T05:03:22.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.201+0000 7ffabda00640 1 -- 172.21.15.97:0/470887249 wait complete. 2024-01-31T05:03:22.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.203+0000 7ffabda00640 1 Processor -- start 2024-01-31T05:03:22.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.203+0000 7ffabda00640 1 -- start start 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffabda00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab80abcf0 0x7ffab80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffabda00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffabda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab815b470 0x7ffab80b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffabda00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab814a7f0 con 0x7ffab80cbf20 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffabda00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab815ca20 con 0x7ffab80abcf0 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffabda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab80b3490 con 0x7ffab815b470 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60694/0 (socket says 172.21.15.97:60694) 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 -- 172.21.15.97:0/3693996637 learned_addr learned my addr 172.21.15.97:0/3693996637 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6a00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab80abcf0 0x7ffab80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 -- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab815b470 msgr2=0x7ffab80b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab815b470 0x7ffab80b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 -- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab80abcf0 msgr2=0x7ffab80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab80abcf0 0x7ffab80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.204+0000 7ffab6000640 1 -- 172.21.15.97:0/3693996637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa00094d0 con 0x7ffab80cbf20 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffab6a00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab80abcf0 0x7ffab80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:22.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffab6000640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80b2a10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097f0 tx=0x7ffaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:22.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa003ac60 con 0x7ffab80cbf20 2024-01-31T05:03:22.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa003adc0 con 0x7ffab80cbf20 2024-01-31T05:03:22.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0044610 con 0x7ffab80cbf20 2024-01-31T05:03:22.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab80b3740 con 0x7ffab80cbf20 2024-01-31T05:03:22.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.205+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab80b3d00 con 0x7ffab80cbf20 2024-01-31T05:03:22.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.206+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab80ce730 con 0x7ffab80cbf20 2024-01-31T05:03:22.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.207+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ffaa004d450 con 0x7ffab80cbf20 2024-01-31T05:03:22.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.208+0000 7ffaaea00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa9006aed0 0x7ffa9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.208+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7ffaa002d040 con 0x7ffab80cbf20 2024-01-31T05:03:22.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.208+0000 7ffab6a00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa9006aed0 0x7ffa9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.209+0000 7ffab6a00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa9006aed0 0x7ffa9006d390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800a810 tx=0x7ffaa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:22.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.213+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ffaa0076b10 con 0x7ffab80cbf20 2024-01-31T05:03:22.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.408+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7ffab8067830 con 0x7ffab80cbf20 2024-01-31T05:03:22.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.409+0000 7ffaaea00640 1 -- 172.21.15.97:0/3693996637 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v43) v1 ==== 97+0+11 (secure 0 0 0) 0x7ffaa0035020 con 0x7ffab80cbf20 2024-01-31T05:03:22.408 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa9006aed0 msgr2=0x7ffa9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa9006aed0 0x7ffa9006d390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffaa800a810 tx=0x7ffaa8008040 comp rx=0 tx=0).stop 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 msgr2=0x7ffab80b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80b2a10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097f0 tx=0x7ffaa00046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 shutdown_connections 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab815b470 0x7ffab80b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa9006aed0 0x7ffa9006d390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab80cbf20 0x7ffab80b2a10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 --2- 172.21.15.97:0/3693996637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab80abcf0 0x7ffab80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 >> 172.21.15.97:0/3693996637 conn(0x7ffab80b0a10 msgr2=0x7ffab80cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 shutdown_connections 2024-01-31T05:03:22.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.411+0000 7ffabda00640 1 -- 172.21.15.97:0/3693996637 wait complete. 2024-01-31T05:03:22.428 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:22.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.606+0000 7f89b7c00640 1 Processor -- start 2024-01-31T05:03:22.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.606+0000 7f89b7c00640 1 -- start start 2024-01-31T05:03:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.607+0000 7f89b7c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89b00cb810 0x7f89b00cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b7c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b006d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b7c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b006d1a0 con 0x7f89b006db70 2024-01-31T05:03:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b7c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b00cc960 con 0x7f89b00cb810 2024-01-31T05:03:22.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b7c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b006c600 con 0x7f89b015b400 2024-01-31T05:03:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b5200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89b00cb810 0x7f89b00cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89afe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b006d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b5200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89b00cb810 0x7f89b00cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.97:57414/0 (socket says 172.21.15.97:57414) 2024-01-31T05:03:22.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.608+0000 7f89b5200640 1 -- 172.21.15.97:0/1184554858 learned_addr learned my addr 172.21.15.97:0/1184554858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:22.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.609+0000 7f89b5200640 1 -- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 msgr2=0x7f89b006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:22.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.609+0000 7f89b5200640 1 --2- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b006d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.609+0000 7f89b5200640 1 -- 172.21.15.97:0/1184554858 >> v1:172.21.15.188:6789/0 conn(0x7f89b006db70 legacy=0x7f89b00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.609+0000 7f89b5200640 1 -- 172.21.15.97:0/1184554858 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00cc130 con 0x7f89b00cb810 2024-01-31T05:03:22.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.609+0000 7f89af400640 1 -- 172.21.15.97:0/1184554858 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399654995 0 0) 0x7f89b006d1a0 con 0x7f89b006db70 2024-01-31T05:03:22.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.609+0000 7f89afe00640 1 --2- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b006d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:03:22.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.611+0000 7f89b5200640 1 --2- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b00cb810 0x7f89b00cbbf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=801d51ff8971c150 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:22.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89af400640 1 -- 172.21.15.97:0/1184554858 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0030840 con 0x7f89b00cb810 2024-01-31T05:03:22.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89af400640 1 -- 172.21.15.97:0/1184554858 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a00309a0 con 0x7f89b00cb810 2024-01-31T05:03:22.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89af400640 1 -- 172.21.15.97:0/1184554858 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0030d30 con 0x7f89b00cb810 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 -- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b00cb810 msgr2=0x7f89b00cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 --2- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b00cb810 0x7f89b00cbbf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009870 tx=0x7f89a002b980 comp rx=0 tx=0).stop 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 -- 172.21.15.97:0/1184554858 shutdown_connections 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 --2- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b006d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 --2- 172.21.15.97:0/1184554858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b00cb810 0x7f89b00cbbf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 -- 172.21.15.97:0/1184554858 >> 172.21.15.97:0/1184554858 conn(0x7f89b00b0a10 msgr2=0x7f89b00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 -- 172.21.15.97:0/1184554858 shutdown_connections 2024-01-31T05:03:22.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.612+0000 7f89b7c00640 1 -- 172.21.15.97:0/1184554858 wait complete. 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.614+0000 7f89b7c00640 1 Processor -- start 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.614+0000 7f89b7c00640 1 -- start start 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.615+0000 7f89b7c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b006db70 0x7f89b0152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.615+0000 7f89b7c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 0x7f89b00b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.615+0000 7f89b7c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.615+0000 7f89b7c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b015d920 con 0x7f89b00cb810 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.615+0000 7f89b7c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b1e10 con 0x7f89b006db70 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.615+0000 7f89b7c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00b20f0 con 0x7f89b015b400 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 0x7f89b00b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 0x7f89b00b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60700/0 (socket says 172.21.15.97:60700) 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 -- 172.21.15.97:0/3442738200 learned_addr learned my addr 172.21.15.97:0/3442738200 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89b5200640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b006db70 0x7f89b0152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89b5c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 -- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 msgr2=0x7f89b015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 -- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b006db70 msgr2=0x7f89b0152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b006db70 0x7f89b0152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89afe00640 1 -- 172.21.15.97:0/3442738200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998009400 con 0x7f89b00cb810 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89b5200640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b006db70 0x7f89b0152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.616+0000 7f89b5c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.617+0000 7f89afe00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 0x7f89b00b2bd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f899800e900 tx=0x7f899800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.617+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8998009660 con 0x7f89b00cb810 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.617+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899800ca00 con 0x7f89b00cb810 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.617+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a00094d0 con 0x7f89b00cb810 2024-01-31T05:03:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.617+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800cdb0 con 0x7f89b00cb810 2024-01-31T05:03:22.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.617+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b00b26b0 con 0x7f89b00cb810 2024-01-31T05:03:22.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.618+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b00677c0 con 0x7f89b00cb810 2024-01-31T05:03:22.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.619+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8998013470 con 0x7f89b00cb810 2024-01-31T05:03:22.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.620+0000 7f89acc00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898806aed0 0x7f898806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:22.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.620+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f89980897d0 con 0x7f89b00cb810 2024-01-31T05:03:22.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.620+0000 7f89b5200640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898806aed0 0x7f898806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:22.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.621+0000 7f89b5200640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898806aed0 0x7f898806d390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89a0036000 tx=0x7f89a003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:22.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.625+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f89980543c0 con 0x7f89b00cb810 2024-01-31T05:03:22.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.843+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89b00b1000 con 0x7f89b00cb810 2024-01-31T05:03:22.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.844+0000 7f89acc00640 1 -- 172.21.15.97:0/3442738200 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f89980579e0 con 0x7f89b00cb810 2024-01-31T05:03:22.844 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:22.844 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:42.346423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:03:22.844 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-01-31T05:03:22.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898806aed0 msgr2=0x7f898806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898806aed0 0x7f898806d390 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f89a0036000 tx=0x7f89a003b040 comp rx=0 tx=0).stop 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 msgr2=0x7f89b00b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 0x7f89b00b2bd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f899800e900 tx=0x7f899800edc0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 shutdown_connections 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89b015b400 0x7f89b015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898806aed0 0x7f898806d390 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89b00cb810 0x7f89b00b2bd0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 --2- 172.21.15.97:0/3442738200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89b006db70 0x7f89b0152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 >> 172.21.15.97:0/3442738200 conn(0x7f89b00b0a10 msgr2=0x7f89b00abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 shutdown_connections 2024-01-31T05:03:22.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:22.847+0000 7f89b7c00640 1 -- 172.21.15.97:0/3442738200 wait complete. 2024-01-31T05:03:22.865 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:23.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.074+0000 7f9f32200640 1 Processor -- start 2024-01-31T05:03:23.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.074+0000 7f9f32200640 1 -- start start 2024-01-31T05:03:23.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f32200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c15b400 0x7f9f2c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f32200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f2c06db70 0x7f9f2c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f32200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c06d1a0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f32200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0cc960 con 0x7f9f2c06db70 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f32200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c06c600 con 0x7f9f2c15b400 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f2be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f2c06db70 0x7f9f2c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.075+0000 7f9f2b400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9f2c0cb810 0x7f9f2c0cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49588/0 (socket says 172.21.15.97:49588) 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2b400640 1 -- 172.21.15.97:0/1291325215 learned_addr learned my addr 172.21.15.97:0/1291325215 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2a000640 1 -- 172.21.15.97:0/1291325215 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018795130 0 0) 0x7f9f2c06d1a0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2aa00640 1 --2- 172.21.15.97:0/1291325215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c15b400 0x7f9f2c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2a000640 1 -- 172.21.15.97:0/1291325215 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f0c003520 con 0x7f9f2c0cb810 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2be00640 1 -- 172.21.15.97:0/1291325215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c15b400 msgr2=0x7f9f2c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2be00640 1 --2- 172.21.15.97:0/1291325215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c15b400 0x7f9f2c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2be00640 1 -- 172.21.15.97:0/1291325215 >> v1:172.21.15.97:6789/0 conn(0x7f9f2c0cb810 legacy=0x7f9f2c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.076+0000 7f9f2be00640 1 -- 172.21.15.97:0/1291325215 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c06c870 con 0x7f9f2c06db70 2024-01-31T05:03:23.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.077+0000 7f9f2be00640 1 --2- 172.21.15.97:0/1291325215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c06db70 0x7f9f2c0ccd00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9f240097a0 tx=0x7f9f2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23dfe971f359042 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.077+0000 7f9f2a000640 1 -- 172.21.15.97:0/1291325215 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24031830 con 0x7f9f2c06db70 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.077+0000 7f9f2a000640 1 -- 172.21.15.97:0/1291325215 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f24031990 con 0x7f9f2c06db70 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.077+0000 7f9f2a000640 1 -- 172.21.15.97:0/1291325215 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24031d20 con 0x7f9f2c06db70 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 -- 172.21.15.97:0/1291325215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c06db70 msgr2=0x7f9f2c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 --2- 172.21.15.97:0/1291325215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c06db70 0x7f9f2c0ccd00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9f240097a0 tx=0x7f9f2402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 -- 172.21.15.97:0/1291325215 shutdown_connections 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 --2- 172.21.15.97:0/1291325215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c06db70 0x7f9f2c0ccd00 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 --2- 172.21.15.97:0/1291325215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c15b400 0x7f9f2c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 -- 172.21.15.97:0/1291325215 >> 172.21.15.97:0/1291325215 conn(0x7f9f2c0b0a10 msgr2=0x7f9f2c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 -- 172.21.15.97:0/1291325215 shutdown_connections 2024-01-31T05:03:23.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.078+0000 7f9f32200640 1 -- 172.21.15.97:0/1291325215 wait complete. 2024-01-31T05:03:23.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.080+0000 7f9f32200640 1 Processor -- start 2024-01-31T05:03:23.079 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.080+0000 7f9f32200640 1 -- start start 2024-01-31T05:03:23.081 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f32200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c06db70 0x7f9f2c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f32200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 0x7f9f2c0ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f32200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c15b400 0x7f9f2c0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f32200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c159190 con 0x7f9f2c0cb810 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f32200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c156480 con 0x7f9f2c15b400 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f32200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c158510 con 0x7f9f2c06db70 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 0x7f9f2c0ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 0x7f9f2c0ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60716/0 (socket says 172.21.15.97:60716) 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f2aa00640 1 -- 172.21.15.97:0/3157340802 learned_addr learned my addr 172.21.15.97:0/3157340802 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.081+0000 7f9f2b400640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c06db70 0x7f9f2c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2aa00640 1 -- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c06db70 msgr2=0x7f9f2c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2aa00640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c06db70 0x7f9f2c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2aa00640 1 -- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c15b400 msgr2=0x7f9f2c0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2aa00640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c15b400 0x7f9f2c0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2aa00640 1 -- 172.21.15.97:0/3157340802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f24009400 con 0x7f9f2c0cb810 2024-01-31T05:03:23.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2b400640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c06db70 0x7f9f2c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f2aa00640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 0x7f9f2c0ca130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c7c0 tx=0x7f9f1400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14007820 con 0x7f9f2c0cb810 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14007980 con 0x7f9f2c0cb810 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14007cf0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c1587f0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.082+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c158db0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.083+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c067140 con 0x7f9f2c0cb810 2024-01-31T05:03:23.083 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.084+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9f140113f0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.085+0000 7f9f23400640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f0406aed0 0x7f9f0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.085+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f9f140896b0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.085+0000 7f9f2b400640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f0406aed0 0x7f9f0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.086+0000 7f9f2b400640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f0406aed0 0x7f9f0406d390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005c00 tx=0x7f9f1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.090+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9f14054210 con 0x7f9f2c0cb810 2024-01-31T05:03:23.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.285+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f2c0b2ba0 con 0x7f9f2c0cb810 2024-01-31T05:03:23.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.288+0000 7f9f23400640 1 -- 172.21.15.97:0/3157340802 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f9f14057830 con 0x7f9f2c0cb810 2024-01-31T05:03:23.288 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:23.289 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f0406aed0 msgr2=0x7f9f0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f0406aed0 0x7f9f0406d390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c005c00 tx=0x7f9f1c009040 comp rx=0 tx=0).stop 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 msgr2=0x7f9f2c0ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 0x7f9f2c0ca130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c7c0 tx=0x7f9f1400cc80 comp rx=0 tx=0).stop 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 shutdown_connections 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f2c15b400 0x7f9f2c0ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f0406aed0 0x7f9f0406d390 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f2c0cb810 0x7f9f2c0ca130 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 --2- 172.21.15.97:0/3157340802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f2c06db70 0x7f9f2c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 >> 172.21.15.97:0/3157340802 conn(0x7f9f2c0b0a10 msgr2=0x7f9f2c0cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 shutdown_connections 2024-01-31T05:03:23.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.291+0000 7f9f32200640 1 -- 172.21.15.97:0/3157340802 wait complete. 2024-01-31T05:03:23.309 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-31T05:03:23.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:23.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.513+0000 7f3995000640 1 Processor -- start 2024-01-31T05:03:23.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.513+0000 7f3995000640 1 -- start start 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3995000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399015d370 0x7f39900cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3995000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39900c89f0 0x7f39900c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3995000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39900c9370 con 0x7f3990160f90 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3995000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39900711a0 con 0x7f39900c89f0 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3995000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3990071480 con 0x7f399015d370 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39900c89f0 0x7f39900c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39900c89f0 0x7f39900c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60726/0 (socket says 172.21.15.97:60726) 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 -- 172.21.15.97:0/1349650533 learned_addr learned my addr 172.21.15.97:0/1349650533 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:23.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f398de00640 1 --2- 172.21.15.97:0/1349650533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399015d370 0x7f39900cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 -- 172.21.15.97:0/1349650533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399015d370 msgr2=0x7f39900cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 --2- 172.21.15.97:0/1349650533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399015d370 0x7f39900cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 -- 172.21.15.97:0/1349650533 >> v1:172.21.15.188:6789/0 conn(0x7f3990160f90 legacy=0x7f39901613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:23.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.515+0000 7f3987e00640 1 -- 172.21.15.97:0/1349650533 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990071760 con 0x7f39900c89f0 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.516+0000 7f3987e00640 1 --2- 172.21.15.97:0/1349650533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39900c89f0 0x7f39900c8e30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3978009870 tx=0x7f397802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3f282df9d1db0f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f398d400640 1 -- 172.21.15.97:0/1349650533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978031640 con 0x7f39900c89f0 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f398d400640 1 -- 172.21.15.97:0/1349650533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39780317a0 con 0x7f39900c89f0 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f398d400640 1 -- 172.21.15.97:0/1349650533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978031b30 con 0x7f39900c89f0 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 -- 172.21.15.97:0/1349650533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39900c89f0 msgr2=0x7f39900c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 --2- 172.21.15.97:0/1349650533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39900c89f0 0x7f39900c8e30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3978009870 tx=0x7f397802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 -- 172.21.15.97:0/1349650533 shutdown_connections 2024-01-31T05:03:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 --2- 172.21.15.97:0/1349650533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39900c89f0 0x7f39900c8e30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 --2- 172.21.15.97:0/1349650533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f399015d370 0x7f39900cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 -- 172.21.15.97:0/1349650533 >> 172.21.15.97:0/1349650533 conn(0x7f39900cf6e0 msgr2=0x7f39900cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 -- 172.21.15.97:0/1349650533 shutdown_connections 2024-01-31T05:03:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.517+0000 7f3995000640 1 -- 172.21.15.97:0/1349650533 wait complete. 2024-01-31T05:03:23.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.519+0000 7f3995000640 1 Processor -- start 2024-01-31T05:03:23.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.519+0000 7f3995000640 1 -- start start 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f3995000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39900c89f0 0x7f399015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f3995000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399015d370 0x7f39901f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f3995000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 0x7f3990151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f3995000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399015f480 con 0x7f3990160f90 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f3995000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399015deb0 con 0x7f399015d370 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f3995000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399015e190 con 0x7f39900c89f0 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f398e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 0x7f3990151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f398e800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 0x7f3990151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60736/0 (socket says 172.21.15.97:60736) 2024-01-31T05:03:23.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.520+0000 7f398e800640 1 -- 172.21.15.97:0/1130660412 learned_addr learned my addr 172.21.15.97:0/1130660412 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f3987e00640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399015d370 0x7f39901f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398de00640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39900c89f0 0x7f399015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398e800640 1 -- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39900c89f0 msgr2=0x7f399015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398e800640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39900c89f0 0x7f399015d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398e800640 1 -- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399015d370 msgr2=0x7f39901f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398e800640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399015d370 0x7f39901f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398e800640 1 -- 172.21.15.97:0/1130660412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780094d0 con 0x7f3990160f90 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f3987e00640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399015d370 0x7f39901f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f398e800640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 0x7f3990151200 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f398800c780 tx=0x7f398800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988003d80 con 0x7f3990160f90 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398800ce90 con 0x7f3990160f90 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399015e470 con 0x7f3990160f90 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.521+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39880052b0 con 0x7f3990160f90 2024-01-31T05:03:23.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.522+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3990151740 con 0x7f3990160f90 2024-01-31T05:03:23.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.522+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3990067dc0 con 0x7f3990160f90 2024-01-31T05:03:23.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.523+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3988005410 con 0x7f3990160f90 2024-01-31T05:03:23.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.524+0000 7f3986000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f396806aed0 0x7f396806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.524+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f39880890d0 con 0x7f3990160f90 2024-01-31T05:03:23.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.525+0000 7f398de00640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f396806aed0 0x7f396806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.525+0000 7f398de00640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f396806aed0 0x7f396806d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3980006fd0 tx=0x7f3980008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.529+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3988053d40 con 0x7f3990160f90 2024-01-31T05:03:23.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.759+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3990073820 con 0x7f3990160f90 2024-01-31T05:03:23.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.760+0000 7f3986000640 1 -- 172.21.15.97:0/1130660412 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f3988017090 con 0x7f3990160f90 2024-01-31T05:03:23.759 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:23.759 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:42.346423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:03:23.760 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-01-31T05:03:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.762+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f396806aed0 msgr2=0x7f396806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.762+0000 7f3995000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f396806aed0 0x7f396806d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3980006fd0 tx=0x7f3980008040 comp rx=0 tx=0).stop 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 msgr2=0x7f3990151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 0x7f3990151200 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f398800c780 tx=0x7f398800cc40 comp rx=0 tx=0).stop 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 shutdown_connections 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f396806aed0 0x7f396806d390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990160f90 0x7f3990151200 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f399015d370 0x7f39901f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 --2- 172.21.15.97:0/1130660412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f39900c89f0 0x7f399015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 >> 172.21.15.97:0/1130660412 conn(0x7f39900cf6e0 msgr2=0x7f39900af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 shutdown_connections 2024-01-31T05:03:23.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.763+0000 7f3995000640 1 -- 172.21.15.97:0/1130660412 wait complete. 2024-01-31T05:03:23.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.976+0000 7fe5fc400640 1 Processor -- start 2024-01-31T05:03:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.976+0000 7fe5fc400640 1 -- start start 2024-01-31T05:03:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fc400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5f40b1d10 0x7fe5f40b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fc400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5f415a600 0x7fe5f415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fc400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f40b2630 con 0x7fe5f415dc50 2024-01-31T05:03:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fc400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f406c8b0 con 0x7fe5f415a600 2024-01-31T05:03:23.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fc400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f415b120 con 0x7fe5f40b1d10 2024-01-31T05:03:23.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5f9000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5f415a600 0x7fe5f415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fa400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe5f415dc50 0x7fe5f415e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49600/0 (socket says 172.21.15.97:49600) 2024-01-31T05:03:23.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5fa400640 1 -- 172.21.15.97:0/454664554 learned_addr learned my addr 172.21.15.97:0/454664554 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:23.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5f9a00640 1 --2- 172.21.15.97:0/454664554 >> v2:172.21.15.188:3300/0 conn(0x7fe5f40b1d10 0x7fe5f40b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5f3e00640 1 -- 172.21.15.97:0/454664554 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621445444 0 0) 0x7fe5f40b2630 con 0x7fe5f415dc50 2024-01-31T05:03:23.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5f9000640 1 -- 172.21.15.97:0/454664554 >> v2:172.21.15.188:3300/0 conn(0x7fe5f40b1d10 msgr2=0x7fe5f40b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:23.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5f9000640 1 --2- 172.21.15.97:0/454664554 >> v2:172.21.15.188:3300/0 conn(0x7fe5f40b1d10 0x7fe5f40b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.978+0000 7fe5f9000640 1 -- 172.21.15.97:0/454664554 >> v1:172.21.15.97:6789/0 conn(0x7fe5f415dc50 legacy=0x7fe5f415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.979+0000 7fe5f9000640 1 -- 172.21.15.97:0/454664554 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f415b400 con 0x7fe5f415a600 2024-01-31T05:03:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.980+0000 7fe5f9000640 1 --2- 172.21.15.97:0/454664554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f415aa60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4072e40 tx=0x7fe5dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db3f2fa6b5c3518 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.980+0000 7fe5f3e00640 1 -- 172.21.15.97:0/454664554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc031830 con 0x7fe5f415a600 2024-01-31T05:03:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.980+0000 7fe5f3e00640 1 -- 172.21.15.97:0/454664554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc031990 con 0x7fe5f415a600 2024-01-31T05:03:23.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.980+0000 7fe5f3e00640 1 -- 172.21.15.97:0/454664554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc031d20 con 0x7fe5f415a600 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.980+0000 7fe5fc400640 1 -- 172.21.15.97:0/454664554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 msgr2=0x7fe5f415aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.980+0000 7fe5fc400640 1 --2- 172.21.15.97:0/454664554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f415aa60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4072e40 tx=0x7fe5dc02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.981+0000 7fe5fc400640 1 -- 172.21.15.97:0/454664554 shutdown_connections 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.981+0000 7fe5fc400640 1 --2- 172.21.15.97:0/454664554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f415aa60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.981+0000 7fe5fc400640 1 --2- 172.21.15.97:0/454664554 >> v2:172.21.15.188:3300/0 conn(0x7fe5f40b1d10 0x7fe5f40b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.981+0000 7fe5fc400640 1 -- 172.21.15.97:0/454664554 >> 172.21.15.97:0/454664554 conn(0x7fe5f40b0a10 msgr2=0x7fe5f40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.981+0000 7fe5fc400640 1 -- 172.21.15.97:0/454664554 shutdown_connections 2024-01-31T05:03:23.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.981+0000 7fe5fc400640 1 -- 172.21.15.97:0/454664554 wait complete. 2024-01-31T05:03:23.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.982+0000 7fe5fc400640 1 Processor -- start 2024-01-31T05:03:23.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.982+0000 7fe5fc400640 1 -- start start 2024-01-31T05:03:23.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.983+0000 7fe5fc400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5f40b1d10 0x7fe5f4070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.983+0000 7fe5fc400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.983+0000 7fe5fc400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5f415dc50 0x7fe5f41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.983+0000 7fe5fc400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4155430 con 0x7fe5f415a600 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.983+0000 7fe5fc400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41545c0 con 0x7fe5f415dc50 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.983+0000 7fe5fc400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f41548a0 con 0x7fe5f40b1d10 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f414e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60752/0 (socket says 172.21.15.97:60752) 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 -- 172.21.15.97:0/3946388817 learned_addr learned my addr 172.21.15.97:0/3946388817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9a00640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5f40b1d10 0x7fe5f4070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 -- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5f40b1d10 msgr2=0x7fe5f4070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5f40b1d10 0x7fe5f4070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 -- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5f415dc50 msgr2=0x7fe5f41592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5fa400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5f415dc50 0x7fe5f41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5f415dc50 0x7fe5f41592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.984+0000 7fe5f9000640 1 -- 172.21.15.97:0/3946388817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc009400 con 0x7fe5f415a600 2024-01-31T05:03:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.985+0000 7fe5f9000640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f414e8d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009890 tx=0x7fe5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.985+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc03ba70 con 0x7fe5f415a600 2024-01-31T05:03:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.985+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc03bbf0 con 0x7fe5f415a600 2024-01-31T05:03:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.985+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f40adb80 con 0x7fe5f415a600 2024-01-31T05:03:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.985+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc045400 con 0x7fe5f415a600 2024-01-31T05:03:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.985+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f4155960 con 0x7fe5f415a600 2024-01-31T05:03:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.986+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f40677c0 con 0x7fe5f415a600 2024-01-31T05:03:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.987+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe5dc04e440 con 0x7fe5f415a600 2024-01-31T05:03:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.988+0000 7fe5f1600640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe5cc06aed0 0x7fe5cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.988+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fe5dc0abca0 con 0x7fe5f415a600 2024-01-31T05:03:23.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.988+0000 7fe5f9a00640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe5cc06aed0 0x7fe5cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:23.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.989+0000 7fe5f9a00640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe5cc06aed0 0x7fe5cc06d390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006fd0 tx=0x7fe5e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:23.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:23.992+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe5dc076770 con 0x7fe5f415a600 2024-01-31T05:03:24.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.221+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f406e740 con 0x7fe5f415a600 2024-01-31T05:03:24.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.223+0000 7fe5f1600640 1 -- 172.21.15.97:0/3946388817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fe5dc035020 con 0x7fe5f415a600 2024-01-31T05:03:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:24.222 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:02:31.698287+0000","modified":"2024-01-31T05:02:42.346423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5556":{"gid":5556,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/111752619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":111752619},{"type":"v1","addr":"172.21.15.188:6834","nonce":111752619}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-31T05:03:24.223 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2024-01-31T05:03:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe5cc06aed0 msgr2=0x7fe5cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe5cc06aed0 0x7fe5cc06d390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006fd0 tx=0x7fe5e4008040 comp rx=0 tx=0).stop 2024-01-31T05:03:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 msgr2=0x7fe5f414e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f414e8d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc009890 tx=0x7fe5dc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 shutdown_connections 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5f415dc50 0x7fe5f41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe5cc06aed0 0x7fe5cc06d390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5f415a600 0x7fe5f414e8d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 --2- 172.21.15.97:0/3946388817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5f40b1d10 0x7fe5f4070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 >> 172.21.15.97:0/3946388817 conn(0x7fe5f40b0a10 msgr2=0x7fe5f40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 shutdown_connections 2024-01-31T05:03:24.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.225+0000 7fe5fc400640 1 -- 172.21.15.97:0/3946388817 wait complete. 2024-01-31T05:03:24.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:24.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.438+0000 7f509f400640 1 Processor -- start 2024-01-31T05:03:24.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.438+0000 7f509f400640 1 -- start start 2024-01-31T05:03:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f509815d520 0x7f50980cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509f400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50980ce430 0x7f5098160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509f400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50980abcf0 con 0x7f5098161500 2024-01-31T05:03:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509f400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509815d900 con 0x7f509815d520 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509f400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5098161970 con 0x7f50980ce430 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509ca00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f509815d520 0x7f50980cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509ca00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f509815d520 0x7f50980cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60758/0 (socket says 172.21.15.97:60758) 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.440+0000 7f509ca00640 1 -- 172.21.15.97:0/1736349476 learned_addr learned my addr 172.21.15.97:0/1736349476 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.441+0000 7f509ca00640 1 -- 172.21.15.97:0/1736349476 >> v2:172.21.15.188:3300/0 conn(0x7f50980ce430 msgr2=0x7f5098160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.441+0000 7f509ca00640 1 --2- 172.21.15.97:0/1736349476 >> v2:172.21.15.188:3300/0 conn(0x7f50980ce430 0x7f5098160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.441+0000 7f509ca00640 1 -- 172.21.15.97:0/1736349476 >> v1:172.21.15.188:6789/0 conn(0x7f5098161500 legacy=0x7f50980b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.441+0000 7f509ca00640 1 -- 172.21.15.97:0/1736349476 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509806ca40 con 0x7f509815d520 2024-01-31T05:03:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.442+0000 7f509ca00640 1 --2- 172.21.15.97:0/1736349476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509815d520 0x7f50980cd710 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f50980cc510 tx=0x7f508002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415f38fb9fb074da server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.442+0000 7f5097400640 1 -- 172.21.15.97:0/1736349476 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080031870 con 0x7f509815d520 2024-01-31T05:03:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.442+0000 7f5097400640 1 -- 172.21.15.97:0/1736349476 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50800319d0 con 0x7f509815d520 2024-01-31T05:03:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.442+0000 7f5097400640 1 -- 172.21.15.97:0/1736349476 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080031d60 con 0x7f509815d520 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 -- 172.21.15.97:0/1736349476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509815d520 msgr2=0x7f50980cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 --2- 172.21.15.97:0/1736349476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509815d520 0x7f50980cd710 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f50980cc510 tx=0x7f508002b910 comp rx=0 tx=0).stop 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 -- 172.21.15.97:0/1736349476 shutdown_connections 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 --2- 172.21.15.97:0/1736349476 >> v2:172.21.15.188:3300/0 conn(0x7f50980ce430 0x7f5098160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 --2- 172.21.15.97:0/1736349476 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f509815d520 0x7f50980cd710 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 -- 172.21.15.97:0/1736349476 >> 172.21.15.97:0/1736349476 conn(0x7f50980b0a10 msgr2=0x7f50980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 -- 172.21.15.97:0/1736349476 shutdown_connections 2024-01-31T05:03:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.443+0000 7f509f400640 1 -- 172.21.15.97:0/1736349476 wait complete. 2024-01-31T05:03:24.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.444+0000 7f509f400640 1 Processor -- start 2024-01-31T05:03:24.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.445+0000 7f509f400640 1 -- start start 2024-01-31T05:03:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.445+0000 7f509f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50980ce430 0x7f509814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f509815d520 0x7f509815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 0x7f509815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509814be90 con 0x7f5098161500 2024-01-31T05:03:24.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509f400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509815f720 con 0x7f50980ce430 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509815fa00 con 0x7f509815d520 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 0x7f509815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 0x7f509815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60760/0 (socket says 172.21.15.97:60760) 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 -- 172.21.15.97:0/2124593950 learned_addr learned my addr 172.21.15.97:0/2124593950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 -- 172.21.15.97:0/2124593950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f509815d520 msgr2=0x7f509815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f509815d520 0x7f509815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 -- 172.21.15.97:0/2124593950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50980ce430 msgr2=0x7f509814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50980ce430 0x7f509814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.446+0000 7f509d400640 1 -- 172.21.15.97:0/2124593950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50800094d0 con 0x7f5098161500 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.447+0000 7f509d400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 0x7f509815f0f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f509000b990 tx=0x7f509000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.447+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509000cba0 con 0x7f5098161500 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.447+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509000cd00 con 0x7f5098161500 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.447+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509815fce0 con 0x7f5098161500 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.447+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090014680 con 0x7f5098161500 2024-01-31T05:03:24.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.447+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50981602a0 con 0x7f5098161500 2024-01-31T05:03:24.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.448+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50981604c0 con 0x7f5098161500 2024-01-31T05:03:24.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.449+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f509001d3b0 con 0x7f5098161500 2024-01-31T05:03:24.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.450+0000 7f5094c00640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f507006aed0 0x7f507006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.450+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f5090089940 con 0x7f5098161500 2024-01-31T05:03:24.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.450+0000 7f509ca00640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f507006aed0 0x7f507006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.450+0000 7f509ca00640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f507006aed0 0x7f507006d390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5080036000 tx=0x7f5080030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:24.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.454+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f50900544a0 con 0x7f5098161500 2024-01-31T05:03:24.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.634+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5098067fd0 con 0x7f5098161500 2024-01-31T05:03:24.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.635+0000 7f5094c00640 1 -- 172.21.15.97:0/2124593950 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f5090019090 con 0x7f5098161500 2024-01-31T05:03:24.635 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:24.635 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":43,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:02:43.886790+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:02:29.547540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-01-31T05:02:30.723512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-01-31T05:02:41.538409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:24.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f507006aed0 msgr2=0x7f507006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f507006aed0 0x7f507006d390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5080036000 tx=0x7f5080030000 comp rx=0 tx=0).stop 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 msgr2=0x7f509815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 0x7f509815f0f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f509000b990 tx=0x7f509000be50 comp rx=0 tx=0).stop 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 shutdown_connections 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f507006aed0 0x7f507006d390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5098161500 0x7f509815f0f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f509815d520 0x7f509815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 --2- 172.21.15.97:0/2124593950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50980ce430 0x7f509814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 >> 172.21.15.97:0/2124593950 conn(0x7f50980b0a10 msgr2=0x7f50980b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 shutdown_connections 2024-01-31T05:03:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.638+0000 7f509f400640 1 -- 172.21.15.97:0/2124593950 wait complete. 2024-01-31T05:03:24.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-31T05:03:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.856+0000 7f2aa7c00640 1 Processor -- start 2024-01-31T05:03:24.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.856+0000 7f2aa7c00640 1 -- start start 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2aa7c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa015b470 0x7f2aa00722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2aa7c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2aa00cbf20 0x7f2aa00ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2aa7c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa006cd20 con 0x7f2aa00abcf0 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2aa7c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa00ce970 con 0x7f2aa00cbf20 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2aa7c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa015b850 con 0x7f2aa015b470 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2a9fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2aa00abcf0 0x7f2aa00cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49614/0 (socket says 172.21.15.97:49614) 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2aa5c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2aa00cbf20 0x7f2aa00ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.858+0000 7f2a9fe00640 1 -- 172.21.15.97:0/1172839173 learned_addr learned my addr 172.21.15.97:0/1172839173 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:24.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.859+0000 7f2aa5200640 1 --2- 172.21.15.97:0/1172839173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa015b470 0x7f2aa00722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.859+0000 7f2aa5c00640 1 -- 172.21.15.97:0/1172839173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa015b470 msgr2=0x7f2aa00722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.859+0000 7f2aa5c00640 1 --2- 172.21.15.97:0/1172839173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa015b470 0x7f2aa00722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.859+0000 7f2aa5c00640 1 -- 172.21.15.97:0/1172839173 >> v1:172.21.15.97:6789/0 conn(0x7f2aa00abcf0 legacy=0x7f2aa00cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.859+0000 7f2aa5c00640 1 -- 172.21.15.97:0/1172839173 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00ceb60 con 0x7f2aa00cbf20 2024-01-31T05:03:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.859+0000 7f2a9f400640 1 -- 172.21.15.97:0/1172839173 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895303471 0 0) 0x7f2aa006cd20 con 0x7f2aa00abcf0 2024-01-31T05:03:24.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.860+0000 7f2aa5c00640 1 --2- 172.21.15.97:0/1172839173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa00ce430 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009df0 tx=0x7f2a9802be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9bbad949a42014 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.860+0000 7f2a9f400640 1 -- 172.21.15.97:0/1172839173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9802cc20 con 0x7f2aa00cbf20 2024-01-31T05:03:24.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.860+0000 7f2a9f400640 1 -- 172.21.15.97:0/1172839173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9802cd80 con 0x7f2aa00cbf20 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.860+0000 7f2a9f400640 1 -- 172.21.15.97:0/1172839173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98034660 con 0x7f2aa00cbf20 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 -- 172.21.15.97:0/1172839173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 msgr2=0x7f2aa00ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/1172839173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa00ce430 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009df0 tx=0x7f2a9802be70 comp rx=0 tx=0).stop 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 -- 172.21.15.97:0/1172839173 shutdown_connections 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/1172839173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa00ce430 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/1172839173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa015b470 0x7f2aa00722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 -- 172.21.15.97:0/1172839173 >> 172.21.15.97:0/1172839173 conn(0x7f2aa00b0a10 msgr2=0x7f2aa00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 -- 172.21.15.97:0/1172839173 shutdown_connections 2024-01-31T05:03:24.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.861+0000 7f2aa7c00640 1 -- 172.21.15.97:0/1172839173 wait complete. 2024-01-31T05:03:24.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.862+0000 7f2aa7c00640 1 Processor -- start 2024-01-31T05:03:24.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 -- start start 2024-01-31T05:03:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa00abcf0 0x7f2aa00c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa0148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aa015b470 0x7f2aa0149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00c85d0 con 0x7f2aa00cbf20 2024-01-31T05:03:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0148e60 con 0x7f2aa015b470 2024-01-31T05:03:24.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.863+0000 7f2aa7c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00b3100 con 0x7f2aa00abcf0 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa0148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa0148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60774/0 (socket says 172.21.15.97:60774) 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 -- 172.21.15.97:0/804774077 learned_addr learned my addr 172.21.15.97:0/804774077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 -- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa00abcf0 msgr2=0x7f2aa00c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa00abcf0 0x7f2aa00c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 -- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aa015b470 msgr2=0x7f2aa0149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2aa5c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aa015b470 0x7f2aa0149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aa015b470 0x7f2aa0149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 -- 172.21.15.97:0/804774077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a98009a70 con 0x7f2aa00cbf20 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.864+0000 7f2a9fe00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa0148920 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c4b0 tx=0x7f2a8800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.865+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a880074e0 con 0x7f2aa00cbf20 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.865+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a88007660 con 0x7f2aa00cbf20 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.865+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88004a20 con 0x7f2aa00cbf20 2024-01-31T05:03:24.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.865+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa00b24f0 con 0x7f2aa00cbf20 2024-01-31T05:03:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.865+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa00b2ab0 con 0x7f2aa00cbf20 2024-01-31T05:03:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.866+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa0158b20 con 0x7f2aa00cbf20 2024-01-31T05:03:24.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.867+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2a88004b80 con 0x7f2aa00cbf20 2024-01-31T05:03:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.867+0000 7f2a9cc00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a7806aed0 0x7f2a7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.868+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f2a88088dd0 con 0x7f2aa00cbf20 2024-01-31T05:03:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.868+0000 7f2aa5200640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a7806aed0 0x7f2a7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:24.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.868+0000 7f2aa5200640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a7806aed0 0x7f2a7806d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2a90006e20 tx=0x7f2a90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:24.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:24.872+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2a88053a90 con 0x7f2aa00cbf20 2024-01-31T05:03:25.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.089+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2aa00b3270 con 0x7f2aa00cbf20 2024-01-31T05:03:25.089 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-01-31T05:03:25.089+0000 7f85e2800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:03:25.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.093+0000 7f2a9cc00640 1 -- 172.21.15.97:0/804774077 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2a88018020 con 0x7f2aa00cbf20 2024-01-31T05:03:25.092 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-31T05:03:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.095+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a7806aed0 msgr2=0x7f2a7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.095+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a7806aed0 0x7f2a7806d390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2a90006e20 tx=0x7f2a90008040 comp rx=0 tx=0).stop 2024-01-31T05:03:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.095+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 msgr2=0x7f2aa0148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:25.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.095+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa0148920 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800c4b0 tx=0x7f2a8800c970 comp rx=0 tx=0).stop 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 shutdown_connections 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2aa015b470 0x7f2aa0149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a7806aed0 0x7f2a7806d390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2aa00cbf20 0x7f2aa0148920 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 --2- 172.21.15.97:0/804774077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2aa00abcf0 0x7f2aa00c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 >> 172.21.15.97:0/804774077 conn(0x7f2aa00b0a10 msgr2=0x7f2aa00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.096+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 shutdown_connections 2024-01-31T05:03:25.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.097+0000 7f2aa7c00640 1 -- 172.21.15.97:0/804774077 wait complete. 2024-01-31T05:03:25.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-31T05:03:25.121 INFO:tasks.ceph.mds.b.smithi188.stderr: -3916> 2024-01-31T05:02:34.150+0000 7fc5b9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:03:25.217 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-31T05:03:25.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.317+0000 7ff405a00640 1 Processor -- start 2024-01-31T05:03:25.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.318+0000 7ff405a00640 1 -- start start 2024-01-31T05:03:25.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff405a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff40006d580 0x7ff40006d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:25.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff405a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff40015d690 0x7ff4000ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:25.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff405a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40006dea0 con 0x7ff40006c350 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff405a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40015e7c0 con 0x7ff40006d580 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff405a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40015db40 con 0x7ff40015d690 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff3fea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff40006d580 0x7ff40006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff3fea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff40006d580 0x7ff40006d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44178/0 (socket says 172.21.15.97:44178) 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.319+0000 7ff3fea00640 1 -- 172.21.15.97:0/2512105719 learned_addr learned my addr 172.21.15.97:0/2512105719 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.320+0000 7ff3fe000640 1 --2- 172.21.15.97:0/2512105719 >> v2:172.21.15.188:3300/0 conn(0x7ff40015d690 0x7ff4000ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.320+0000 7ff3fea00640 1 -- 172.21.15.97:0/2512105719 >> v2:172.21.15.188:3300/0 conn(0x7ff40015d690 msgr2=0x7ff4000ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.320+0000 7ff3fea00640 1 --2- 172.21.15.97:0/2512105719 >> v2:172.21.15.188:3300/0 conn(0x7ff40015d690 0x7ff4000ce430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.320+0000 7ff3fea00640 1 -- 172.21.15.97:0/2512105719 >> v1:172.21.15.188:6789/0 conn(0x7ff40006c350 legacy=0x7ff40006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:25.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.320+0000 7ff3fea00640 1 -- 172.21.15.97:0/2512105719 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40015de20 con 0x7ff40006d580 2024-01-31T05:03:25.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.321+0000 7ff3fea00640 1 --2- 172.21.15.97:0/2512105719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006d580 0x7ff40006d960 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff4000cedd0 tx=0x7ff3f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec29691974ebb02 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:25.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.321+0000 7ff3fd600640 1 -- 172.21.15.97:0/2512105719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004230 con 0x7ff40006d580 2024-01-31T05:03:25.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.321+0000 7ff3fd600640 1 -- 172.21.15.97:0/2512105719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0004390 con 0x7ff40006d580 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.321+0000 7ff3fd600640 1 -- 172.21.15.97:0/2512105719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003810 con 0x7ff40006d580 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 -- 172.21.15.97:0/2512105719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006d580 msgr2=0x7ff40006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 --2- 172.21.15.97:0/2512105719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006d580 0x7ff40006d960 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff4000cedd0 tx=0x7ff3f002b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 -- 172.21.15.97:0/2512105719 shutdown_connections 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 --2- 172.21.15.97:0/2512105719 >> v2:172.21.15.188:3300/0 conn(0x7ff40015d690 0x7ff4000ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 --2- 172.21.15.97:0/2512105719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006d580 0x7ff40006d960 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 -- 172.21.15.97:0/2512105719 >> 172.21.15.97:0/2512105719 conn(0x7ff4000b0a10 msgr2=0x7ff4000b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 -- 172.21.15.97:0/2512105719 shutdown_connections 2024-01-31T05:03:25.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.322+0000 7ff405a00640 1 -- 172.21.15.97:0/2512105719 wait complete. 2024-01-31T05:03:25.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.323+0000 7ff405a00640 1 Processor -- start 2024-01-31T05:03:25.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.324+0000 7ff405a00640 1 -- start start 2024-01-31T05:03:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.324+0000 7ff405a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 0x7ff4000abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.324+0000 7ff405a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff40006d580 0x7ff4000cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff405a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff40015d690 0x7ff4001443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff405a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff400099d10 con 0x7ff40006c350 2024-01-31T05:03:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff405a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000999c0 con 0x7ff40015d690 2024-01-31T05:03:25.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff405a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000ac5f0 con 0x7ff40006d580 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 0x7ff4000abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 0x7ff4000abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44194/0 (socket says 172.21.15.97:44194) 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 -- 172.21.15.97:0/4088257718 learned_addr learned my addr 172.21.15.97:0/4088257718 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 -- 172.21.15.97:0/4088257718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff40006d580 msgr2=0x7ff4000cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff40006d580 0x7ff4000cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 -- 172.21.15.97:0/4088257718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff40015d690 msgr2=0x7ff4001443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff40015d690 0x7ff4001443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 -- 172.21.15.97:0/4088257718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0009460 con 0x7ff40006c350 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.325+0000 7ff3fea00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 0x7ff4000abcf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff3f002b930 tx=0x7ff3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.326+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003c40 con 0x7ff40006c350 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.326+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f003f070 con 0x7ff40006c350 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.326+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004500 con 0x7ff40006c350 2024-01-31T05:03:25.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.326+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4000ac8d0 con 0x7ff40006c350 2024-01-31T05:03:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.326+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4000ac230 con 0x7ff40006c350 2024-01-31T05:03:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.327+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4000ac3c0 con 0x7ff40006c350 2024-01-31T05:03:25.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.328+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff3f0041030 con 0x7ff40006c350 2024-01-31T05:03:25.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.329+0000 7ff3f6a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff3d806aed0 0x7ff3d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:25.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.329+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6510+0+0 (secure 0 0 0) 0x7ff3f00abbd0 con 0x7ff40006c350 2024-01-31T05:03:25.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.329+0000 7ff3fe000640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff3d806aed0 0x7ff3d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:25.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.330+0000 7ff3fe000640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff3d806aed0 0x7ff3d806d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4000ae380 tx=0x7ff3e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:25.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.333+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff3f0076770 con 0x7ff40006c350 2024-01-31T05:03:25.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:25.561+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4000680a0 con 0x7ff40006c350 2024-01-31T05:03:26.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.105+0000 7ff3f6a00640 1 -- 172.21.15.97:0/4088257718 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff3f0079d90 con 0x7ff40006c350 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff3d806aed0 msgr2=0x7ff3d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff3d806aed0 0x7ff3d806d390 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff4000ae380 tx=0x7ff3e8008040 comp rx=0 tx=0).stop 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 msgr2=0x7ff4000abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 0x7ff4000abcf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff3f002b930 tx=0x7ff3f00046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 shutdown_connections 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff40015d690 0x7ff4001443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff40006d580 0x7ff4000cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff3d806aed0 0x7ff3d806d390 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 --2- 172.21.15.97:0/4088257718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40006c350 0x7ff4000abcf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.108+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 >> 172.21.15.97:0/4088257718 conn(0x7ff4000b0a10 msgr2=0x7ff4000cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.109+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 shutdown_connections 2024-01-31T05:03:26.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.109+0000 7ff405a00640 1 -- 172.21.15.97:0/4088257718 wait complete. 2024-01-31T05:03:26.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-31T05:03:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.338+0000 7f9147c00640 1 Processor -- start 2024-01-31T05:03:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.338+0000 7f9147c00640 1 -- start start 2024-01-31T05:03:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9147c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f914015a230 0x7f91400ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9147c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400cab10 con 0x7f91400c9430 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9147c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400c99f0 con 0x7f914006e1e0 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9147c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914006c720 con 0x7f914015a230 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9145200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f914015a230 0x7f91400ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9137c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f91400c9430 0x7f914006dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60324/0 (socket says 172.21.15.97:60324) 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.340+0000 7f9137c00640 1 -- 172.21.15.97:0/260572768 learned_addr learned my addr 172.21.15.97:0/260572768 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.341+0000 7f9145200640 1 -- 172.21.15.97:0/260572768 >> v1:172.21.15.188:6789/0 conn(0x7f914006e1e0 legacy=0x7f9140071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:26.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.341+0000 7f9145200640 1 -- 172.21.15.97:0/260572768 >> v1:172.21.15.97:6789/0 conn(0x7f91400c9430 legacy=0x7f914006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:26.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.341+0000 7f9145200640 1 -- 172.21.15.97:0/260572768 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400ae050 con 0x7f914015a230 2024-01-31T05:03:26.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.342+0000 7f9145200640 1 --2- 172.21.15.97:0/260572768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f91400ca5d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72661e0152e12e86 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:26.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.342+0000 7f913fe00640 1 -- 172.21.15.97:0/260572768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130032870 con 0x7f914015a230 2024-01-31T05:03:26.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.342+0000 7f913fe00640 1 -- 172.21.15.97:0/260572768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91300329d0 con 0x7f914015a230 2024-01-31T05:03:26.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.342+0000 7f913fe00640 1 -- 172.21.15.97:0/260572768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130032d60 con 0x7f914015a230 2024-01-31T05:03:26.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 -- 172.21.15.97:0/260572768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 msgr2=0x7f91400ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:26.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 --2- 172.21.15.97:0/260572768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f91400ca5d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002d9b0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 -- 172.21.15.97:0/260572768 shutdown_connections 2024-01-31T05:03:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 --2- 172.21.15.97:0/260572768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f91400ca5d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 -- 172.21.15.97:0/260572768 >> 172.21.15.97:0/260572768 conn(0x7f91400cede0 msgr2=0x7f91400cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 -- 172.21.15.97:0/260572768 shutdown_connections 2024-01-31T05:03:26.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.343+0000 7f9147c00640 1 -- 172.21.15.97:0/260572768 wait complete. 2024-01-31T05:03:26.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.345+0000 7f9147c00640 1 Processor -- start 2024-01-31T05:03:26.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.345+0000 7f9147c00640 1 -- start start 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9147c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914006e1e0 0x7f91400b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9147c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91400c9430 0x7f9140154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9147c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f9140154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9147c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400c6b30 con 0x7f914015a230 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9147c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400c6e10 con 0x7f914006e1e0 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9147c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400c70f0 con 0x7f91400c9430 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9145c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f9140154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:26.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9145c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f9140154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44204/0 (socket says 172.21.15.97:44204) 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.346+0000 7f9145c00640 1 -- 172.21.15.97:0/1688479885 learned_addr learned my addr 172.21.15.97:0/1688479885 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9137c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91400c9430 0x7f9140154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9145c00640 1 -- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91400c9430 msgr2=0x7f9140154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9145c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91400c9430 0x7f9140154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9145c00640 1 -- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914006e1e0 msgr2=0x7f91400b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9145c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914006e1e0 0x7f91400b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:26.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9145c00640 1 -- 172.21.15.97:0/1688479885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9130009400 con 0x7f914015a230 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9145c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f9140154a50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f913800c780 tx=0x7f913800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138004b10 con 0x7f914015a230 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138004c70 con 0x7f914015a230 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800aa90 con 0x7f914015a230 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914009b1c0 con 0x7f914015a230 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.347+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914009ade0 con 0x7f914015a230 2024-01-31T05:03:26.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.348+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914009b3e0 con 0x7f914015a230 2024-01-31T05:03:26.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.349+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f913800abf0 con 0x7f914015a230 2024-01-31T05:03:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.350+0000 7f913d600640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f911806aed0 0x7f911806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.350+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6510+0+0 (secure 0 0 0) 0x7f9138089ad0 con 0x7f914015a230 2024-01-31T05:03:26.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.351+0000 7f9145200640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f911806aed0 0x7f911806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:26.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.351+0000 7f9145200640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f911806aed0 0x7f911806d390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9130038000 tx=0x7f9130032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:26.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.355+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9138054660 con 0x7f914015a230 2024-01-31T05:03:26.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:26.545+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9140067830 con 0x7f914015a230 2024-01-31T05:03:27.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.108+0000 7f913d600640 1 -- 172.21.15.97:0/1688479885 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9138057c80 con 0x7f914015a230 2024-01-31T05:03:27.108 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-01-31T05:03:27.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f911806aed0 msgr2=0x7f911806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:27.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f911806aed0 0x7f911806d390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9130038000 tx=0x7f9130032000 comp rx=0 tx=0).stop 2024-01-31T05:03:27.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 msgr2=0x7f9140154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:27.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f9140154a50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f913800c780 tx=0x7f913800cc40 comp rx=0 tx=0).stop 2024-01-31T05:03:27.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 shutdown_connections 2024-01-31T05:03:27.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f911806aed0 0x7f911806d390 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f914015a230 0x7f9140154a50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91400c9430 0x7f9140154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 --2- 172.21.15.97:0/1688479885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914006e1e0 0x7f91400b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 >> 172.21.15.97:0/1688479885 conn(0x7f91400cede0 msgr2=0x7f91400ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:27.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 shutdown_connections 2024-01-31T05:03:27.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.112+0000 7f9147c00640 1 -- 172.21.15.97:0/1688479885 wait complete. 2024-01-31T05:03:27.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-31T05:03:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.362+0000 7feca8c00640 1 Processor -- start 2024-01-31T05:03:27.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.362+0000 7feca8c00640 1 -- start start 2024-01-31T05:03:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca8c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca40721f0 0x7feca4070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca8c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feca40c9280 0x7feca40c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca8c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca406e730 con 0x7feca415ed30 2024-01-31T05:03:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca8c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca4155c10 con 0x7feca40c9280 2024-01-31T05:03:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca8c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca40726a0 con 0x7feca40721f0 2024-01-31T05:03:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feca40c9280 0x7feca40c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feca40c9280 0x7feca40c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44220/0 (socket says 172.21.15.97:44220) 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feca415ed30 0x7feca415f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60338/0 (socket says 172.21.15.97:60338) 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 -- 172.21.15.97:0/3832522562 learned_addr learned my addr 172.21.15.97:0/3832522562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7fec9a800640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca40721f0 0x7feca4070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 -- 172.21.15.97:0/3832522562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca40721f0 msgr2=0x7feca4070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca40721f0 0x7feca4070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 -- 172.21.15.97:0/3832522562 >> v1:172.21.15.97:6789/0 conn(0x7feca415ed30 legacy=0x7feca415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca2a00640 1 -- 172.21.15.97:0/3832522562 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006819656 0 0) 0x7feca406e730 con 0x7feca415ed30 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7feca3e00640 1 -- 172.21.15.97:0/3832522562 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca406e450 con 0x7feca40c9280 2024-01-31T05:03:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.364+0000 7fec9a800640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca40721f0 0x7feca4070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.365+0000 7feca3e00640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40c9280 0x7feca40c9710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009870 tx=0x7fec9c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edaa7a60eff6d67 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca2a00640 1 -- 172.21.15.97:0/3832522562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c031640 con 0x7feca40c9280 2024-01-31T05:03:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca2a00640 1 -- 172.21.15.97:0/3832522562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c0317a0 con 0x7feca40c9280 2024-01-31T05:03:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca2a00640 1 -- 172.21.15.97:0/3832522562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c031b30 con 0x7feca40c9280 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca8c00640 1 -- 172.21.15.97:0/3832522562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40c9280 msgr2=0x7feca40c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca8c00640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40c9280 0x7feca40c9710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009870 tx=0x7fec9c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca8c00640 1 -- 172.21.15.97:0/3832522562 shutdown_connections 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca8c00640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40c9280 0x7feca40c9710 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca8c00640 1 --2- 172.21.15.97:0/3832522562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca40721f0 0x7feca4070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.366+0000 7feca8c00640 1 -- 172.21.15.97:0/3832522562 >> 172.21.15.97:0/3832522562 conn(0x7feca40cec30 msgr2=0x7feca40cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.367+0000 7feca8c00640 1 -- 172.21.15.97:0/3832522562 shutdown_connections 2024-01-31T05:03:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.367+0000 7feca8c00640 1 -- 172.21.15.97:0/3832522562 wait complete. 2024-01-31T05:03:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.368+0000 7feca8c00640 1 Processor -- start 2024-01-31T05:03:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.368+0000 7feca8c00640 1 -- start start 2024-01-31T05:03:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 0x7feca414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca8c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feca40c9280 0x7feca41f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca8c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca415ed30 0x7feca40c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca8c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca41506b0 con 0x7feca40721f0 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca8c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca4150990 con 0x7feca40c9280 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca8c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca4150c70 con 0x7feca415ed30 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.369+0000 7feca3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 0x7feca414fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 0x7feca414fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44226/0 (socket says 172.21.15.97:44226) 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 -- 172.21.15.97:0/2030798786 learned_addr learned my addr 172.21.15.97:0/2030798786 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7fec9a800640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feca40c9280 0x7feca41f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3e00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca415ed30 0x7feca40c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 -- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca415ed30 msgr2=0x7feca40c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca415ed30 0x7feca40c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 -- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feca40c9280 msgr2=0x7feca41f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feca40c9280 0x7feca41f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 -- 172.21.15.97:0/2030798786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c0094d0 con 0x7feca40721f0 2024-01-31T05:03:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7fec9a800640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feca40c9280 0x7feca41f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.370+0000 7feca3400640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 0x7feca414fe60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fec9400c890 tx=0x7fec9400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.371+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400a640 con 0x7feca40721f0 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.371+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9400a7c0 con 0x7feca40721f0 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.371+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca4150f50 con 0x7feca40721f0 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.371+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400ab70 con 0x7feca40721f0 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.371+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca41f27d0 con 0x7feca40721f0 2024-01-31T05:03:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.372+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca40676b0 con 0x7feca40721f0 2024-01-31T05:03:27.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.373+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fec9400acd0 con 0x7feca40721f0 2024-01-31T05:03:27.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.374+0000 7fec9be00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fec7c06aed0 0x7fec7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:27.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.374+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6072+0+0 (secure 0 0 0) 0x7fec940890e0 con 0x7feca40721f0 2024-01-31T05:03:27.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.374+0000 7fec9a800640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fec7c06aed0 0x7fec7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:27.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.375+0000 7fec9a800640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fec7c06aed0 0x7fec7c06d390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec8c006fd0 tx=0x7fec8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:27.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.379+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fec94053e40 con 0x7feca40721f0 2024-01-31T05:03:27.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:27.570+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feca40ac840 con 0x7feca40721f0 2024-01-31T05:03:28.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.112+0000 7fec9be00640 1 -- 172.21.15.97:0/2030798786 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7fec94057460 con 0x7feca40721f0 2024-01-31T05:03:28.112 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-01-31T05:03:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.115+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fec7c06aed0 msgr2=0x7fec7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.115+0000 7feca8c00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fec7c06aed0 0x7fec7c06d390 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec8c006fd0 tx=0x7fec8c008040 comp rx=0 tx=0).stop 2024-01-31T05:03:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.115+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 msgr2=0x7feca414fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.115+0000 7feca8c00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 0x7feca414fe60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fec9400c890 tx=0x7fec9400cd50 comp rx=0 tx=0).stop 2024-01-31T05:03:28.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 shutdown_connections 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feca415ed30 0x7feca40c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feca40c9280 0x7feca41f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fec7c06aed0 0x7fec7c06d390 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 --2- 172.21.15.97:0/2030798786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca40721f0 0x7feca414fe60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 >> 172.21.15.97:0/2030798786 conn(0x7feca40cec30 msgr2=0x7feca415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 shutdown_connections 2024-01-31T05:03:28.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.116+0000 7feca8c00640 1 -- 172.21.15.97:0/2030798786 wait complete. 2024-01-31T05:03:28.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-01-31T05:03:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.354+0000 7fc80fe00640 1 Processor -- start 2024-01-31T05:03:28.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.355+0000 7fc80fe00640 1 -- start start 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8080c1db0 0x7fc808071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8080ce6e0 0x7fc8080c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8080c26d0 con 0x7fc8080cd070 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80fe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc808072750 con 0x7fc8080ce6e0 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8080713c0 con 0x7fc8080c1db0 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80ca00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8080ce6e0 0x7fc8080c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80ca00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8080ce6e0 0x7fc8080c2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44236/0 (socket says 172.21.15.97:44236) 2024-01-31T05:03:28.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.356+0000 7fc80ca00640 1 -- 172.21.15.97:0/1881383308 learned_addr learned my addr 172.21.15.97:0/1881383308 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:28.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.357+0000 7fc80d400640 1 --2- 172.21.15.97:0/1881383308 >> v2:172.21.15.188:3300/0 conn(0x7fc8080c1db0 0x7fc808071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:28.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.357+0000 7fc80ca00640 1 -- 172.21.15.97:0/1881383308 >> v2:172.21.15.188:3300/0 conn(0x7fc8080c1db0 msgr2=0x7fc808071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.357+0000 7fc80ca00640 1 --2- 172.21.15.97:0/1881383308 >> v2:172.21.15.188:3300/0 conn(0x7fc8080c1db0 0x7fc808071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.357+0000 7fc80ca00640 1 -- 172.21.15.97:0/1881383308 >> v1:172.21.15.188:6789/0 conn(0x7fc8080cd070 legacy=0x7fc808161e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.357+0000 7fc80ca00640 1 -- 172.21.15.97:0/1881383308 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808071eb0 con 0x7fc8080ce6e0 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.358+0000 7fc80ca00640 1 --2- 172.21.15.97:0/1881383308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080ce6e0 0x7fc8080c2190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009870 tx=0x7fc7f002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f01ca3d2103b8cd server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.358+0000 7fc807e00640 1 -- 172.21.15.97:0/1881383308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0031860 con 0x7fc8080ce6e0 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc807e00640 1 -- 172.21.15.97:0/1881383308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f00319c0 con 0x7fc8080ce6e0 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc807e00640 1 -- 172.21.15.97:0/1881383308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0031d50 con 0x7fc8080ce6e0 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 -- 172.21.15.97:0/1881383308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080ce6e0 msgr2=0x7fc8080c2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 --2- 172.21.15.97:0/1881383308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080ce6e0 0x7fc8080c2190 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0009870 tx=0x7fc7f002b900 comp rx=0 tx=0).stop 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 -- 172.21.15.97:0/1881383308 shutdown_connections 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 --2- 172.21.15.97:0/1881383308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080ce6e0 0x7fc8080c2190 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 --2- 172.21.15.97:0/1881383308 >> v2:172.21.15.188:3300/0 conn(0x7fc8080c1db0 0x7fc808071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 -- 172.21.15.97:0/1881383308 >> 172.21.15.97:0/1881383308 conn(0x7fc8080cede0 msgr2=0x7fc8080cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:28.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 -- 172.21.15.97:0/1881383308 shutdown_connections 2024-01-31T05:03:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.359+0000 7fc80fe00640 1 -- 172.21.15.97:0/1881383308 wait complete. 2024-01-31T05:03:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.361+0000 7fc80fe00640 1 Processor -- start 2024-01-31T05:03:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.361+0000 7fc80fe00640 1 -- start start 2024-01-31T05:03:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080c1db0 0x7fc8080c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 0x7fc808150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080ce6e0 0x7fc8080c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080c3110 con 0x7fc8080cd070 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080c33f0 con 0x7fc8080c1db0 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080c36d0 con 0x7fc8080ce6e0 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 0x7fc808150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 0x7fc808150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44240/0 (socket says 172.21.15.97:44240) 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.362+0000 7fc80ca00640 1 -- 172.21.15.97:0/2319423016 learned_addr learned my addr 172.21.15.97:0/2319423016 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80d400640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080c1db0 0x7fc8080c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80de00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080ce6e0 0x7fc8080c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80ca00640 1 -- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080ce6e0 msgr2=0x7fc8080c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80ca00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080ce6e0 0x7fc8080c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80ca00640 1 -- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080c1db0 msgr2=0x7fc8080c28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80ca00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080c1db0 0x7fc8080c28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80ca00640 1 -- 172.21.15.97:0/2319423016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00094d0 con 0x7fc8080cd070 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80d400640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080c1db0 0x7fc8080c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80ca00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 0x7fc808150200 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00090b0 tx=0x7fc7f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0031810 con 0x7fc8080cd070 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f00443f0 con 0x7fc8080cd070 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00447a0 con 0x7fc8080cd070 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.363+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8080c39b0 con 0x7fc8080cd070 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.364+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8080c2e00 con 0x7fc8080cd070 2024-01-31T05:03:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.365+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8080c3ed0 con 0x7fc8080cd070 2024-01-31T05:03:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.365+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc7f0044900 con 0x7fc8080cd070 2024-01-31T05:03:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.367+0000 7fc805600640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc7e006af00 0x7fc7e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.367+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5658+0+0 (secure 0 0 0) 0x7fc7f00abf10 con 0x7fc8080cd070 2024-01-31T05:03:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.367+0000 7fc80d400640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc7e006af00 0x7fc7e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.368+0000 7fc80d400640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc7e006af00 0x7fc7e006d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800b820 tx=0x7fc7f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.371+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc8080c3ed0 con 0x7fc8080cd070 2024-01-31T05:03:28.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:28.565+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8080c3ed0 con 0x7fc8080cd070 2024-01-31T05:03:29.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.138+0000 7fc805600640 1 -- 172.21.15.97:0/2319423016 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v47) v1 ==== 160+0+0 (secure 0 0 0) 0x7fc8080c3ed0 con 0x7fc8080cd070 2024-01-31T05:03:29.138 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'data_new' removed 2024-01-31T05:03:29.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.142+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc7e006af00 msgr2=0x7fc7e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.142+0000 7fc80fe00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc7e006af00 0x7fc7e006d3c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800b820 tx=0x7fc7f8008040 comp rx=0 tx=0).stop 2024-01-31T05:03:29.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.142+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 msgr2=0x7fc808150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.142+0000 7fc80fe00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 0x7fc808150200 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00090b0 tx=0x7fc7f00046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 shutdown_connections 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8080ce6e0 0x7fc8080c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc7e006af00 0x7fc7e006d3c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8080cd070 0x7fc808150200 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 --2- 172.21.15.97:0/2319423016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8080c1db0 0x7fc8080c28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 >> 172.21.15.97:0/2319423016 conn(0x7fc8080cede0 msgr2=0x7fc80806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 shutdown_connections 2024-01-31T05:03:29.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.143+0000 7fc80fe00640 1 -- 172.21.15.97:0/2319423016 wait complete. 2024-01-31T05:03:29.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-31T05:03:29.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.379+0000 7fb942000640 1 Processor -- start 2024-01-31T05:03:29.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.380+0000 7fb942000640 1 -- start start 2024-01-31T05:03:29.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb942000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb93c15d590 0x7fb93c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb942000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb942000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb93c161080 0x7fb93c161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb942000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93c0d0460 con 0x7fb93c161080 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb942000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93c0cded0 con 0x7fb93c15d590 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb942000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c15d970 con 0x7fb93c0cfae0 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb93be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb93c161080 0x7fb93c161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb93be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb93c161080 0x7fb93c161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44252/0 (socket says 172.21.15.97:44252) 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb93be00640 1 -- 172.21.15.97:0/90375498 learned_addr learned my addr 172.21.15.97:0/90375498 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb93aa00640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.381+0000 7fb93b400640 1 --2- 172.21.15.97:0/90375498 >> v2:172.21.15.188:3300/0 conn(0x7fb93c15d590 0x7fb93c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93be00640 1 -- 172.21.15.97:0/90375498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 msgr2=0x7fb93c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93be00640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93be00640 1 -- 172.21.15.97:0/90375498 >> v2:172.21.15.188:3300/0 conn(0x7fb93c15d590 msgr2=0x7fb93c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93be00640 1 --2- 172.21.15.97:0/90375498 >> v2:172.21.15.188:3300/0 conn(0x7fb93c15d590 0x7fb93c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93be00640 1 -- 172.21.15.97:0/90375498 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c0cf210 con 0x7fb93c161080 2024-01-31T05:03:29.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93aa00640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.382+0000 7fb93b400640 1 --2- 172.21.15.97:0/90375498 >> v2:172.21.15.188:3300/0 conn(0x7fb93c15d590 0x7fb93c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.383+0000 7fb93be00640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c161510 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb93c08b830 tx=0x7fb93402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c17a280b718898 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.383+0000 7fb93a000640 1 -- 172.21.15.97:0/90375498 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93402cb00 con 0x7fb93c161080 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.383+0000 7fb93a000640 1 -- 172.21.15.97:0/90375498 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93402cc60 con 0x7fb93c161080 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.383+0000 7fb93a000640 1 -- 172.21.15.97:0/90375498 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340343f0 con 0x7fb93c161080 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 -- 172.21.15.97:0/90375498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 msgr2=0x7fb93c161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c161510 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb93c08b830 tx=0x7fb93402bf60 comp rx=0 tx=0).stop 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 -- 172.21.15.97:0/90375498 shutdown_connections 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c161510 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 --2- 172.21.15.97:0/90375498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 --2- 172.21.15.97:0/90375498 >> v2:172.21.15.188:3300/0 conn(0x7fb93c15d590 0x7fb93c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 -- 172.21.15.97:0/90375498 >> 172.21.15.97:0/90375498 conn(0x7fb93c09c560 msgr2=0x7fb93c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 -- 172.21.15.97:0/90375498 shutdown_connections 2024-01-31T05:03:29.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.384+0000 7fb942000640 1 -- 172.21.15.97:0/90375498 wait complete. 2024-01-31T05:03:29.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.386+0000 7fb942000640 1 Processor -- start 2024-01-31T05:03:29.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.386+0000 7fb942000640 1 -- start start 2024-01-31T05:03:29.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb942000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c15dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb942000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93c15d590 0x7fb93c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb942000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb942000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1f6380 con 0x7fb93c161080 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb942000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1f6660 con 0x7fb93c15d590 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb942000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1f6940 con 0x7fb93c0cfae0 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb93be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.387+0000 7fb93be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44262/0 (socket says 172.21.15.97:44262) 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 -- 172.21.15.97:0/2019639014 learned_addr learned my addr 172.21.15.97:0/2019639014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93b400640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c15dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93aa00640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93c15d590 0x7fb93c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 -- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 msgr2=0x7fb93c15dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c15dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 -- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93c15d590 msgr2=0x7fb93c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93c15d590 0x7fb93c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 -- 172.21.15.97:0/2019639014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb934009b30 con 0x7fb93c161080 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.388+0000 7fb93be00640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c0ccca0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb934036040 tx=0x7fb934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.389+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934009510 con 0x7fb93c161080 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.389+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93403e450 con 0x7fb93c161080 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.389+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c1f6c20 con 0x7fb93c161080 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.389+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93403e800 con 0x7fb93c161080 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.389+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c0c7a70 con 0x7fb93c161080 2024-01-31T05:03:29.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.390+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c0ce9d0 con 0x7fb93c161080 2024-01-31T05:03:29.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.391+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb93403e960 con 0x7fb93c161080 2024-01-31T05:03:29.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.392+0000 7fb933400640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb91406aed0 0x7fb91406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.392+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb9340ac3c0 con 0x7fb93c161080 2024-01-31T05:03:29.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.392+0000 7fb93b400640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb91406aed0 0x7fb91406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.392+0000 7fb93b400640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb91406aed0 0x7fb91406d390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00a910 tx=0x7fb92c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:29.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.396+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb934077580 con 0x7fb93c161080 2024-01-31T05:03:29.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.588+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb93c1f64c0 con 0x7fb93c161080 2024-01-31T05:03:29.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.594+0000 7fb933400640 1 -- 172.21.15.97:0/2019639014 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb93c1f64c0 con 0x7fb93c161080 2024-01-31T05:03:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.596+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb91406aed0 msgr2=0x7fb91406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.596+0000 7fb942000640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb91406aed0 0x7fb91406d390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00a910 tx=0x7fb92c008040 comp rx=0 tx=0).stop 2024-01-31T05:03:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.596+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 msgr2=0x7fb93c0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.596+0000 7fb942000640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c0ccca0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb934036040 tx=0x7fb934004a40 comp rx=0 tx=0).stop 2024-01-31T05:03:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 shutdown_connections 2024-01-31T05:03:29.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb91406aed0 0x7fb91406d390 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb93c161080 0x7fb93c0ccca0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb93c15d590 0x7fb93c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 --2- 172.21.15.97:0/2019639014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb93c0cfae0 0x7fb93c15dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 >> 172.21.15.97:0/2019639014 conn(0x7fb93c09c560 msgr2=0x7fb93c15f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 shutdown_connections 2024-01-31T05:03:29.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.597+0000 7fb942000640 1 -- 172.21.15.97:0/2019639014 wait complete. 2024-01-31T05:03:29.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-01-31T05:03:29.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.812+0000 7f8a12600640 1 Processor -- start 2024-01-31T05:03:29.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.813+0000 7f8a12600640 1 -- start start 2024-01-31T05:03:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.814+0000 7f8a12600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c0caf00 0x7f8a0c0b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.814+0000 7f8a12600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a0c15d440 0x7f8a0c15d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.814+0000 7f8a12600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0b22c0 con 0x7f8a0c159a50 2024-01-31T05:03:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.814+0000 7f8a12600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c15ddb0 con 0x7f8a0c15d440 2024-01-31T05:03:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.814+0000 7f8a12600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cddb0 con 0x7f8a0c0caf00 2024-01-31T05:03:29.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a0c15d440 0x7f8a0c15d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0b400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8a0c159a50 0x7f8a0c0cb650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60342/0 (socket says 172.21.15.97:60342) 2024-01-31T05:03:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0b400640 1 -- 172.21.15.97:0/1491523587 learned_addr learned my addr 172.21.15.97:0/1491523587 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0be00640 1 -- 172.21.15.97:0/1491523587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c0caf00 msgr2=0x7f8a0c0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:29.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a02800640 1 --2- 172.21.15.97:0/1491523587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c0caf00 0x7f8a0c0b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0aa00640 1 -- 172.21.15.97:0/1491523587 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440142820 0 0) 0x7f8a0c0b22c0 con 0x7f8a0c159a50 2024-01-31T05:03:29.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0be00640 1 --2- 172.21.15.97:0/1491523587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c0caf00 0x7f8a0c0b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0be00640 1 -- 172.21.15.97:0/1491523587 >> v1:172.21.15.97:6789/0 conn(0x7f8a0c159a50 legacy=0x7f8a0c0cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.815+0000 7f8a0be00640 1 -- 172.21.15.97:0/1491523587 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0ce090 con 0x7f8a0c15d440 2024-01-31T05:03:29.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.816+0000 7f8a0be00640 1 --2- 172.21.15.97:0/1491523587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c15d440 0x7f8a0c15d870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05dcb42fa23d30f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:29.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a0aa00640 1 -- 172.21.15.97:0/1491523587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402cc50 con 0x7f8a0c15d440 2024-01-31T05:03:29.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a0aa00640 1 -- 172.21.15.97:0/1491523587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0402cdb0 con 0x7f8a0c15d440 2024-01-31T05:03:29.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a0aa00640 1 -- 172.21.15.97:0/1491523587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04034660 con 0x7f8a0c15d440 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 -- 172.21.15.97:0/1491523587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c15d440 msgr2=0x7f8a0c15d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 --2- 172.21.15.97:0/1491523587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c15d440 0x7f8a0c15d870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0402bea0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 -- 172.21.15.97:0/1491523587 shutdown_connections 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 --2- 172.21.15.97:0/1491523587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c15d440 0x7f8a0c15d870 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 --2- 172.21.15.97:0/1491523587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c0caf00 0x7f8a0c0b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 -- 172.21.15.97:0/1491523587 >> 172.21.15.97:0/1491523587 conn(0x7f8a0c0b0a10 msgr2=0x7f8a0c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 -- 172.21.15.97:0/1491523587 shutdown_connections 2024-01-31T05:03:29.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.817+0000 7f8a12600640 1 -- 172.21.15.97:0/1491523587 wait complete. 2024-01-31T05:03:29.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.819+0000 7f8a12600640 1 Processor -- start 2024-01-31T05:03:29.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.819+0000 7f8a12600640 1 -- start start 2024-01-31T05:03:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a12600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 0x7f8a0c1f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a12600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a0c159a50 0x7f8a0c0c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a12600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c15d440 0x7f8a0c0ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a12600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0c8090 con 0x7f8a0c0caf00 2024-01-31T05:03:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a12600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1f7650 con 0x7f8a0c159a50 2024-01-31T05:03:29.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a12600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0c8270 con 0x7f8a0c15d440 2024-01-31T05:03:29.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a0b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 0x7f8a0c1f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a0b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 0x7f8a0c1f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44290/0 (socket says 172.21.15.97:44290) 2024-01-31T05:03:29.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.820+0000 7f8a0b400640 1 -- 172.21.15.97:0/4113601047 learned_addr learned my addr 172.21.15.97:0/4113601047 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a0b400640 1 -- 172.21.15.97:0/4113601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c15d440 msgr2=0x7f8a0c0ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a0b400640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c15d440 0x7f8a0c0ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a0b400640 1 -- 172.21.15.97:0/4113601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a0c159a50 msgr2=0x7f8a0c0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a0b400640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a0c159a50 0x7f8a0c0c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a0b400640 1 -- 172.21.15.97:0/4113601047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009a70 con 0x7f8a0c0caf00 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a0b400640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 0x7f8a0c1f7110 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c4a0 tx=0x7f89fc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:29.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc005ce0 con 0x7f8a0c0caf00 2024-01-31T05:03:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc005e40 con 0x7f8a0c0caf00 2024-01-31T05:03:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0072b0 con 0x7f8a0c0caf00 2024-01-31T05:03:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.821+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0c8550 con 0x7f8a0c0caf00 2024-01-31T05:03:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.822+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c0c8b10 con 0x7f8a0c0caf00 2024-01-31T05:03:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.823+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c0677c0 con 0x7f8a0c0caf00 2024-01-31T05:03:29.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.824+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f89fc007410 con 0x7f8a0c0caf00 2024-01-31T05:03:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.824+0000 7f8a03e00640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f89e406aed0 0x7f89e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.825+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f89fc088a30 con 0x7f8a0c0caf00 2024-01-31T05:03:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.825+0000 7f8a02800640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f89e406aed0 0x7f89e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:29.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.825+0000 7f8a02800640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f89e406aed0 0x7f89e406d390 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c15cf40 tx=0x7f89f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:29.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:29.829+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f89fc053ac0 con 0x7f8a0c0caf00 2024-01-31T05:03:30.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.095+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f8a0c09ca80 con 0x7f8a0c0caf00 2024-01-31T05:03:30.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.142+0000 7f8a03e00640 1 -- 172.21.15.97:0/4113601047 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v166) v1 ==== 147+0+0 (secure 0 0 0) 0x7f89fc017020 con 0x7f8a0c0caf00 2024-01-31T05:03:30.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f89e406aed0 msgr2=0x7f89e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f89e406aed0 0x7f89e406d390 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c15cf40 tx=0x7f89f4008040 comp rx=0 tx=0).stop 2024-01-31T05:03:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 msgr2=0x7f8a0c1f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 0x7f8a0c1f7110 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c4a0 tx=0x7f89fc00c960 comp rx=0 tx=0).stop 2024-01-31T05:03:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 shutdown_connections 2024-01-31T05:03:30.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a0c15d440 0x7f8a0c0ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a0c159a50 0x7f8a0c0c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f89e406aed0 0x7f89e406d390 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 --2- 172.21.15.97:0/4113601047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c0caf00 0x7f8a0c1f7110 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.145+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 >> 172.21.15.97:0/4113601047 conn(0x7f8a0c0b0a10 msgr2=0x7f8a0c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.146+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 shutdown_connections 2024-01-31T05:03:30.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.146+0000 7f8a12600640 1 -- 172.21.15.97:0/4113601047 wait complete. 2024-01-31T05:03:30.167 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-01-31T05:03:30.168 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-31T05:03:30.168 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:30.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.346+0000 7f5d9be00640 1 Processor -- start 2024-01-31T05:03:30.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.346+0000 7f5d9be00640 1 -- start start 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d9be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d940cb810 0x7f5d940cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9415b400 0x7f5d9406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d9be00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9406d1a0 con 0x7f5d9406db70 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d9be00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d940cc960 con 0x7f5d940cb810 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d9be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9406c600 con 0x7f5d9415b400 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d99400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d940cb810 0x7f5d940cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d99e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d9406db70 0x7f5d940ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60352/0 (socket says 172.21.15.97:60352) 2024-01-31T05:03:30.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.348+0000 7f5d99e00640 1 -- 172.21.15.97:0/1167215599 learned_addr learned my addr 172.21.15.97:0/1167215599 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d98a00640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9415b400 0x7f5d9406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d99400640 1 -- 172.21.15.97:0/1167215599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9415b400 msgr2=0x7f5d9406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d99400640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9415b400 0x7f5d9406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d99400640 1 -- 172.21.15.97:0/1167215599 >> v1:172.21.15.97:6789/0 conn(0x7f5d9406db70 legacy=0x7f5d940ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d99400640 1 -- 172.21.15.97:0/1167215599 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940cc130 con 0x7f5d940cb810 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d93e00640 1 -- 172.21.15.97:0/1167215599 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264434299 0 0) 0x7f5d9406d1a0 con 0x7f5d9406db70 2024-01-31T05:03:30.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.349+0000 7f5d98a00640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9415b400 0x7f5d9406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.350+0000 7f5d99400640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d940cbbf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5d840097a0 tx=0x7f5d8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5eab9d7b32f798 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.350+0000 7f5d93e00640 1 -- 172.21.15.97:0/1167215599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8402d040 con 0x7f5d940cb810 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.350+0000 7f5d93e00640 1 -- 172.21.15.97:0/1167215599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84031810 con 0x7f5d940cb810 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.350+0000 7f5d93e00640 1 -- 172.21.15.97:0/1167215599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84031ba0 con 0x7f5d940cb810 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 -- 172.21.15.97:0/1167215599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 msgr2=0x7f5d940cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d940cbbf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5d840097a0 tx=0x7f5d8402b8b0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 -- 172.21.15.97:0/1167215599 shutdown_connections 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9415b400 0x7f5d9406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 --2- 172.21.15.97:0/1167215599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d940cbbf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 -- 172.21.15.97:0/1167215599 >> 172.21.15.97:0/1167215599 conn(0x7f5d940b0a10 msgr2=0x7f5d940b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 -- 172.21.15.97:0/1167215599 shutdown_connections 2024-01-31T05:03:30.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.351+0000 7f5d9be00640 1 -- 172.21.15.97:0/1167215599 wait complete. 2024-01-31T05:03:30.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.353+0000 7f5d9be00640 1 Processor -- start 2024-01-31T05:03:30.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.353+0000 7f5d9be00640 1 -- start start 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9406db70 0x7f5d94152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d941f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9415b400 0x7f5d9415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d9be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94152f90 con 0x7f5d940cb810 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d9be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940b2c50 con 0x7f5d9415b400 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d9be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941f3e20 con 0x7f5d9406db70 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d98a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d941f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d98a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d941f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44302/0 (socket says 172.21.15.97:44302) 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d98a00640 1 -- 172.21.15.97:0/2571419433 learned_addr learned my addr 172.21.15.97:0/2571419433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.354+0000 7f5d99400640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9406db70 0x7f5d94152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d99e00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9415b400 0x7f5d9415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d98a00640 1 -- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9406db70 msgr2=0x7f5d94152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d98a00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9406db70 0x7f5d94152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d98a00640 1 -- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9415b400 msgr2=0x7f5d9415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d98a00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9415b400 0x7f5d9415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d98a00640 1 -- 172.21.15.97:0/2571419433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84009400 con 0x7f5d940cb810 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d99400640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9406db70 0x7f5d94152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d99e00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9415b400 0x7f5d9415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d98a00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d941f38e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c7c0 tx=0x7f5d7c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007820 con 0x7f5d940cb810 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c007980 con 0x7f5d940cb810 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9415bd10 con 0x7f5d940cb810 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007d10 con 0x7f5d940cb810 2024-01-31T05:03:30.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.355+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9415c2d0 con 0x7f5d940cb810 2024-01-31T05:03:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.356+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940677c0 con 0x7f5d940cb810 2024-01-31T05:03:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.357+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5d7c0113f0 con 0x7f5d940cb810 2024-01-31T05:03:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.358+0000 7f5d91600640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d6c06aed0 0x7f5d6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.358+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5d7c0895d0 con 0x7f5d940cb810 2024-01-31T05:03:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.358+0000 7f5d99400640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d6c06aed0 0x7f5d6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.359+0000 7f5d99400640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d6c06aed0 0x7f5d6c06d390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d84036000 tx=0x7f5d84008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:30.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.362+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5d7c0545c0 con 0x7f5d940cb810 2024-01-31T05:03:30.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.540+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d940b1000 con 0x7f5d940cb810 2024-01-31T05:03:30.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.542+0000 7f5d91600640 1 -- 172.21.15.97:0/2571419433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13192 (secure 0 0 0) 0x7f5d7c057be0 con 0x7f5d940cb810 2024-01-31T05:03:30.541 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:30.541 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:29.127527+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6834/111752619":"2024-02-01T05:03:25.090378+0000","172.21.15.188:6832/111752619":"2024-02-01T05:03:25.090378+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:30.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d6c06aed0 msgr2=0x7f5d6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d6c06aed0 0x7f5d6c06d390 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d84036000 tx=0x7f5d84008030 comp rx=0 tx=0).stop 2024-01-31T05:03:30.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 msgr2=0x7f5d941f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d941f38e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00c7c0 tx=0x7f5d7c00cc80 comp rx=0 tx=0).stop 2024-01-31T05:03:30.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 shutdown_connections 2024-01-31T05:03:30.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9415b400 0x7f5d9415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5d6c06aed0 0x7f5d6c06d390 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d940cb810 0x7f5d941f38e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 --2- 172.21.15.97:0/2571419433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d9406db70 0x7f5d94152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.544+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 >> 172.21.15.97:0/2571419433 conn(0x7f5d940b0a10 msgr2=0x7f5d94070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.545+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 shutdown_connections 2024-01-31T05:03:30.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.545+0000 7f5d9be00640 1 -- 172.21.15.97:0/2571419433 wait complete. 2024-01-31T05:03:30.561 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:30.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:30.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.764+0000 7fd8b9200640 1 Processor -- start 2024-01-31T05:03:30.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.765+0000 7fd8b9200640 1 -- start start 2024-01-31T05:03:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b9200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8b415d590 0x7fd8b415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b9200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8b40cff20 0x7fd8b4161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b9200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40cded0 con 0x7fd8b40cfae0 2024-01-31T05:03:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b9200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b415d970 con 0x7fd8b415d590 2024-01-31T05:03:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b9200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40cf210 con 0x7fd8b40cff20 2024-01-31T05:03:30.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b1e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8b415d590 0x7fd8b415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b1400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8b40cfae0 0x7fd8b4161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60368/0 (socket says 172.21.15.97:60368) 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.766+0000 7fd8b1400640 1 -- 172.21.15.97:0/1371496530 learned_addr learned my addr 172.21.15.97:0/1371496530 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.767+0000 7fd8b2800640 1 --2- 172.21.15.97:0/1371496530 >> v2:172.21.15.188:3300/0 conn(0x7fd8b40cff20 0x7fd8b4161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.767+0000 7fd8b1e00640 1 -- 172.21.15.97:0/1371496530 >> v2:172.21.15.188:3300/0 conn(0x7fd8b40cff20 msgr2=0x7fd8b4161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.767+0000 7fd8b1e00640 1 --2- 172.21.15.97:0/1371496530 >> v2:172.21.15.188:3300/0 conn(0x7fd8b40cff20 0x7fd8b4161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.767+0000 7fd8b1e00640 1 -- 172.21.15.97:0/1371496530 >> v1:172.21.15.97:6789/0 conn(0x7fd8b40cfae0 legacy=0x7fd8b4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.767+0000 7fd8b1e00640 1 -- 172.21.15.97:0/1371496530 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40ce170 con 0x7fd8b415d590 2024-01-31T05:03:30.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.767+0000 7fd8b0a00640 1 -- 172.21.15.97:0/1371496530 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631349796 0 0) 0x7fd8b40cded0 con 0x7fd8b40cfae0 2024-01-31T05:03:30.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.768+0000 7fd8b1e00640 1 --2- 172.21.15.97:0/1371496530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b415d590 0x7fd8b415e540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40cdbd0 tx=0x7fd8a402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cf12296e27cbba server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.768+0000 7fd8b0a00640 1 -- 172.21.15.97:0/1371496530 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a402cd00 con 0x7fd8b415d590 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.768+0000 7fd8b0a00640 1 -- 172.21.15.97:0/1371496530 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a402ce60 con 0x7fd8b415d590 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.768+0000 7fd8b0a00640 1 -- 172.21.15.97:0/1371496530 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4034660 con 0x7fd8b415d590 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 -- 172.21.15.97:0/1371496530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b415d590 msgr2=0x7fd8b415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 --2- 172.21.15.97:0/1371496530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b415d590 0x7fd8b415e540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40cdbd0 tx=0x7fd8a402bf50 comp rx=0 tx=0).stop 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 -- 172.21.15.97:0/1371496530 shutdown_connections 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 --2- 172.21.15.97:0/1371496530 >> v2:172.21.15.188:3300/0 conn(0x7fd8b40cff20 0x7fd8b4161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 --2- 172.21.15.97:0/1371496530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b415d590 0x7fd8b415e540 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 -- 172.21.15.97:0/1371496530 >> 172.21.15.97:0/1371496530 conn(0x7fd8b409c560 msgr2=0x7fd8b40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:30.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 -- 172.21.15.97:0/1371496530 shutdown_connections 2024-01-31T05:03:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.769+0000 7fd8b9200640 1 -- 172.21.15.97:0/1371496530 wait complete. 2024-01-31T05:03:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.770+0000 7fd8b9200640 1 Processor -- start 2024-01-31T05:03:30.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.771+0000 7fd8b9200640 1 -- start start 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.771+0000 7fd8b9200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 0x7fd8b415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.771+0000 7fd8b9200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8b40cff20 0x7fd8b40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b9200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8b415d590 0x7fd8b40c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b9200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40ca2e0 con 0x7fd8b40cfae0 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b9200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40cab80 con 0x7fd8b415d590 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b9200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415b460 con 0x7fd8b40cff20 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 0x7fd8b415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 0x7fd8b415af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44316/0 (socket says 172.21.15.97:44316) 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 -- 172.21.15.97:0/2602477112 learned_addr learned my addr 172.21.15.97:0/2602477112 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:30.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 -- 172.21.15.97:0/2602477112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8b40cff20 msgr2=0x7fd8b40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8b40cff20 0x7fd8b40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 -- 172.21.15.97:0/2602477112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8b415d590 msgr2=0x7fd8b40c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8b415d590 0x7fd8b40c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 -- 172.21.15.97:0/2602477112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a4009a70 con 0x7fd8b40cfae0 2024-01-31T05:03:30.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.772+0000 7fd8b1e00640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 0x7fd8b415af20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4037040 tx=0x7fd8a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.773+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a403d600 con 0x7fd8b40cfae0 2024-01-31T05:03:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.773+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a403d760 con 0x7fd8b40cfae0 2024-01-31T05:03:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.773+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b40b08d0 con 0x7fd8b40cfae0 2024-01-31T05:03:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.773+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a403db10 con 0x7fd8b40cfae0 2024-01-31T05:03:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.773+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40ca810 con 0x7fd8b40cfae0 2024-01-31T05:03:30.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.774+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b4067830 con 0x7fd8b40cfae0 2024-01-31T05:03:30.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.775+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd8a403b040 con 0x7fd8b40cfae0 2024-01-31T05:03:30.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.776+0000 7fd8aa000640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd88c06aed0 0x7fd88c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:30.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.776+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd8a40ab560 con 0x7fd8b40cfae0 2024-01-31T05:03:30.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.776+0000 7fd8b1400640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd88c06aed0 0x7fd88c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:30.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.777+0000 7fd8b1400640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd88c06aed0 0x7fd88c06d390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd89c007fd0 tx=0x7fd89c007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.780+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd8a40765f0 con 0x7fd8b40cfae0 2024-01-31T05:03:30.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.957+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8b4073600 con 0x7fd8b40cfae0 2024-01-31T05:03:30.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.958+0000 7fd8aa000640 1 -- 172.21.15.97:0/2602477112 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8a4079c10 con 0x7fd8b40cfae0 2024-01-31T05:03:30.958 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd88c06aed0 msgr2=0x7fd88c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd88c06aed0 0x7fd88c06d390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd89c007fd0 tx=0x7fd89c007920 comp rx=0 tx=0).stop 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 msgr2=0x7fd8b415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 0x7fd8b415af20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4037040 tx=0x7fd8a4004a40 comp rx=0 tx=0).stop 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 shutdown_connections 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8b415d590 0x7fd8b40c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8b40cff20 0x7fd8b40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd88c06aed0 0x7fd88c06d390 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 --2- 172.21.15.97:0/2602477112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8b40cfae0 0x7fd8b415af20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 >> 172.21.15.97:0/2602477112 conn(0x7fd8b409c560 msgr2=0x7fd8b40b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:30.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 shutdown_connections 2024-01-31T05:03:30.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:30.961+0000 7fd8b9200640 1 -- 172.21.15.97:0/2602477112 wait complete. 2024-01-31T05:03:30.978 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-01-31T05:03:31.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.171+0000 7efff1200640 1 Processor -- start 2024-01-31T05:03:31.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.171+0000 7efff1200640 1 -- start start 2024-01-31T05:03:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7efff1200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7effec15ec70 0x7effec071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7efff1200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec070400 con 0x7effec0ae3b0 2024-01-31T05:03:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7efff1200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec0ca580 con 0x7effec0723a0 2024-01-31T05:03:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7efff1200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec0ad8c0 con 0x7effec15ec70 2024-01-31T05:03:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7effeaa00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7effec0ae3b0 0x7effec06cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60376/0 (socket says 172.21.15.97:60376) 2024-01-31T05:03:31.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7effeaa00640 1 -- 172.21.15.97:0/1810179968 learned_addr learned my addr 172.21.15.97:0/1810179968 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7effeb400640 1 --2- 172.21.15.97:0/1810179968 >> v2:172.21.15.188:3300/0 conn(0x7effec15ec70 0x7effec071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118101793 0 0) 0x7effec070400 con 0x7effec0ae3b0 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.173+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effcc003520 con 0x7effec0ae3b0 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231223548 0 0) 0x7effcc003520 con 0x7effec0ae3b0 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 >> v2:172.21.15.188:3300/0 conn(0x7effec15ec70 msgr2=0x7effec071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 --2- 172.21.15.97:0/1810179968 >> v2:172.21.15.188:3300/0 conn(0x7effec15ec70 0x7effec071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 >> v1:172.21.15.188:6789/0 conn(0x7effec0723a0 legacy=0x7effec15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec070650 con 0x7effec0ae3b0 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7effd4001670 con 0x7effec0ae3b0 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effeb400640 1 --2- 172.21.15.97:0/1810179968 >> v2:172.21.15.188:3300/0 conn(0x7effec15ec70 0x7effec071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 >> v1:172.21.15.97:6789/0 conn(0x7effec0ae3b0 legacy=0x7effec06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:31.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effcc0047c0 0x7effcc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effcc005380 0x7effcc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec0ad8c0 con 0x7effcc0039c0 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc00c260 con 0x7effcc0047c0 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc00c420 con 0x7effcc005380 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effeb400640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effeb400640 1 -- 172.21.15.97:0/1810179968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effcc005380 msgr2=0x7effcc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.174+0000 7effeb400640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effcc005380 0x7effcc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.175+0000 7effeb400640 1 -- 172.21.15.97:0/1810179968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effcc0047c0 msgr2=0x7effcc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.175+0000 7effeb400640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effcc0047c0 0x7effcc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.175+0000 7effeb400640 1 -- 172.21.15.97:0/1810179968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc003520 con 0x7effcc0039c0 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.175+0000 7effebe00640 1 -- 172.21.15.97:0/1810179968 reap_dead start 2024-01-31T05:03:31.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.175+0000 7effeb400640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7effdc0098a0 tx=0x7effcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28efc11c051fd362 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc004230 con 0x7effcc0039c0 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc004390 con 0x7effcc0039c0 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7effea000640 1 -- 172.21.15.97:0/1810179968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc003830 con 0x7effcc0039c0 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 -- 172.21.15.97:0/1810179968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 msgr2=0x7effcc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7effdc0098a0 tx=0x7effcc0057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 -- 172.21.15.97:0/1810179968 shutdown_connections 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 --2- 172.21.15.97:0/1810179968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effcc003de0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 -- 172.21.15.97:0/1810179968 >> 172.21.15.97:0/1810179968 conn(0x7effec0cec40 msgr2=0x7effec0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 -- 172.21.15.97:0/1810179968 shutdown_connections 2024-01-31T05:03:31.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.176+0000 7efff1200640 1 -- 172.21.15.97:0/1810179968 wait complete. 2024-01-31T05:03:31.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.178+0000 7efff1200640 1 Processor -- start 2024-01-31T05:03:31.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.178+0000 7efff1200640 1 -- start start 2024-01-31T05:03:31.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7efff1200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effec0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7efff1200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effec1f4f00 0x7effec1f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7efff1200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec0aea70 0x7effec0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7efff1200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec09a030 con 0x7effcc0039c0 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7efff1200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec09a310 con 0x7effec0aea70 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7efff1200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec09a5f0 con 0x7effec1f4f00 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7effeb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effec0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7effeb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effec0999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44340/0 (socket says 172.21.15.97:44340) 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.179+0000 7effeb400640 1 -- 172.21.15.97:0/3618121202 learned_addr learned my addr 172.21.15.97:0/3618121202 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effeb400640 1 -- 172.21.15.97:0/3618121202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effec1f4f00 msgr2=0x7effec1f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effeb400640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effec1f4f00 0x7effec1f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effeb400640 1 -- 172.21.15.97:0/3618121202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec0aea70 msgr2=0x7effec0c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effeb400640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec0aea70 0x7effec0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effeb400640 1 -- 172.21.15.97:0/3618121202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec09a8d0 con 0x7effcc0039c0 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effeb400640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effec0999c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7effdc000c00 tx=0x7effdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc030660 con 0x7effcc0039c0 2024-01-31T05:03:31.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc0307c0 con 0x7effcc0039c0 2024-01-31T05:03:31.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc030b70 con 0x7effcc0039c0 2024-01-31T05:03:31.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec09abb0 con 0x7effcc0039c0 2024-01-31T05:03:31.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.180+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec1f25b0 con 0x7effcc0039c0 2024-01-31T05:03:31.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.181+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec0676b0 con 0x7effcc0039c0 2024-01-31T05:03:31.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.182+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7effdc030cd0 con 0x7effcc0039c0 2024-01-31T05:03:31.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.183+0000 7effe3400640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7effc406af00 0x7effc406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:31.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.183+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7effdc0ab360 con 0x7effcc0039c0 2024-01-31T05:03:31.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.183+0000 7effeaa00640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7effc406af00 0x7effc406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:31.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.184+0000 7effeaa00640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7effc406af00 0x7effc406d3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effec0b49e0 tx=0x7effd40014e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:31.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.188+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7effdc0763e0 con 0x7effcc0039c0 2024-01-31T05:03:31.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:31.450+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7effec0cfa70 con 0x7effcc0039c0 2024-01-31T05:03:32.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.155+0000 7effe3400640 1 -- 172.21.15.97:0/3618121202 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v168) v1 ==== 137+0+0 (secure 0 0 0) 0x7effdc079a00 con 0x7effcc0039c0 2024-01-31T05:03:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.158+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7effc406af00 msgr2=0x7effc406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.158+0000 7efff1200640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7effc406af00 0x7effc406d3c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7effec0b49e0 tx=0x7effd40014e0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.158+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 msgr2=0x7effec0999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.158+0000 7efff1200640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effec0999c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7effdc000c00 tx=0x7effdc004730 comp rx=0 tx=0).stop 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 shutdown_connections 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effec0aea70 0x7effec0c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effec1f4f00 0x7effec1f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7effc406af00 0x7effc406d3c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 --2- 172.21.15.97:0/3618121202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effcc0039c0 0x7effec0999c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 >> 172.21.15.97:0/3618121202 conn(0x7effec0cec40 msgr2=0x7effec15a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 shutdown_connections 2024-01-31T05:03:32.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.159+0000 7efff1200640 1 -- 172.21.15.97:0/3618121202 wait complete. 2024-01-31T05:03:32.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-31T05:03:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.379+0000 7f1a58000640 1 Processor -- start 2024-01-31T05:03:32.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.379+0000 7f1a58000640 1 -- start start 2024-01-31T05:03:32.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a58000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5015b470 0x7f1a500722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a58000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a500abcf0 0x7f1a500ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a58000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5006cd20 con 0x7f1a500cb810 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a58000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a500ce970 con 0x7f1a500abcf0 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a58000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015b850 con 0x7f1a5015b470 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a54c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a500abcf0 0x7f1a500ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a56000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1a500cb810 0x7f1a500ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:60378/0 (socket says 172.21.15.97:60378) 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.381+0000 7f1a56000640 1 -- 172.21.15.97:0/2353578223 learned_addr learned my addr 172.21.15.97:0/2353578223 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.382+0000 7f1a55600640 1 --2- 172.21.15.97:0/2353578223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5015b470 0x7f1a500722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.382+0000 7f1a54c00640 1 -- 172.21.15.97:0/2353578223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5015b470 msgr2=0x7f1a500722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.382+0000 7f1a54c00640 1 --2- 172.21.15.97:0/2353578223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5015b470 0x7f1a500722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.382+0000 7f1a54c00640 1 -- 172.21.15.97:0/2353578223 >> v1:172.21.15.97:6789/0 conn(0x7f1a500cb810 legacy=0x7f1a500ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.382+0000 7f1a54c00640 1 -- 172.21.15.97:0/2353578223 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a500ceb60 con 0x7f1a500abcf0 2024-01-31T05:03:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.383+0000 7f1a54c00640 1 --2- 172.21.15.97:0/2353578223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a500abcf0 0x7f1a500ce430 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009dc0 tx=0x7f1a4002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836f8d7593a456d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.383+0000 7f1a4fe00640 1 -- 172.21.15.97:0/2353578223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4002cc40 con 0x7f1a500abcf0 2024-01-31T05:03:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.383+0000 7f1a4fe00640 1 -- 172.21.15.97:0/2353578223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4002cda0 con 0x7f1a500abcf0 2024-01-31T05:03:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.383+0000 7f1a4fe00640 1 -- 172.21.15.97:0/2353578223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40034660 con 0x7f1a500abcf0 2024-01-31T05:03:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 -- 172.21.15.97:0/2353578223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a500abcf0 msgr2=0x7f1a500ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 --2- 172.21.15.97:0/2353578223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a500abcf0 0x7f1a500ce430 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009dc0 tx=0x7f1a4002be90 comp rx=0 tx=0).stop 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 -- 172.21.15.97:0/2353578223 shutdown_connections 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 --2- 172.21.15.97:0/2353578223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a500abcf0 0x7f1a500ce430 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 --2- 172.21.15.97:0/2353578223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a5015b470 0x7f1a500722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 -- 172.21.15.97:0/2353578223 >> 172.21.15.97:0/2353578223 conn(0x7f1a500b0a10 msgr2=0x7f1a500b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 -- 172.21.15.97:0/2353578223 shutdown_connections 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.384+0000 7f1a58000640 1 -- 172.21.15.97:0/2353578223 wait complete. 2024-01-31T05:03:32.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.386+0000 7f1a58000640 1 Processor -- start 2024-01-31T05:03:32.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.386+0000 7f1a58000640 1 -- start start 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a58000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a500abcf0 0x7f1a5013fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a58000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a500cb810 0x7f1a5009c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a58000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 0x7f1a5009c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a58000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50140810 con 0x7f1a5015b470 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a58000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015d630 con 0x7f1a500abcf0 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a58000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015d910 con 0x7f1a500cb810 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a56000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 0x7f1a5009c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a56000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 0x7f1a5009c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44364/0 (socket says 172.21.15.97:44364) 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a56000640 1 -- 172.21.15.97:0/2602517204 learned_addr learned my addr 172.21.15.97:0/2602517204 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:32.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.387+0000 7f1a54c00640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a500cb810 0x7f1a5009c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a56000640 1 -- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a500cb810 msgr2=0x7f1a5009c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a56000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a500cb810 0x7f1a5009c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a56000640 1 -- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a500abcf0 msgr2=0x7f1a5013fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a55600640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a500abcf0 0x7f1a5013fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a56000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a500abcf0 0x7f1a5013fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a56000640 1 -- 172.21.15.97:0/2602517204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a40009a70 con 0x7f1a5015b470 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a54c00640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a500cb810 0x7f1a5009c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a56000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 0x7f1a5009c7b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800c420 tx=0x7f1a4800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48005ce0 con 0x7f1a5015b470 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48005e40 con 0x7f1a5015b470 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480072d0 con 0x7f1a5015b470 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5015dbf0 con 0x7f1a5015b470 2024-01-31T05:03:32.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.388+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5015e1b0 con 0x7f1a5015b470 2024-01-31T05:03:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.389+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a500cc100 con 0x7f1a5015b470 2024-01-31T05:03:32.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.390+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1a48007430 con 0x7f1a5015b470 2024-01-31T05:03:32.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.391+0000 7f1a4d600640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1a2806af00 0x7f1a2806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:32.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.391+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1a48088900 con 0x7f1a5015b470 2024-01-31T05:03:32.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.391+0000 7f1a55600640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1a2806af00 0x7f1a2806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:32.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.392+0000 7f1a55600640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1a2806af00 0x7f1a2806d3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1a50099ee0 tx=0x7f1a38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:32.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.396+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1a48053a30 con 0x7f1a5015b470 2024-01-31T05:03:32.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.590+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1a50067830 con 0x7f1a5015b470 2024-01-31T05:03:32.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.597+0000 7f1a4d600640 1 -- 172.21.15.97:0/2602517204 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1a48017020 con 0x7f1a5015b470 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1a2806af00 msgr2=0x7f1a2806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1a2806af00 0x7f1a2806d3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1a50099ee0 tx=0x7f1a38008040 comp rx=0 tx=0).stop 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 msgr2=0x7f1a5009c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 0x7f1a5009c7b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800c420 tx=0x7f1a4800c8e0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 shutdown_connections 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1a2806af00 0x7f1a2806d3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a5015b470 0x7f1a5009c7b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a500cb810 0x7f1a5009c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 --2- 172.21.15.97:0/2602517204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a500abcf0 0x7f1a5013fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 >> 172.21.15.97:0/2602517204 conn(0x7f1a500b0a10 msgr2=0x7f1a5006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 shutdown_connections 2024-01-31T05:03:32.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:32.600+0000 7f1a58000640 1 -- 172.21.15.97:0/2602517204 wait complete. 2024-01-31T05:03:32.619 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-31T05:03:32.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:03:32.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:03:32.830 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-31T05:03:32.831 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:32.831 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:32.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:03:32.887 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:03:32.887 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:32.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:03:32.990 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:32.990 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:33.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:03:33.047 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:03:33.047 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:33.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:03:33.103 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-31T05:03:33.103 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:33.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.322+0000 7fd29a400640 1 Processor -- start 2024-01-31T05:03:33.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.322+0000 7fd29a400640 1 -- start start 2024-01-31T05:03:33.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd29a400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd29415a600 0x7fd29415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd29a400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29406c8b0 con 0x7fd29406d090 2024-01-31T05:03:33.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd29a400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29415b120 con 0x7fd2940b1d10 2024-01-31T05:03:33.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd29a400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29415b400 con 0x7fd29415a600 2024-01-31T05:03:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd28a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd29415a600 0x7fd29415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd28a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd29415a600 0x7fd29415aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44368/0 (socket says 172.21.15.97:44368) 2024-01-31T05:03:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd28a800640 1 -- 172.21.15.97:0/2163822574 learned_addr learned my addr 172.21.15.97:0/2163822574 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.324+0000 7fd28a800640 1 -- 172.21.15.97:0/2163822574 >> v1:172.21.15.188:6789/0 conn(0x7fd2940b1d10 legacy=0x7fd2940b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:33.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.325+0000 7fd28a800640 1 -- 172.21.15.97:0/2163822574 >> v1:172.21.15.97:6789/0 conn(0x7fd29406d090 legacy=0x7fd29406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.325+0000 7fd28a800640 1 -- 172.21.15.97:0/2163822574 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29406cb70 con 0x7fd29415a600 2024-01-31T05:03:33.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.325+0000 7fd292a00640 1 -- 172.21.15.97:0/2163822574 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705350584 0 0) 0x7fd29406c8b0 con 0x7fd29406d090 2024-01-31T05:03:33.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.326+0000 7fd28a800640 1 --2- 172.21.15.97:0/2163822574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd29415aa60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009870 tx=0x7fd27c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42721a929895f09a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:33.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.326+0000 7fd292a00640 1 -- 172.21.15.97:0/2163822574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c030860 con 0x7fd29415a600 2024-01-31T05:03:33.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.326+0000 7fd292a00640 1 -- 172.21.15.97:0/2163822574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c0309c0 con 0x7fd29415a600 2024-01-31T05:03:33.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.326+0000 7fd292a00640 1 -- 172.21.15.97:0/2163822574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c030d50 con 0x7fd29415a600 2024-01-31T05:03:33.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.326+0000 7fd29a400640 1 -- 172.21.15.97:0/2163822574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 msgr2=0x7fd29415aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.326+0000 7fd29a400640 1 --2- 172.21.15.97:0/2163822574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd29415aa60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009870 tx=0x7fd27c02b9a0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.327+0000 7fd29a400640 1 -- 172.21.15.97:0/2163822574 shutdown_connections 2024-01-31T05:03:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.327+0000 7fd29a400640 1 --2- 172.21.15.97:0/2163822574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd29415aa60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.327+0000 7fd29a400640 1 -- 172.21.15.97:0/2163822574 >> 172.21.15.97:0/2163822574 conn(0x7fd2940b0a10 msgr2=0x7fd2940b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.327+0000 7fd29a400640 1 -- 172.21.15.97:0/2163822574 shutdown_connections 2024-01-31T05:03:33.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.327+0000 7fd29a400640 1 -- 172.21.15.97:0/2163822574 wait complete. 2024-01-31T05:03:33.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.328+0000 7fd29a400640 1 Processor -- start 2024-01-31T05:03:33.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 -- start start 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd29406d090 0x7fd294146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1d10 0x7fd294155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd2940999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294146db0 con 0x7fd29415a600 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd294156570 con 0x7fd29406d090 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.329+0000 7fd29a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd294156850 con 0x7fd2940b1d10 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd2940999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd2940999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44382/0 (socket says 172.21.15.97:44382) 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 -- 172.21.15.97:0/1431349823 learned_addr learned my addr 172.21.15.97:0/1431349823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd29406d090 0x7fd294146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd28a800640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1d10 0x7fd294155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 -- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1d10 msgr2=0x7fd294155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1d10 0x7fd294155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 -- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd29406d090 msgr2=0x7fd294146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd29406d090 0x7fd294146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.330+0000 7fd293e00640 1 -- 172.21.15.97:0/1431349823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c0094d0 con 0x7fd29415a600 2024-01-31T05:03:33.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.331+0000 7fd293e00640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd2940999c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c760 tx=0x7fd28c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.331+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c003da0 con 0x7fd29415a600 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.331+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00ce70 con 0x7fd29415a600 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.331+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c005290 con 0x7fd29415a600 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.331+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd294156b30 con 0x7fd29415a600 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.331+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941570f0 con 0x7fd29415a600 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.332+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2941572e0 con 0x7fd29415a600 2024-01-31T05:03:33.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.333+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd28c0053f0 con 0x7fd29415a600 2024-01-31T05:03:33.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.334+0000 7fd28be00640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd26c06aed0 0x7fd26c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.334+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd28c0892d0 con 0x7fd29415a600 2024-01-31T05:03:33.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.334+0000 7fd293400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd26c06aed0 0x7fd26c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.335+0000 7fd293400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd26c06aed0 0x7fd26c06d390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd284004ea0 tx=0x7fd284009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:33.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.339+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd28c054340 con 0x7fd29415a600 2024-01-31T05:03:33.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.571+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd29406e740 con 0x7fd29415a600 2024-01-31T05:03:33.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.572+0000 7fd28be00640 1 -- 172.21.15.97:0/1431349823 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fd28c057960 con 0x7fd29415a600 2024-01-31T05:03:33.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:33.572 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/881492211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":881492211},{"type":"v1","addr":"172.21.15.188:6835","nonce":881492211}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4918,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2955467530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2955467530},{"type":"v1","addr":"172.21.15.97:6837","nonce":2955467530}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4931,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1452104010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1452104010},{"type":"v1","addr":"172.21.15.188:6834","nonce":1452104010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/596998971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":596998971},{"type":"v1","addr":"172.21.15.97:6836","nonce":596998971}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-01-31T05:03:33.573 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 20 2024-01-31T05:03:33.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.574+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd26c06aed0 msgr2=0x7fd26c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.574+0000 7fd29a400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd26c06aed0 0x7fd26c06d390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd284004ea0 tx=0x7fd284009040 comp rx=0 tx=0).stop 2024-01-31T05:03:33.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.574+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 msgr2=0x7fd2940999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.574+0000 7fd29a400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd2940999c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00c760 tx=0x7fd28c00cc20 comp rx=0 tx=0).stop 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 shutdown_connections 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd26c06aed0 0x7fd26c06d390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd29415a600 0x7fd2940999c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2940b1d10 0x7fd294155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 --2- 172.21.15.97:0/1431349823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd29406d090 0x7fd294146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 >> 172.21.15.97:0/1431349823 conn(0x7fd2940b0a10 msgr2=0x7fd29406d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 shutdown_connections 2024-01-31T05:03:33.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.575+0000 7fd29a400640 1 -- 172.21.15.97:0/1431349823 wait complete. 2024-01-31T05:03:33.595 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-31T05:03:33.595 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-31T05:03:33.595 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-31T05:03:33.595 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:03:33.595 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-31T05:03:33.596 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-31T05:03:33.596 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-31T05:03:33.596 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:03:33.596 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-31T05:03:33.596 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-31T05:03:33.596 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-31T05:03:33.596 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:03:33.596 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-31T05:03:33.597 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-31T05:03:33.597 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-31T05:03:33.597 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:03:33.612 INFO:tasks.ceph.mds.a:Stopped 2024-01-31T05:03:33.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-31T05:03:33.615 INFO:tasks.ceph.mds.d:Stopped 2024-01-31T05:03:33.615 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-31T05:03:33.618 INFO:tasks.ceph.mds.c:Stopped 2024-01-31T05:03:33.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-31T05:03:33.619 INFO:tasks.ceph.mds.b:Stopped 2024-01-31T05:03:33.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-31T05:03:33.665 INFO:tasks.ceph.mds.a:Started 2024-01-31T05:03:33.665 DEBUG:teuthology.parallel:result is None 2024-01-31T05:03:33.666 INFO:tasks.ceph.mds.d:Started 2024-01-31T05:03:33.667 DEBUG:teuthology.parallel:result is None 2024-01-31T05:03:33.671 INFO:tasks.ceph.mds.c:Started 2024-01-31T05:03:33.671 DEBUG:teuthology.parallel:result is None 2024-01-31T05:03:33.675 INFO:tasks.ceph.mds.b:Started 2024-01-31T05:03:33.675 DEBUG:teuthology.parallel:result is None 2024-01-31T05:03:33.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-31T05:03:33.818 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2024-01-31T05:03:33.825 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-01-31T05:03:33.827 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-01-31T05:03:33.836 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-01-31T05:03:33.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.861+0000 7f4af7800640 1 Processor -- start 2024-01-31T05:03:33.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.862+0000 7f4af7800640 1 -- start start 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af7800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af00cde90 0x7f4af00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af7800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4af015b760 0x7f4af00cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af7800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af00cd010 con 0x7f4af015b380 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af7800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00cc770 con 0x7f4af015b760 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af7800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00cca50 con 0x7f4af00cde90 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af5800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4af015b760 0x7f4af00cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af00cde90 0x7f4af00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af4e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4af015b380 0x7f4af00cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:35294/0 (socket says 172.21.15.97:35294) 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af5800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4af015b760 0x7f4af00cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44406/0 (socket says 172.21.15.97:44406) 2024-01-31T05:03:33.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.863+0000 7f4af4e00640 1 -- 172.21.15.97:0/4166577370 learned_addr learned my addr 172.21.15.97:0/4166577370 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.864+0000 7f4af5800640 1 -- 172.21.15.97:0/4166577370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af00cde90 msgr2=0x7f4af00cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.864+0000 7f4af5800640 1 --2- 172.21.15.97:0/4166577370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af00cde90 0x7f4af00cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.864+0000 7f4af5800640 1 -- 172.21.15.97:0/4166577370 >> v1:172.21.15.188:6789/0 conn(0x7f4af015b380 legacy=0x7f4af00cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.864+0000 7f4aef400640 1 -- 172.21.15.97:0/4166577370 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411400258 0 0) 0x7f4af00cd010 con 0x7f4af015b380 2024-01-31T05:03:33.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.864+0000 7f4af5800640 1 -- 172.21.15.97:0/4166577370 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00ccd30 con 0x7f4af015b760 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.865+0000 7f4af5800640 1 --2- 172.21.15.97:0/4166577370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b760 0x7f4af00cf390 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009d90 tx=0x7f4ae802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c477383c4c404f5b server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.865+0000 7f4aef400640 1 -- 172.21.15.97:0/4166577370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae802cc50 con 0x7f4af015b760 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.865+0000 7f4aef400640 1 -- 172.21.15.97:0/4166577370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae802cdb0 con 0x7f4af015b760 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.865+0000 7f4aef400640 1 -- 172.21.15.97:0/4166577370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8034660 con 0x7f4af015b760 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.865+0000 7f4af7800640 1 -- 172.21.15.97:0/4166577370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b760 msgr2=0x7f4af00cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.865+0000 7f4af7800640 1 --2- 172.21.15.97:0/4166577370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b760 0x7f4af00cf390 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009d90 tx=0x7f4ae802bea0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.866+0000 7f4af7800640 1 -- 172.21.15.97:0/4166577370 shutdown_connections 2024-01-31T05:03:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.866+0000 7f4af7800640 1 --2- 172.21.15.97:0/4166577370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b760 0x7f4af00cf390 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.866+0000 7f4af7800640 1 --2- 172.21.15.97:0/4166577370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af00cde90 0x7f4af00cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.866+0000 7f4af7800640 1 -- 172.21.15.97:0/4166577370 >> 172.21.15.97:0/4166577370 conn(0x7f4af00d1070 msgr2=0x7f4af00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.866+0000 7f4af7800640 1 -- 172.21.15.97:0/4166577370 shutdown_connections 2024-01-31T05:03:33.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.866+0000 7f4af7800640 1 -- 172.21.15.97:0/4166577370 wait complete. 2024-01-31T05:03:33.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.867+0000 7f4af7800640 1 Processor -- start 2024-01-31T05:03:33.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.867+0000 7f4af7800640 1 -- start start 2024-01-31T05:03:33.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.868+0000 7f4af7800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cde90 0x7f4af015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.868+0000 7f4af7800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 0x7f4af01f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.868+0000 7f4af7800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af015b760 0x7f4af01546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.868+0000 7f4af7800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01f3700 con 0x7f4af015b380 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.868+0000 7f4af7800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01f39d0 con 0x7f4af00cde90 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.868+0000 7f4af7800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01f3cb0 con 0x7f4af015b760 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 0x7f4af01f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 0x7f4af01f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44418/0 (socket says 172.21.15.97:44418) 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 -- 172.21.15.97:0/2334235021 learned_addr learned my addr 172.21.15.97:0/2334235021 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4af4e00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cde90 0x7f4af015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 -- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af015b760 msgr2=0x7f4af01546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af015b760 0x7f4af01546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 -- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cde90 msgr2=0x7f4af015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cde90 0x7f4af015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 -- 172.21.15.97:0/2334235021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009a70 con 0x7f4af015b380 2024-01-31T05:03:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4af4e00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cde90 0x7f4af015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:33.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.869+0000 7f4aefe00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 0x7f4af01f31c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800e870 tx=0x7f4ad800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.870+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80094f0 con 0x7f4af015b380 2024-01-31T05:03:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.870+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad800c940 con 0x7f4af015b380 2024-01-31T05:03:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.870+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800ccf0 con 0x7f4af015b380 2024-01-31T05:03:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.870+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af01f3f90 con 0x7f4af015b380 2024-01-31T05:03:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.870+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01f6a90 con 0x7f4af015b380 2024-01-31T05:03:33.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.872+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af0067830 con 0x7f4af015b380 2024-01-31T05:03:33.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.872+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4ad8013470 con 0x7f4af015b380 2024-01-31T05:03:33.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.873+0000 7f4aecc00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ac806aed0 0x7f4ac806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:33.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.873+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4ad8089530 con 0x7f4af015b380 2024-01-31T05:03:33.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.874+0000 7f4af4e00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ac806aed0 0x7f4ac806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:33.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.874+0000 7f4af4e00640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ac806aed0 0x7f4ac806d390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0000f60 tx=0x7f4ae0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:33.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:33.878+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4ad80545c0 con 0x7f4af015b380 2024-01-31T05:03:34.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.086+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4af00d03a0 con 0x7f4af015b380 2024-01-31T05:03:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.179+0000 7f4aecc00640 1 -- 172.21.15.97:0/2334235021 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ad8057be0 con 0x7f4af015b380 2024-01-31T05:03:34.180 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2024-01-31T05:03:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ac806aed0 msgr2=0x7f4ac806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ac806aed0 0x7f4ac806d390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0000f60 tx=0x7f4ae0009040 comp rx=0 tx=0).stop 2024-01-31T05:03:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 msgr2=0x7f4af01f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 0x7f4af01f31c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800e870 tx=0x7f4ad800ed30 comp rx=0 tx=0).stop 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 shutdown_connections 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af015b760 0x7f4af01546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4ac806aed0 0x7f4ac806d390 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4af015b380 0x7f4af01f31c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 --2- 172.21.15.97:0/2334235021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cde90 0x7f4af015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.183+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 >> 172.21.15.97:0/2334235021 conn(0x7f4af00d1070 msgr2=0x7f4af00cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.184+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 shutdown_connections 2024-01-31T05:03:34.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.184+0000 7f4af7800640 1 -- 172.21.15.97:0/2334235021 wait complete. 2024-01-31T05:03:34.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-31T05:03:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.405+0000 7f4810a00640 1 Processor -- start 2024-01-31T05:03:34.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.405+0000 7f4810a00640 1 -- start start 2024-01-31T05:03:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f4810a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48080b1d10 0x7f48080b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f4810a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f480815a600 0x7f480815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f4810a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815dc50 0x7f480815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f4810a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080b2630 con 0x7f48080b1d10 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f4810a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480806c8b0 con 0x7f480815a600 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f4810a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480815b120 con 0x7f480815dc50 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48080b1d10 0x7f48080b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48080b1d10 0x7f48080b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44428/0 (socket says 172.21.15.97:44428) 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 -- 172.21.15.97:0/1173752026 learned_addr learned my addr 172.21.15.97:0/1173752026 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480d600640 1 --2- 172.21.15.97:0/1173752026 >> v2:172.21.15.188:3300/0 conn(0x7f480815a600 0x7f480815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480ea00640 1 --2- 172.21.15.97:0/1173752026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815dc50 0x7f480815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 -- 172.21.15.97:0/1173752026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815dc50 msgr2=0x7f480815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 --2- 172.21.15.97:0/1173752026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815dc50 0x7f480815e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 -- 172.21.15.97:0/1173752026 >> v2:172.21.15.188:3300/0 conn(0x7f480815a600 msgr2=0x7f480815aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:34.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 --2- 172.21.15.97:0/1173752026 >> v2:172.21.15.188:3300/0 conn(0x7f480815a600 0x7f480815aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.407+0000 7f480e000640 1 -- 172.21.15.97:0/1173752026 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480815b400 con 0x7f48080b1d10 2024-01-31T05:03:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.408+0000 7f480e000640 1 --2- 172.21.15.97:0/1173752026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080b20f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4808158250 tx=0x7f47f802db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d171d80fcfde4d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.409+0000 7f480cc00640 1 -- 172.21.15.97:0/1173752026 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f802f8e0 con 0x7f48080b1d10 2024-01-31T05:03:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.409+0000 7f480cc00640 1 -- 172.21.15.97:0/1173752026 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f802fa40 con 0x7f48080b1d10 2024-01-31T05:03:34.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.409+0000 7f480cc00640 1 -- 172.21.15.97:0/1173752026 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f802fdd0 con 0x7f48080b1d10 2024-01-31T05:03:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.409+0000 7f4810a00640 1 -- 172.21.15.97:0/1173752026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 msgr2=0x7f48080b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.409+0000 7f4810a00640 1 --2- 172.21.15.97:0/1173752026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080b20f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4808158250 tx=0x7f47f802db20 comp rx=0 tx=0).stop 2024-01-31T05:03:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 -- 172.21.15.97:0/1173752026 shutdown_connections 2024-01-31T05:03:34.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 --2- 172.21.15.97:0/1173752026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815dc50 0x7f480815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 --2- 172.21.15.97:0/1173752026 >> v2:172.21.15.188:3300/0 conn(0x7f480815a600 0x7f480815aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 --2- 172.21.15.97:0/1173752026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080b20f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 -- 172.21.15.97:0/1173752026 >> 172.21.15.97:0/1173752026 conn(0x7f48080b0a10 msgr2=0x7f48080b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:34.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 -- 172.21.15.97:0/1173752026 shutdown_connections 2024-01-31T05:03:34.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.410+0000 7f4810a00640 1 -- 172.21.15.97:0/1173752026 wait complete. 2024-01-31T05:03:34.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.412+0000 7f4810a00640 1 Processor -- start 2024-01-31T05:03:34.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.412+0000 7f4810a00640 1 -- start start 2024-01-31T05:03:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.413+0000 7f4810a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.413+0000 7f4810a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815a600 0x7f48080b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.413+0000 7f4810a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480815dc50 0x7f480815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.413+0000 7f4810a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480815ce10 con 0x7f48080b1d10 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.413+0000 7f4810a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480814ce40 con 0x7f480815dc50 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.413+0000 7f4810a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480815c570 con 0x7f480815a600 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44440/0 (socket says 172.21.15.97:44440) 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 -- 172.21.15.97:0/2586275780 learned_addr learned my addr 172.21.15.97:0/2586275780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480d600640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815a600 0x7f48080b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480ea00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480815dc50 0x7f480815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 -- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815a600 msgr2=0x7f48080b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815a600 0x7f48080b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 -- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480815dc50 msgr2=0x7f480815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480815dc50 0x7f480815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 -- 172.21.15.97:0/2586275780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80094d0 con 0x7f48080b1d10 2024-01-31T05:03:34.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480ea00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480815dc50 0x7f480815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.414+0000 7f480e000640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080afb00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f47f802db60 tx=0x7f47f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.415+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8037c60 con 0x7f48080b1d10 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.415+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8037de0 con 0x7f48080b1d10 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.415+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80426b0 con 0x7f48080b1d10 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.415+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48080adb80 con 0x7f48080b1d10 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.415+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480815cb30 con 0x7f48080b1d10 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.417+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080677c0 con 0x7f48080b1d10 2024-01-31T05:03:34.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.417+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f47f8042810 con 0x7f48080b1d10 2024-01-31T05:03:34.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.418+0000 7f4806000640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f47e006aed0 0x7f47e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.418+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f47f8033070 con 0x7f48080b1d10 2024-01-31T05:03:34.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.418+0000 7f480d600640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f47e006aed0 0x7f47e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.419+0000 7f480d600640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f47e006aed0 0x7f47e006d390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48080cc100 tx=0x7f47f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:34.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.423+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f47f80770e0 con 0x7f48080b1d10 2024-01-31T05:03:34.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.685+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f480806e740 con 0x7f48080b1d10 2024-01-31T05:03:34.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.687+0000 7f4806000640 1 -- 172.21.15.97:0/2586275780 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f47f8046090 con 0x7f48080b1d10 2024-01-31T05:03:34.686 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:34.686 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDB07llYd8zLRAAXY4glNCfVX1dTNGZUbfFmQ==", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llbUoqERAAS5j0bqhOLOEBYeJJAcw1gg==", 2024-01-31T05:03:34.687 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llaOqoARAAR5IabzNA8GwYKZPChWBslQ==", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:03:34.688 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llaDyVHhAAxCxPKz8CdUNYF2635sVSPg==", 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDH07llltQODBAAlc7RryuyTlHz0D9phNkn2w==", 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDK07llEPFhJRAABOCmKBieoS3MS1W+SZKB3A==", 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.690 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDN07llKeEkLxAAJx3yznrb9PPxr2ok3pcXAw==", 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.691 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDQ07ll1vrzOBAAO9IwnSQp37YmgtkSvJ9pTA==", 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2024-01-31T05:03:34.692 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDY07llYzg7DRAAcbb39NsWEVWRdIornxlExQ==", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDb07llNPq2FhAASqQSEA/5vGxqRv3LOVIW8Q==", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.693 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDe07ll9GdBIBAAW+tnotDCbokaSSouTInb/w==", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDh07llDRo1ORAASF67CwBasg4V3RLF0lvQjw==", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A==", 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw pool=cephfs_data" 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDD07llR2txBBAA7So8sHU4Wrev8Jt381SvFA==", 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.695 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDC07ll2MQRNhAAOVu6KYtnRga+FC4ViTTGzA==", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.697 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.697 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2024-01-31T05:03:34.697 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDD07llpqYODxAAKYaORRDDq/CCur8PKSPKNw==", 2024-01-31T05:03:34.697 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.697 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.699 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDA07llafWAIBAAyGsoMQmBd7Wf2ZwG5QCPeA==", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDB07llSQTaIRAAGEtHw/4lHnjbsB87JXmqYQ==", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-01-31T05:03:34.700 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDB07llWApVFhAAsTCld+Pxo/BYtCHNVlnvPQ==", 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-01-31T05:03:34.701 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f47e006aed0 msgr2=0x7f47e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f47e006aed0 0x7f47e006d390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48080cc100 tx=0x7f47f0008040 comp rx=0 tx=0).stop 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 msgr2=0x7f48080afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080afb00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f47f802db60 tx=0x7f47f80046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 shutdown_connections 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f480815dc50 0x7f480815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f480815a600 0x7f48080b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f47e006aed0 0x7f47e006d390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 --2- 172.21.15.97:0/2586275780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080b1d10 0x7f48080afb00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 >> 172.21.15.97:0/2586275780 conn(0x7f48080b0a10 msgr2=0x7f480806d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:34.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 shutdown_connections 2024-01-31T05:03:34.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.688+0000 7f4810a00640 1 -- 172.21.15.97:0/2586275780 wait complete. 2024-01-31T05:03:34.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:34.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.916+0000 7f4b31200640 1 Processor -- start 2024-01-31T05:03:34.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.916+0000 7f4b31200640 1 -- start start 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.917+0000 7f4b31200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b2c15ec70 0x7f4b2c071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b31200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b31200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c070400 con 0x7f4b2c0ae3b0 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b31200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c0ad8c0 con 0x7f4b2c15ec70 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b31200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c0ca580 con 0x7f4b2c0723a0 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2b400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b2c15ec70 0x7f4b2c071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b22800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4b2c0ae3b0 0x7f4b2c06cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55298/0 (socket says 172.21.15.97:55298) 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b22800640 1 -- 172.21.15.97:0/846827614 learned_addr learned my addr 172.21.15.97:0/846827614 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2be00640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2b400640 1 -- 172.21.15.97:0/846827614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 msgr2=0x7f4b2c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:34.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2b400640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2b400640 1 -- 172.21.15.97:0/846827614 >> v1:172.21.15.97:6789/0 conn(0x7f4b2c0ae3b0 legacy=0x7f4b2c06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2b400640 1 -- 172.21.15.97:0/846827614 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c070650 con 0x7f4b2c15ec70 2024-01-31T05:03:34.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2aa00640 1 -- 172.21.15.97:0/846827614 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698083381 0 0) 0x7f4b2c070400 con 0x7f4b2c0ae3b0 2024-01-31T05:03:34.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.918+0000 7f4b2be00640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.919+0000 7f4b2b400640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c071e60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009d90 tx=0x7f4b1c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a26347c5c1f463 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b2aa00640 1 -- 172.21.15.97:0/846827614 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c02cb30 con 0x7f4b2c15ec70 2024-01-31T05:03:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b2aa00640 1 -- 172.21.15.97:0/846827614 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c02cc90 con 0x7f4b2c15ec70 2024-01-31T05:03:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b2aa00640 1 -- 172.21.15.97:0/846827614 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c034440 con 0x7f4b2c15ec70 2024-01-31T05:03:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b31200640 1 -- 172.21.15.97:0/846827614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 msgr2=0x7f4b2c071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b31200640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c071e60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009d90 tx=0x7f4b1c02bd80 comp rx=0 tx=0).stop 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b31200640 1 -- 172.21.15.97:0/846827614 shutdown_connections 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b31200640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b31200640 1 --2- 172.21.15.97:0/846827614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c071e60 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.920+0000 7f4b31200640 1 -- 172.21.15.97:0/846827614 >> 172.21.15.97:0/846827614 conn(0x7f4b2c0cec40 msgr2=0x7f4b2c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.921+0000 7f4b31200640 1 -- 172.21.15.97:0/846827614 shutdown_connections 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.921+0000 7f4b31200640 1 -- 172.21.15.97:0/846827614 wait complete. 2024-01-31T05:03:34.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.922+0000 7f4b31200640 1 Processor -- start 2024-01-31T05:03:34.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.922+0000 7f4b31200640 1 -- start start 2024-01-31T05:03:34.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.923+0000 7f4b31200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c1f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.923+0000 7f4b31200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c0ae3b0 0x7f4b2c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.923+0000 7f4b31200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.923+0000 7f4b31200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c151d30 con 0x7f4b2c15ec70 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.923+0000 7f4b31200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c09be10 con 0x7f4b2c0ae3b0 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.923+0000 7f4b31200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c09af50 con 0x7f4b2c0723a0 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c09aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42102/0 (socket says 172.21.15.97:42102) 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 -- 172.21.15.97:0/1163177483 learned_addr learned my addr 172.21.15.97:0/1163177483 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2b400640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c1f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b22800640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c0ae3b0 0x7f4b2c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 -- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 msgr2=0x7f4b2c1f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c1f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 -- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c0ae3b0 msgr2=0x7f4b2c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c0ae3b0 0x7f4b2c1517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.924+0000 7f4b2be00640 1 -- 172.21.15.97:0/1163177483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c009a70 con 0x7f4b2c15ec70 2024-01-31T05:03:34.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.925+0000 7f4b2be00640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c09aa10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400c7c0 tx=0x7f4b2400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.925+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004230 con 0x7f4b2c15ec70 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.925+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b24004390 con 0x7f4b2c15ec70 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.925+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24007860 con 0x7f4b2c15ec70 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.925+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c09b200 con 0x7f4b2c15ec70 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.925+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c09b7c0 con 0x7f4b2c15ec70 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.926+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c0b2b40 con 0x7f4b2c15ec70 2024-01-31T05:03:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.927+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4b240079c0 con 0x7f4b2c15ec70 2024-01-31T05:03:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.928+0000 7f4b23e00640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b0406af00 0x7f4b0406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.928+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4b24089630 con 0x7f4b2c15ec70 2024-01-31T05:03:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.928+0000 7f4b2b400640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b0406af00 0x7f4b0406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:34.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.929+0000 7f4b2b400640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b0406af00 0x7f4b0406d3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c036000 tx=0x7f4b1c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:34.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:34.932+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4b240547d0 con 0x7f4b2c15ec70 2024-01-31T05:03:35.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.125+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b2c0676b0 con 0x7f4b2c15ec70 2024-01-31T05:03:35.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.127+0000 7f4b23e00640 1 -- 172.21.15.97:0/1163177483 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13063 (secure 0 0 0) 0x7f4b24057df0 con 0x7f4b2c15ec70 2024-01-31T05:03:35.127 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:35.127 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":51,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:34.194309+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:35.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.129+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b0406af00 msgr2=0x7f4b0406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.129+0000 7f4b31200640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b0406af00 0x7f4b0406d3c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c036000 tx=0x7f4b1c030000 comp rx=0 tx=0).stop 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.129+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 msgr2=0x7f4b2c09aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.129+0000 7f4b31200640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c09aa10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400c7c0 tx=0x7f4b2400cc80 comp rx=0 tx=0).stop 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 shutdown_connections 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b0406af00 0x7f4b0406d3c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b2c15ec70 0x7f4b2c09aa10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b2c0ae3b0 0x7f4b2c1517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 --2- 172.21.15.97:0/1163177483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b2c0723a0 0x7f4b2c1f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 >> 172.21.15.97:0/1163177483 conn(0x7f4b2c0cec40 msgr2=0x7f4b2c0afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 shutdown_connections 2024-01-31T05:03:35.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.130+0000 7f4b31200640 1 -- 172.21.15.97:0/1163177483 wait complete. 2024-01-31T05:03:35.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:35.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:35.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.359+0000 7f4989800640 1 Processor -- start 2024-01-31T05:03:35.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.359+0000 7f4989800640 1 -- start start 2024-01-31T05:03:35.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4989800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49840b2570 0x7f49840cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4989800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49840d1b90 0x7f49840cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4989800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f498415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4989800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840ce2c0 con 0x7f49840d1b90 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4989800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498406ce00 con 0x7f49840b2570 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4989800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49840cb760 con 0x7f49840cde30 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4982000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49840d1b90 0x7f49840cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4982000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49840d1b90 0x7f49840cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42106/0 (socket says 172.21.15.97:42106) 2024-01-31T05:03:35.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4982000640 1 -- 172.21.15.97:0/708452869 learned_addr learned my addr 172.21.15.97:0/708452869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:35.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.361+0000 7f4982a00640 1 --2- 172.21.15.97:0/708452869 >> v2:172.21.15.188:3300/0 conn(0x7f49840b2570 0x7f49840cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4982000640 1 -- 172.21.15.97:0/708452869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 msgr2=0x7f498415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4983400640 1 --2- 172.21.15.97:0/708452869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f498415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4982000640 1 --2- 172.21.15.97:0/708452869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f498415cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4982000640 1 -- 172.21.15.97:0/708452869 >> v2:172.21.15.188:3300/0 conn(0x7f49840b2570 msgr2=0x7f49840cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4982000640 1 --2- 172.21.15.97:0/708452869 >> v2:172.21.15.188:3300/0 conn(0x7f49840b2570 0x7f49840cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4982000640 1 -- 172.21.15.97:0/708452869 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840b2950 con 0x7f49840d1b90 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.362+0000 7f4982a00640 1 --2- 172.21.15.97:0/708452869 >> v2:172.21.15.188:3300/0 conn(0x7f49840b2570 0x7f49840cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.363+0000 7f4982000640 1 --2- 172.21.15.97:0/708452869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840d1b90 0x7f49840cd8f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f498406e560 tx=0x7f496c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e7e4ec2b2eaf5a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.363+0000 7f4981600640 1 -- 172.21.15.97:0/708452869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c031640 con 0x7f49840d1b90 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.363+0000 7f4981600640 1 -- 172.21.15.97:0/708452869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c0317a0 con 0x7f49840d1b90 2024-01-31T05:03:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.363+0000 7f4981600640 1 -- 172.21.15.97:0/708452869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c031b30 con 0x7f49840d1b90 2024-01-31T05:03:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 -- 172.21.15.97:0/708452869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840d1b90 msgr2=0x7f49840cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 --2- 172.21.15.97:0/708452869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840d1b90 0x7f49840cd8f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f498406e560 tx=0x7f496c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 -- 172.21.15.97:0/708452869 shutdown_connections 2024-01-31T05:03:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 --2- 172.21.15.97:0/708452869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f498415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 --2- 172.21.15.97:0/708452869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840d1b90 0x7f49840cd8f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 --2- 172.21.15.97:0/708452869 >> v2:172.21.15.188:3300/0 conn(0x7f49840b2570 0x7f49840cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 -- 172.21.15.97:0/708452869 >> 172.21.15.97:0/708452869 conn(0x7f49840d1070 msgr2=0x7f49840d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 -- 172.21.15.97:0/708452869 shutdown_connections 2024-01-31T05:03:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.364+0000 7f4989800640 1 -- 172.21.15.97:0/708452869 wait complete. 2024-01-31T05:03:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.366+0000 7f4989800640 1 Processor -- start 2024-01-31T05:03:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.366+0000 7f4989800640 1 -- start start 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4989800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 0x7f498414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4989800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f49840c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4989800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840d1b90 0x7f4984150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4989800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498414e360 con 0x7f49840b2570 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4989800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840c5080 con 0x7f49840d1b90 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4989800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49841f65b0 con 0x7f49840cde30 2024-01-31T05:03:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 0x7f498414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 0x7f498414de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42110/0 (socket says 172.21.15.97:42110) 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 -- 172.21.15.97:0/1685103464 learned_addr learned my addr 172.21.15.97:0/1685103464 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4983400640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840d1b90 0x7f4984150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 -- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 msgr2=0x7f49840c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982000640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f49840c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f49840c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 -- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840d1b90 msgr2=0x7f4984150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.367+0000 7f4982a00640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840d1b90 0x7f4984150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f4982a00640 1 -- 172.21.15.97:0/1685103464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c0094d0 con 0x7f49840b2570 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f4983400640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840d1b90 0x7f4984150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:35.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f4982a00640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 0x7f498414de20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f497400cee0 tx=0x7f497400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400d840 con 0x7f49840b2570 2024-01-31T05:03:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400d9a0 con 0x7f49840b2570 2024-01-31T05:03:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400dd30 con 0x7f49840b2570 2024-01-31T05:03:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49841f6890 con 0x7f49840b2570 2024-01-31T05:03:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.368+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49841f6e50 con 0x7f49840b2570 2024-01-31T05:03:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.370+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4984067830 con 0x7f49840b2570 2024-01-31T05:03:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.370+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4974015440 con 0x7f49840b2570 2024-01-31T05:03:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.372+0000 7f497aa00640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f495c06aed0 0x7f495c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.372+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4974089c80 con 0x7f49840b2570 2024-01-31T05:03:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.372+0000 7f4982000640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f495c06aed0 0x7f495c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.373+0000 7f4982000640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f495c06aed0 0x7f495c06d390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f496c036000 tx=0x7f496c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:35.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.376+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4974054ea0 con 0x7f49840b2570 2024-01-31T05:03:35.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.571+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49840d17d0 con 0x7f49840b2570 2024-01-31T05:03:35.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.572+0000 7f497aa00640 1 -- 172.21.15.97:0/1685103464 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f497401a020 con 0x7f49840b2570 2024-01-31T05:03:35.571 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:35.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.574+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f495c06aed0 msgr2=0x7f495c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.574+0000 7f4989800640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f495c06aed0 0x7f495c06d390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f496c036000 tx=0x7f496c03c040 comp rx=0 tx=0).stop 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 msgr2=0x7f498414de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 0x7f498414de20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f497400cee0 tx=0x7f497400a710 comp rx=0 tx=0).stop 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 shutdown_connections 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49840d1b90 0x7f4984150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49840cde30 0x7f49840c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f495c06aed0 0x7f495c06d390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 --2- 172.21.15.97:0/1685103464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49840b2570 0x7f498414de20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 >> 172.21.15.97:0/1685103464 conn(0x7f49840d1070 msgr2=0x7f498406fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 shutdown_connections 2024-01-31T05:03:35.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.575+0000 7f4989800640 1 -- 172.21.15.97:0/1685103464 wait complete. 2024-01-31T05:03:35.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-31T05:03:35.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.799+0000 7f3380600640 1 Processor -- start 2024-01-31T05:03:35.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.799+0000 7f3380600640 1 -- start start 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f3380600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f33780cb810 0x7f33780cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f3380600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f33780ccd00 0x7f33780cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f3380600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337806d1a0 con 0x7f337815b400 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f3380600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33780cc960 con 0x7f33780ccd00 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f3380600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337806c600 con 0x7f33780cb810 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f337d200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f337815b400 0x7f337806d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55302/0 (socket says 172.21.15.97:55302) 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f337d200640 1 -- 172.21.15.97:0/3688360097 learned_addr learned my addr 172.21.15.97:0/3688360097 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:35.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.801+0000 7f337e600640 1 --2- 172.21.15.97:0/3688360097 >> v2:172.21.15.97:3300/0 conn(0x7f33780ccd00 0x7f33780cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f337dc00640 1 --2- 172.21.15.97:0/3688360097 >> v2:172.21.15.188:3300/0 conn(0x7f33780cb810 0x7f33780cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f3377e00640 1 -- 172.21.15.97:0/3688360097 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020042325 0 0) 0x7f337806d1a0 con 0x7f337815b400 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f3377e00640 1 -- 172.21.15.97:0/3688360097 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3358003520 con 0x7f337815b400 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f337e600640 1 -- 172.21.15.97:0/3688360097 >> v2:172.21.15.188:3300/0 conn(0x7f33780cb810 msgr2=0x7f33780cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f337e600640 1 --2- 172.21.15.97:0/3688360097 >> v2:172.21.15.188:3300/0 conn(0x7f33780cb810 0x7f33780cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f337e600640 1 -- 172.21.15.97:0/3688360097 >> v1:172.21.15.97:6789/0 conn(0x7f337815b400 legacy=0x7f337806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.802+0000 7f337e600640 1 -- 172.21.15.97:0/3688360097 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33780cd6b0 con 0x7f33780ccd00 2024-01-31T05:03:35.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.803+0000 7f337e600640 1 --2- 172.21.15.97:0/3688360097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780ccd00 0x7f33780cd170 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3378067410 tx=0x7f337002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267cd7da028b9dcb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.803+0000 7f3377e00640 1 -- 172.21.15.97:0/3688360097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002cc80 con 0x7f33780ccd00 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.803+0000 7f3377e00640 1 -- 172.21.15.97:0/3688360097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337002cde0 con 0x7f33780ccd00 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.803+0000 7f3377e00640 1 -- 172.21.15.97:0/3688360097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370034660 con 0x7f33780ccd00 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 -- 172.21.15.97:0/3688360097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780ccd00 msgr2=0x7f33780cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 --2- 172.21.15.97:0/3688360097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780ccd00 0x7f33780cd170 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3378067410 tx=0x7f337002bed0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 -- 172.21.15.97:0/3688360097 shutdown_connections 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 --2- 172.21.15.97:0/3688360097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780ccd00 0x7f33780cd170 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 --2- 172.21.15.97:0/3688360097 >> v2:172.21.15.188:3300/0 conn(0x7f33780cb810 0x7f33780cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 -- 172.21.15.97:0/3688360097 >> 172.21.15.97:0/3688360097 conn(0x7f33780b0a10 msgr2=0x7f33780b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 -- 172.21.15.97:0/3688360097 shutdown_connections 2024-01-31T05:03:35.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.804+0000 7f3380600640 1 -- 172.21.15.97:0/3688360097 wait complete. 2024-01-31T05:03:35.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.806+0000 7f3380600640 1 Processor -- start 2024-01-31T05:03:35.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.806+0000 7f3380600640 1 -- start start 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f3380600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 0x7f33781529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f3380600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33780ccd00 0x7f33781f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f3380600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f337815b400 0x7f337815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f3380600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780cd8a0 con 0x7f33780cb810 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f3380600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780ce420 con 0x7f33780ccd00 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f3380600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3378152f00 con 0x7f337815b400 2024-01-31T05:03:35.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f337dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 0x7f33781529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f337dc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 0x7f33781529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42122/0 (socket says 172.21.15.97:42122) 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f337dc00640 1 -- 172.21.15.97:0/950406940 learned_addr learned my addr 172.21.15.97:0/950406940 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.807+0000 7f337d200640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33780ccd00 0x7f33781f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337dc00640 1 -- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f337815b400 msgr2=0x7f337815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337dc00640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f337815b400 0x7f337815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337dc00640 1 -- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33780ccd00 msgr2=0x7f33781f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337dc00640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33780ccd00 0x7f33781f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337dc00640 1 -- 172.21.15.97:0/950406940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370009a70 con 0x7f33780cb810 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337d200640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33780ccd00 0x7f33781f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:35.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f337dc00640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 0x7f33781529c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f336800d800 tx=0x7f336800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800b6c0 con 0x7f33780cb810 2024-01-31T05:03:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336800b820 con 0x7f33780cb810 2024-01-31T05:03:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33780cda80 con 0x7f33780cb810 2024-01-31T05:03:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336800bbb0 con 0x7f33780cb810 2024-01-31T05:03:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.808+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33780ce040 con 0x7f33780cb810 2024-01-31T05:03:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.809+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33780677c0 con 0x7f33780cb810 2024-01-31T05:03:35.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.811+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f336800bd10 con 0x7f33780cb810 2024-01-31T05:03:35.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.811+0000 7f3375600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f335006aed0 0x7f335006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:35.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.812+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3368089970 con 0x7f33780cb810 2024-01-31T05:03:35.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.812+0000 7f337d200640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f335006aed0 0x7f335006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:35.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.812+0000 7f337d200640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f335006aed0 0x7f335006d390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3378070830 tx=0x7f3360009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:35.816+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3368054ac0 con 0x7f33780cb810 2024-01-31T05:03:36.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.011+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f33780ac6c0 con 0x7f33780cb810 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.012+0000 7f3375600640 1 -- 172.21.15.97:0/950406940 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7f33680580e0 con 0x7f33780cb810 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:36.012 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-01-31T05:03:36.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f335006aed0 msgr2=0x7f335006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f335006aed0 0x7f335006d390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3378070830 tx=0x7f3360009040 comp rx=0 tx=0).stop 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 msgr2=0x7f33781529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 0x7f33781529c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f336800d800 tx=0x7f336800dcc0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 shutdown_connections 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f337815b400 0x7f337815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33780ccd00 0x7f33781f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f335006aed0 0x7f335006d390 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 --2- 172.21.15.97:0/950406940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33780cb810 0x7f33781529c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 >> 172.21.15.97:0/950406940 conn(0x7f33780b0a10 msgr2=0x7f33780abfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 shutdown_connections 2024-01-31T05:03:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.015+0000 7f3380600640 1 -- 172.21.15.97:0/950406940 wait complete. 2024-01-31T05:03:36.031 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-31T05:03:36.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-31T05:03:36.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.245+0000 7fd050e00640 1 Processor -- start 2024-01-31T05:03:36.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.245+0000 7fd050e00640 1 -- start start 2024-01-31T05:03:36.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd050e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd04c15ed30 0x7fd04c0721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:36.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd050e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c06e730 con 0x7fd04c0c9280 2024-01-31T05:03:36.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd050e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c155c10 con 0x7fd04c15f110 2024-01-31T05:03:36.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd050e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c072730 con 0x7fd04c15ed30 2024-01-31T05:03:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd04b400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd04c15ed30 0x7fd04c0721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd04be00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd04c0c9280 0x7fd04c0c96b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55318/0 (socket says 172.21.15.97:55318) 2024-01-31T05:03:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.247+0000 7fd04be00640 1 -- 172.21.15.97:0/3152726741 learned_addr learned my addr 172.21.15.97:0/3152726741 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.248+0000 7fd04b400640 1 -- 172.21.15.97:0/3152726741 >> v1:172.21.15.188:6789/0 conn(0x7fd04c15f110 legacy=0x7fd04c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.248+0000 7fd04b400640 1 -- 172.21.15.97:0/3152726741 >> v1:172.21.15.97:6789/0 conn(0x7fd04c0c9280 legacy=0x7fd04c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:36.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.248+0000 7fd04b400640 1 -- 172.21.15.97:0/3152726741 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c06e450 con 0x7fd04c15ed30 2024-01-31T05:03:36.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.249+0000 7fd04b400640 1 --2- 172.21.15.97:0/3152726741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0721f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097d0 tx=0x7fd03c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa79f597b003ff5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:36.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.249+0000 7fd04a000640 1 -- 172.21.15.97:0/3152726741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c031780 con 0x7fd04c15ed30 2024-01-31T05:03:36.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.249+0000 7fd04a000640 1 -- 172.21.15.97:0/3152726741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c0318e0 con 0x7fd04c15ed30 2024-01-31T05:03:36.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.249+0000 7fd04a000640 1 -- 172.21.15.97:0/3152726741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c031c70 con 0x7fd04c15ed30 2024-01-31T05:03:36.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.249+0000 7fd050e00640 1 -- 172.21.15.97:0/3152726741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 msgr2=0x7fd04c0721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:36.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.249+0000 7fd050e00640 1 --2- 172.21.15.97:0/3152726741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0721f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097d0 tx=0x7fd03c02b820 comp rx=0 tx=0).stop 2024-01-31T05:03:36.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.250+0000 7fd050e00640 1 -- 172.21.15.97:0/3152726741 shutdown_connections 2024-01-31T05:03:36.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.250+0000 7fd050e00640 1 --2- 172.21.15.97:0/3152726741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0721f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.250+0000 7fd050e00640 1 -- 172.21.15.97:0/3152726741 >> 172.21.15.97:0/3152726741 conn(0x7fd04c0cec30 msgr2=0x7fd04c0cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:36.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.250+0000 7fd050e00640 1 -- 172.21.15.97:0/3152726741 shutdown_connections 2024-01-31T05:03:36.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.250+0000 7fd050e00640 1 -- 172.21.15.97:0/3152726741 wait complete. 2024-01-31T05:03:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.251+0000 7fd050e00640 1 Processor -- start 2024-01-31T05:03:36.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.251+0000 7fd050e00640 1 -- start start 2024-01-31T05:03:36.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.252+0000 7fd050e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c0c9280 0x7fd04c1f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:36.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.252+0000 7fd050e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:36.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.252+0000 7fd050e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15f110 0x7fd04c157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:36.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.252+0000 7fd050e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1f1f30 con 0x7fd04c15ed30 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.252+0000 7fd050e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15a820 con 0x7fd04c15f110 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.252+0000 7fd050e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15ab00 con 0x7fd04c0c9280 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42154/0 (socket says 172.21.15.97:42154) 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 -- 172.21.15.97:0/271727509 learned_addr learned my addr 172.21.15.97:0/271727509 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04b400640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c0c9280 0x7fd04c1f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04be00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15f110 0x7fd04c157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 -- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c0c9280 msgr2=0x7fd04c1f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c0c9280 0x7fd04c1f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 -- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15f110 msgr2=0x7fd04c157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15f110 0x7fd04c157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.253+0000 7fd04aa00640 1 -- 172.21.15.97:0/271727509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c009400 con 0x7fd04c15ed30 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.254+0000 7fd04aa00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0afa90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c430 tx=0x7fd03400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:36.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.254+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034011790 con 0x7fd04c15ed30 2024-01-31T05:03:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.254+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0340118f0 con 0x7fd04c15ed30 2024-01-31T05:03:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.254+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034011ce0 con 0x7fd04c15ed30 2024-01-31T05:03:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.254+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c15ade0 con 0x7fd04c15ed30 2024-01-31T05:03:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.254+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c15b3a0 con 0x7fd04c15ed30 2024-01-31T05:03:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.255+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c0c6a60 con 0x7fd04c15ed30 2024-01-31T05:03:36.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.256+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd03401c460 con 0x7fd04c15ed30 2024-01-31T05:03:36.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.257+0000 7fd043400640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd02406aed0 0x7fd02406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:36.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.257+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd0340894b0 con 0x7fd04c15ed30 2024-01-31T05:03:36.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.257+0000 7fd04b400640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd02406aed0 0x7fd02406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:36.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.258+0000 7fd04b400640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd02406aed0 0x7fd02406d390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd03c036000 tx=0x7fd03c03d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:36.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.261+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd0340544f0 con 0x7fd04c15ed30 2024-01-31T05:03:36.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:36.457+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd04c0676b0 con 0x7fd04c15ed30 2024-01-31T05:03:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.190+0000 7fd043400640 1 -- 172.21.15.97:0/271727509 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd034057b10 con 0x7fd04c15ed30 2024-01-31T05:03:37.190 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd02406aed0 msgr2=0x7fd02406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd02406aed0 0x7fd02406d390 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd03c036000 tx=0x7fd03c03d040 comp rx=0 tx=0).stop 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 msgr2=0x7fd04c0afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0afa90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c430 tx=0x7fd03400c8f0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 shutdown_connections 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15f110 0x7fd04c157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd02406aed0 0x7fd02406d390 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd04c15ed30 0x7fd04c0afa90 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 --2- 172.21.15.97:0/271727509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c0c9280 0x7fd04c1f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 >> 172.21.15.97:0/271727509 conn(0x7fd04c0cec30 msgr2=0x7fd04c0c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.194+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 shutdown_connections 2024-01-31T05:03:37.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.195+0000 7fd050e00640 1 -- 172.21.15.97:0/271727509 wait complete. 2024-01-31T05:03:37.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-31T05:03:37.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.422+0000 7fef33000640 1 Processor -- start 2024-01-31T05:03:37.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.423+0000 7fef33000640 1 -- start start 2024-01-31T05:03:37.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.424+0000 7fef33000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef2c15a350 0x7fef2c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.424+0000 7fef33000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef2c15a730 0x7fef2c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.424+0000 7fef33000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c06dfc0 0x7fef2c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.424+0000 7fef33000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c0b2c90 con 0x7fef2c15a350 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.424+0000 7fef33000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2c15abc0 con 0x7fef2c15a730 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.424+0000 7fef33000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c15aea0 con 0x7fef2c06dfc0 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef2c15a350 0x7fef2c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef2c15a350 0x7fef2c0abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42166/0 (socket says 172.21.15.97:42166) 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 -- 172.21.15.97:0/1802860204 learned_addr learned my addr 172.21.15.97:0/1802860204 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef31000640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c06dfc0 0x7fef2c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2b400640 1 --2- 172.21.15.97:0/1802860204 >> v2:172.21.15.188:3300/0 conn(0x7fef2c15a730 0x7fef2c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 -- 172.21.15.97:0/1802860204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c06dfc0 msgr2=0x7fef2c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c06dfc0 0x7fef2c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 -- 172.21.15.97:0/1802860204 >> v2:172.21.15.188:3300/0 conn(0x7fef2c15a730 msgr2=0x7fef2c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 --2- 172.21.15.97:0/1802860204 >> v2:172.21.15.188:3300/0 conn(0x7fef2c15a730 0x7fef2c06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2be00640 1 -- 172.21.15.97:0/1802860204 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c15b180 con 0x7fef2c15a350 2024-01-31T05:03:37.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef2b400640 1 --2- 172.21.15.97:0/1802860204 >> v2:172.21.15.188:3300/0 conn(0x7fef2c15a730 0x7fef2c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:37.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.425+0000 7fef31000640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c06dfc0 0x7fef2c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.427+0000 7fef2be00640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15a350 0x7fef2c0abcf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fef2c06c300 tx=0x7fef1c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1849ae34b9731d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.427+0000 7fef2aa00640 1 -- 172.21.15.97:0/1802860204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c02ee70 con 0x7fef2c15a350 2024-01-31T05:03:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.427+0000 7fef2aa00640 1 -- 172.21.15.97:0/1802860204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c004030 con 0x7fef2c15a350 2024-01-31T05:03:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.427+0000 7fef2aa00640 1 -- 172.21.15.97:0/1802860204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c004420 con 0x7fef2c15a350 2024-01-31T05:03:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 -- 172.21.15.97:0/1802860204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15a350 msgr2=0x7fef2c0abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:37.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15a350 0x7fef2c0abcf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fef2c06c300 tx=0x7fef1c02e0c0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 -- 172.21.15.97:0/1802860204 shutdown_connections 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c06dfc0 0x7fef2c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 --2- 172.21.15.97:0/1802860204 >> v2:172.21.15.188:3300/0 conn(0x7fef2c15a730 0x7fef2c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 --2- 172.21.15.97:0/1802860204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15a350 0x7fef2c0abcf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 -- 172.21.15.97:0/1802860204 >> 172.21.15.97:0/1802860204 conn(0x7fef2c0b0a10 msgr2=0x7fef2c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 -- 172.21.15.97:0/1802860204 shutdown_connections 2024-01-31T05:03:37.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.428+0000 7fef33000640 1 -- 172.21.15.97:0/1802860204 wait complete. 2024-01-31T05:03:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.430+0000 7fef33000640 1 Processor -- start 2024-01-31T05:03:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.430+0000 7fef33000640 1 -- start start 2024-01-31T05:03:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef33000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c06dfc0 0x7fef2c14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef33000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c15a350 0x7fef2c15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef33000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 0x7fef2c0715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef33000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c14ef50 con 0x7fef2c15ce10 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef33000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c15b8c0 con 0x7fef2c06dfc0 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef33000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c15bb10 con 0x7fef2c15a350 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 0x7fef2c0715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 0x7fef2c0715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42176/0 (socket says 172.21.15.97:42176) 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 -- 172.21.15.97:0/3875805738 learned_addr learned my addr 172.21.15.97:0/3875805738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef2b400640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c15a350 0x7fef2c15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef2be00640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c06dfc0 0x7fef2c14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 -- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c15a350 msgr2=0x7fef2c15c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c15a350 0x7fef2c15c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 -- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c06dfc0 msgr2=0x7fef2c14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c06dfc0 0x7fef2c14ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef31000640 1 -- 172.21.15.97:0/3875805738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c00b700 con 0x7fef2c15ce10 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.431+0000 7fef2b400640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c15a350 0x7fef2c15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.432+0000 7fef31000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 0x7fef2c0715a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fef2400cef0 tx=0x7fef2400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.432+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400d840 con 0x7fef2c15ce10 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.432+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2400d9a0 con 0x7fef2c15ce10 2024-01-31T05:03:37.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.432+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400dd50 con 0x7fef2c15ce10 2024-01-31T05:03:37.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.432+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2c15bdf0 con 0x7fef2c15ce10 2024-01-31T05:03:37.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.432+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c15c3b0 con 0x7fef2c15ce10 2024-01-31T05:03:37.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.433+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c15c5d0 con 0x7fef2c15ce10 2024-01-31T05:03:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.435+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fef2401e3b0 con 0x7fef2c15ce10 2024-01-31T05:03:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.436+0000 7fef23e00640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef0406aed0 0x7fef0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.436+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fef2408a3f0 con 0x7fef2c15ce10 2024-01-31T05:03:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.436+0000 7fef2be00640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef0406aed0 0x7fef0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:37.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.437+0000 7fef2be00640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef0406aed0 0x7fef0406d390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef1c037000 tx=0x7fef1c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:37.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.440+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fef240553d0 con 0x7fef2c15ce10 2024-01-31T05:03:37.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:37.635+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fef2c067830 con 0x7fef2c15ce10 2024-01-31T05:03:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.196+0000 7fef23e00640 1 -- 172.21.15.97:0/3875805738 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7fef240589f0 con 0x7fef2c15ce10 2024-01-31T05:03:38.196 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-01-31T05:03:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.199+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef0406aed0 msgr2=0x7fef0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.199+0000 7fef33000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef0406aed0 0x7fef0406d390 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fef1c037000 tx=0x7fef1c032000 comp rx=0 tx=0).stop 2024-01-31T05:03:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 msgr2=0x7fef2c0715a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:38.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 0x7fef2c0715a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fef2400cef0 tx=0x7fef2400a710 comp rx=0 tx=0).stop 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 shutdown_connections 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fef0406aed0 0x7fef0406d390 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef2c15ce10 0x7fef2c0715a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef2c15a350 0x7fef2c15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 --2- 172.21.15.97:0/3875805738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef2c06dfc0 0x7fef2c14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 >> 172.21.15.97:0/3875805738 conn(0x7fef2c0b0a10 msgr2=0x7fef2c0cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 shutdown_connections 2024-01-31T05:03:38.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.200+0000 7fef33000640 1 -- 172.21.15.97:0/3875805738 wait complete. 2024-01-31T05:03:38.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-31T05:03:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.411+0000 7f2041400640 1 Processor -- start 2024-01-31T05:03:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.411+0000 7f2041400640 1 -- start start 2024-01-31T05:03:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2041400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f203c15b080 0x7f203c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2041400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 0x7f203c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2041400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c0b3f60 con 0x7f203c0ae360 2024-01-31T05:03:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2041400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0b41c0 con 0x7f203c15b080 2024-01-31T05:03:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2041400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c15c540 con 0x7f203c0ce680 2024-01-31T05:03:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2033e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f203c0ae360 0x7f203c0cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55326/0 (socket says 172.21.15.97:55326) 2024-01-31T05:03:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.413+0000 7f2033e00640 1 -- 172.21.15.97:0/1041940127 learned_addr learned my addr 172.21.15.97:0/1041940127 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203be00640 1 --2- 172.21.15.97:0/1041940127 >> v2:172.21.15.188:3300/0 conn(0x7f203c15b080 0x7f203c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748672884 0 0) 0x7f203c0b3f60 con 0x7f203c0ae360 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f2040a00640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 0x7f203c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f201c003520 con 0x7f203c0ae360 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370315870 0 0) 0x7f201c003520 con 0x7f203c0ae360 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 msgr2=0x7f203c0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 0x7f203c0b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 >> v2:172.21.15.188:3300/0 conn(0x7f203c15b080 msgr2=0x7f203c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 --2- 172.21.15.97:0/1041940127 >> v2:172.21.15.188:3300/0 conn(0x7f203c15b080 0x7f203c0b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c15b780 con 0x7f203c0ae360 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f2024001670 con 0x7f203c0ae360 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 >> v1:172.21.15.97:6789/0 conn(0x7f203c0ae360 legacy=0x7f203c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f201c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f201c0047c0 0x7f201c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 0x7f201c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0b41c0 con 0x7f201c0039e0 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c15c540 con 0x7f201c0047c0 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201c00ba30 con 0x7f203c0ce680 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.414+0000 7f203be00640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f201c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.415+0000 7f203be00640 1 -- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 msgr2=0x7f201c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.415+0000 7f203be00640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c0ce680 0x7f201c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.415+0000 7f203be00640 1 -- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f201c0047c0 msgr2=0x7f201c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.415+0000 7f203be00640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f201c0047c0 0x7f201c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.415+0000 7f203be00640 1 -- 172.21.15.97:0/1041940127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c003520 con 0x7f201c0039e0 2024-01-31T05:03:38.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.416+0000 7f203be00640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f201c003e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f202c009ac0 tx=0x7f201c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e1fac560a6fdb8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.416+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c02c710 con 0x7f201c0039e0 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.416+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c02c870 con 0x7f201c0039e0 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.416+0000 7f203b400640 1 -- 172.21.15.97:0/1041940127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c02cc00 con 0x7f201c0039e0 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2041400640 1 -- 172.21.15.97:0/1041940127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 msgr2=0x7f201c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2041400640 1 --2- 172.21.15.97:0/1041940127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f201c003e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f202c009ac0 tx=0x7f201c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2040a00640 1 -- 172.21.15.97:0/1041940127 reap_dead start 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2041400640 1 -- 172.21.15.97:0/1041940127 shutdown_connections 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2041400640 1 -- 172.21.15.97:0/1041940127 >> 172.21.15.97:0/1041940127 conn(0x7f203c06ad40 msgr2=0x7f203c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2041400640 1 -- 172.21.15.97:0/1041940127 shutdown_connections 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.417+0000 7f2041400640 1 -- 172.21.15.97:0/1041940127 wait complete. 2024-01-31T05:03:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.418+0000 7f2041400640 1 Processor -- start 2024-01-31T05:03:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.419+0000 7f2041400640 1 -- start start 2024-01-31T05:03:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.419+0000 7f2041400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f203c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.419+0000 7f2041400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c09b2b0 0x7f203c14a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f2041400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f203c0999c0 0x7f203c09be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f2041400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c09a210 con 0x7f201c0039e0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f2041400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c09a4f0 con 0x7f203c0999c0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f2041400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c09a7d0 con 0x7f203c09b2b0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f203c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f203c158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42190/0 (socket says 172.21.15.97:42190) 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 -- 172.21.15.97:0/124790918 learned_addr learned my addr 172.21.15.97:0/124790918 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 -- 172.21.15.97:0/124790918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c09b2b0 msgr2=0x7f203c14a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c09b2b0 0x7f203c14a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 -- 172.21.15.97:0/124790918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f203c0999c0 msgr2=0x7f203c09be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f203c0999c0 0x7f203c09be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.420+0000 7f203be00640 1 -- 172.21.15.97:0/124790918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c09aab0 con 0x7f201c0039e0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.421+0000 7f203be00640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f203c158350 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f202c000c00 tx=0x7f202c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.421+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c003810 con 0x7f201c0039e0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.421+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c02de90 con 0x7f201c0039e0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.421+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c003b70 con 0x7f201c0039e0 2024-01-31T05:03:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.421+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c09ad90 con 0x7f201c0039e0 2024-01-31T05:03:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.421+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c09c340 con 0x7f201c0039e0 2024-01-31T05:03:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.422+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c0676b0 con 0x7f201c0039e0 2024-01-31T05:03:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.423+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f202c003cd0 con 0x7f201c0039e0 2024-01-31T05:03:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.424+0000 7f2038c00640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f201406aed0 0x7f201406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.424+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f202c0ab770 con 0x7f201c0039e0 2024-01-31T05:03:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.424+0000 7f2033e00640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f201406aed0 0x7f201406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:38.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.425+0000 7f2033e00640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f201406aed0 0x7f201406d390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20240018a0 tx=0x7f2024001550 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:38.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.429+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f202c076550 con 0x7f201c0039e0 2024-01-31T05:03:38.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:38.643+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f203c0ce440 con 0x7f201c0039e0 2024-01-31T05:03:39.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.244+0000 7f2038c00640 1 -- 172.21.15.97:0/124790918 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7f202c079b70 con 0x7f201c0039e0 2024-01-31T05:03:39.247 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 8 and data pool 9 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f201406aed0 msgr2=0x7f201406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f201406aed0 0x7f201406d390 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f20240018a0 tx=0x7f2024001550 comp rx=0 tx=0).stop 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 msgr2=0x7f203c158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f203c158350 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f202c000c00 tx=0x7f202c004730 comp rx=0 tx=0).stop 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 shutdown_connections 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f203c0999c0 0x7f203c09be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f203c09b2b0 0x7f203c14a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f201406aed0 0x7f201406d390 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 --2- 172.21.15.97:0/124790918 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f201c0039e0 0x7f203c158350 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 >> 172.21.15.97:0/124790918 conn(0x7f203c06ad40 msgr2=0x7f203c0ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:39.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 shutdown_connections 2024-01-31T05:03:39.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.261+0000 7f2041400640 1 -- 172.21.15.97:0/124790918 wait complete. 2024-01-31T05:03:39.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:39.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.464+0000 7f4b6ae00640 1 Processor -- start 2024-01-31T05:03:39.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.464+0000 7f4b6ae00640 1 -- start start 2024-01-31T05:03:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.465+0000 7f4b6ae00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b640ae3b0 0x7f4b6406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.465+0000 7f4b6ae00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b64070400 con 0x7f4b6415ec70 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.465+0000 7f4b6ae00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b640ca580 con 0x7f4b6406d0d0 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.465+0000 7f4b6ae00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b640ad8c0 con 0x7f4b640ae3b0 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.466+0000 7f4b69000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b640ae3b0 0x7f4b6406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.466+0000 7f4b69a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4b6415ec70 0x7f4b64071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55330/0 (socket says 172.21.15.97:55330) 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.466+0000 7f4b69a00640 1 -- 172.21.15.97:0/3259325229 learned_addr learned my addr 172.21.15.97:0/3259325229 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.466+0000 7f4b69000640 1 -- 172.21.15.97:0/3259325229 >> v1:172.21.15.188:6789/0 conn(0x7f4b6406d0d0 legacy=0x7f4b6415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.466+0000 7f4b69000640 1 -- 172.21.15.97:0/3259325229 >> v1:172.21.15.97:6789/0 conn(0x7f4b6415ec70 legacy=0x7f4b64071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.466+0000 7f4b69000640 1 -- 172.21.15.97:0/3259325229 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b64070650 con 0x7f4b640ae3b0 2024-01-31T05:03:39.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.467+0000 7f4b69000640 1 --2- 172.21.15.97:0/3259325229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b640ae3b0 0x7f4b6406cb90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0097a0 tx=0x7f4b4c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e5e0e4968bad81 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:39.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b63e00640 1 -- 172.21.15.97:0/3259325229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c02f050 con 0x7f4b640ae3b0 2024-01-31T05:03:39.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b63e00640 1 -- 172.21.15.97:0/3259325229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c031700 con 0x7f4b640ae3b0 2024-01-31T05:03:39.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b63e00640 1 -- 172.21.15.97:0/3259325229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c031a90 con 0x7f4b640ae3b0 2024-01-31T05:03:39.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 -- 172.21.15.97:0/3259325229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b640ae3b0 msgr2=0x7f4b6406cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/3259325229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b640ae3b0 0x7f4b6406cb90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0097a0 tx=0x7f4b4c02b7a0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 -- 172.21.15.97:0/3259325229 shutdown_connections 2024-01-31T05:03:39.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/3259325229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b640ae3b0 0x7f4b6406cb90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 -- 172.21.15.97:0/3259325229 >> 172.21.15.97:0/3259325229 conn(0x7f4b640cec40 msgr2=0x7f4b640cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:39.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 -- 172.21.15.97:0/3259325229 shutdown_connections 2024-01-31T05:03:39.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.468+0000 7f4b6ae00640 1 -- 172.21.15.97:0/3259325229 wait complete. 2024-01-31T05:03:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.470+0000 7f4b6ae00640 1 Processor -- start 2024-01-31T05:03:39.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.470+0000 7f4b6ae00640 1 -- start start 2024-01-31T05:03:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.471+0000 7f4b6ae00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b6406d0d0 0x7f4b6409bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.471+0000 7f4b6ae00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b640ae3b0 0x7f4b6409c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.471+0000 7f4b6ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 0x7f4b64143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.471+0000 7f4b6ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6415e1e0 con 0x7f4b6415ec70 2024-01-31T05:03:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.471+0000 7f4b6ae00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6415d160 con 0x7f4b640ae3b0 2024-01-31T05:03:39.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.471+0000 7f4b6ae00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6415d420 con 0x7f4b6406d0d0 2024-01-31T05:03:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 0x7f4b64143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 0x7f4b64143750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42204/0 (socket says 172.21.15.97:42204) 2024-01-31T05:03:39.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 -- 172.21.15.97:0/1042145716 learned_addr learned my addr 172.21.15.97:0/1042145716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 -- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b6406d0d0 msgr2=0x7f4b6409bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b69a00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b6406d0d0 0x7f4b6409bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b69000640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b640ae3b0 0x7f4b6409c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b6406d0d0 0x7f4b6409bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 -- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b640ae3b0 msgr2=0x7f4b6409c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b640ae3b0 0x7f4b6409c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 -- 172.21.15.97:0/1042145716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009400 con 0x7f4b6415ec70 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.472+0000 7f4b6a400640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 0x7f4b64143750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00b970 tx=0x7f4b5c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.473+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00cb40 con 0x7f4b6415ec70 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.473+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c00cca0 con 0x7f4b6415ec70 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.473+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c014680 con 0x7f4b6415ec70 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.473+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6415d700 con 0x7f4b6415ec70 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.473+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6415dcc0 con 0x7f4b6415ec70 2024-01-31T05:03:39.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.474+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6415deb0 con 0x7f4b6415ec70 2024-01-31T05:03:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.475+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4b5c01d3b0 con 0x7f4b6415ec70 2024-01-31T05:03:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.476+0000 7f4b61600640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b3c06af00 0x7f4b3c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.476+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4b5c0896a0 con 0x7f4b6415ec70 2024-01-31T05:03:39.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.476+0000 7f4b69a00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b3c06af00 0x7f4b3c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.476 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.477+0000 7f4b69a00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b3c06af00 0x7f4b3c06d3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000f60 tx=0x7f4b54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:39.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.481+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f4b5c054300 con 0x7f4b6415ec70 2024-01-31T05:03:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.673+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b640676b0 con 0x7f4b6415ec70 2024-01-31T05:03:39.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.675+0000 7f4b61600640 1 -- 172.21.15.97:0/1042145716 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f4b5c057920 con 0x7f4b6415ec70 2024-01-31T05:03:39.675 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:39.676 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:38.645283+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b3c06af00 msgr2=0x7f4b3c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b3c06af00 0x7f4b3c06d3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000f60 tx=0x7f4b54009040 comp rx=0 tx=0).stop 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 msgr2=0x7f4b64143750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 0x7f4b64143750 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00b970 tx=0x7f4b5c00be30 comp rx=0 tx=0).stop 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 shutdown_connections 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4b3c06af00 0x7f4b3c06d3c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6415ec70 0x7f4b64143750 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b640ae3b0 0x7f4b6409c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 --2- 172.21.15.97:0/1042145716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b6406d0d0 0x7f4b6409bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 >> 172.21.15.97:0/1042145716 conn(0x7f4b640cec40 msgr2=0x7f4b640afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:39.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 shutdown_connections 2024-01-31T05:03:39.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.678+0000 7f4b6ae00640 1 -- 172.21.15.97:0/1042145716 wait complete. 2024-01-31T05:03:39.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:39.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.887+0000 7f7ad6e00640 1 Processor -- start 2024-01-31T05:03:39.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.887+0000 7f7ad6e00640 1 -- start start 2024-01-31T05:03:39.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.888+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 0x7f7ad015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.888+0000 7f7ad6e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad006c8b0 con 0x7f7ad00b1d10 2024-01-31T05:03:39.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7ad6e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad015b120 con 0x7f7ad006d090 2024-01-31T05:03:39.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad015b400 con 0x7f7ad015a600 2024-01-31T05:03:39.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acfe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ad00b1d10 0x7f7ad00b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55344/0 (socket says 172.21.15.97:55344) 2024-01-31T05:03:39.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acfe00640 1 -- 172.21.15.97:0/1688582771 learned_addr learned my addr 172.21.15.97:0/1688582771 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813186075 0 0) 0x7f7ad006c8b0 con 0x7f7ad00b1d10 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab0005730 con 0x7f7ad00b1d10 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acf400640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 0x7f7ad015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895112275 0 0) 0x7f7ab0005730 con 0x7f7ad00b1d10 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 msgr2=0x7f7ad015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 0x7f7ad015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 >> v1:172.21.15.188:6789/0 conn(0x7f7ad006d090 legacy=0x7f7ad006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad006cb70 con 0x7f7ad00b1d10 2024-01-31T05:03:39.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.889+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f7ac0001670 con 0x7f7ad00b1d10 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 >> v1:172.21.15.97:6789/0 conn(0x7f7ad00b1d10 legacy=0x7f7ad00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0005bb0 0x7f7ab0005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 0x7f7ab00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab0006f10 0x7f7ab0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad015b400 con 0x7f7ab0006f10 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab000dd20 con 0x7f7ab0005bb0 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab000dee0 con 0x7f7ad015a600 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7ad4e00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab0006f10 0x7f7ab0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acfe00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0005bb0 0x7f7ab0005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7acf400640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 0x7f7ab00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7ad4e00640 1 -- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 msgr2=0x7f7ab00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7ad4e00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad015a600 0x7f7ab00069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7ad4e00640 1 -- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0005bb0 msgr2=0x7f7ab0005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:39.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7ad4e00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0005bb0 0x7f7ab0005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.890+0000 7f7ad4e00640 1 -- 172.21.15.97:0/1688582771 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0005730 con 0x7f7ab0006f10 2024-01-31T05:03:39.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.891+0000 7f7acfe00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0005bb0 0x7f7ab0005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:39.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad4e00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab0006f10 0x7f7ab0007390 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80044b0 tx=0x7f7ab0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66c11a06844f1cb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:39.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8030620 con 0x7f7ab0006f10 2024-01-31T05:03:39.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac8030780 con 0x7f7ab0006f10 2024-01-31T05:03:39.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7acea00640 1 -- 172.21.15.97:0/1688582771 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8030b30 con 0x7f7ab0006f10 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad6e00640 1 -- 172.21.15.97:0/1688582771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab0006f10 msgr2=0x7f7ab0007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/1688582771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ab0006f10 0x7f7ab0007390 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80044b0 tx=0x7f7ab0007a00 comp rx=0 tx=0).stop 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad4e00640 1 -- 172.21.15.97:0/1688582771 reap_dead start 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad6e00640 1 -- 172.21.15.97:0/1688582771 shutdown_connections 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad6e00640 1 -- 172.21.15.97:0/1688582771 >> 172.21.15.97:0/1688582771 conn(0x7f7ad00b0a10 msgr2=0x7f7ad00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.892+0000 7f7ad6e00640 1 -- 172.21.15.97:0/1688582771 shutdown_connections 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.893+0000 7f7ad6e00640 1 -- 172.21.15.97:0/1688582771 wait complete. 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.894+0000 7f7ad6e00640 1 Processor -- start 2024-01-31T05:03:39.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.894+0000 7f7ad6e00640 1 -- start start 2024-01-31T05:03:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0006f10 0x7f7ad009c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 0x7f7ad0153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7ad6e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad014fa60 0x7f7ad0153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad009c980 con 0x7f7ad014f510 2024-01-31T05:03:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0072460 con 0x7f7ad014fa60 2024-01-31T05:03:39.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7ad6e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad015d550 con 0x7f7ab0006f10 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.895+0000 7f7acf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 0x7f7ad0153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 0x7f7ad0153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42224/0 (socket says 172.21.15.97:42224) 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 -- 172.21.15.97:0/3151692200 learned_addr learned my addr 172.21.15.97:0/3151692200 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 -- 172.21.15.97:0/3151692200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0006f10 msgr2=0x7f7ad009c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0006f10 0x7f7ad009c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 -- 172.21.15.97:0/3151692200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad014fa60 msgr2=0x7f7ad0153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad014fa60 0x7f7ad0153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 -- 172.21.15.97:0/3151692200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad0071ea0 con 0x7f7ad014f510 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7acf400640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 0x7f7ad0153a10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e7e0 tx=0x7f7ab800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800c650 con 0x7f7ad014f510 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab800c7b0 con 0x7f7ad014f510 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad0072180 con 0x7f7ad014f510 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800cb60 con 0x7f7ad014f510 2024-01-31T05:03:39.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.896+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad00999c0 con 0x7f7ad014f510 2024-01-31T05:03:39.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.898+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad0071d00 con 0x7f7ad014f510 2024-01-31T05:03:39.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.899+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7ab800ccc0 con 0x7f7ad014f510 2024-01-31T05:03:39.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.900+0000 7f7ac7e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7aa806af00 0x7f7aa806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:39.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.900+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7ab80893a0 con 0x7f7ad014f510 2024-01-31T05:03:39.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.900+0000 7f7acfe00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7aa806af00 0x7f7aa806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:39.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.901+0000 7f7acfe00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7aa806af00 0x7f7aa806d3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00017a0 tx=0x7f7ac0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:39.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:39.904+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7ad0071d00 con 0x7f7ad014f510 2024-01-31T05:03:40.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.084+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad0071d00 con 0x7f7ad014f510 2024-01-31T05:03:40.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.086+0000 7f7ac7e00640 1 -- 172.21.15.97:0/3151692200 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16784 (secure 0 0 0) 0x7f7ad0071d00 con 0x7f7ad014f510 2024-01-31T05:03:40.086 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:40.086 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":54,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:38.645283+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7aa806af00 msgr2=0x7f7aa806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7aa806af00 0x7f7aa806d3c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00017a0 tx=0x7f7ac0009040 comp rx=0 tx=0).stop 2024-01-31T05:03:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 msgr2=0x7f7ad0153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 0x7f7ad0153a10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800e7e0 tx=0x7f7ab800eca0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 shutdown_connections 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad014fa60 0x7f7ad0153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7aa806af00 0x7f7aa806d3c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad014f510 0x7f7ad0153a10 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 --2- 172.21.15.97:0/3151692200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0006f10 0x7f7ad009c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 >> 172.21.15.97:0/3151692200 conn(0x7f7ad00b0a10 msgr2=0x7f7ad01579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.088+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 shutdown_connections 2024-01-31T05:03:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.089+0000 7f7ad6e00640 1 -- 172.21.15.97:0/3151692200 wait complete. 2024-01-31T05:03:40.106 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-31T05:03:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.315+0000 7ffab6400640 1 Processor -- start 2024-01-31T05:03:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.315+0000 7ffab6400640 1 -- start start 2024-01-31T05:03:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.316+0000 7ffab6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 0x7ffab00ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.316+0000 7ffab6400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab0159d80 con 0x7ffab00adea0 2024-01-31T05:03:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.316+0000 7ffab6400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab00cc5e0 con 0x7ffab00ccd00 2024-01-31T05:03:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.316+0000 7ffab6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab015e210 con 0x7ffab01599a0 2024-01-31T05:03:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.316+0000 7ffaa7e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffab00adea0 0x7ffab00b1d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55350/0 (socket says 172.21.15.97:55350) 2024-01-31T05:03:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.316+0000 7ffaa7e00640 1 -- 172.21.15.97:0/4250255553 learned_addr learned my addr 172.21.15.97:0/4250255553 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaf400640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 0x7ffab00ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117604987 0 0) 0x7ffab0159d80 con 0x7ffab00adea0 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa90003520 con 0x7ffab00adea0 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 220069889 0 0) 0x7ffa90003520 con 0x7ffab00adea0 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 msgr2=0x7ffab00ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 0x7ffab00ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 >> v1:172.21.15.188:6789/0 conn(0x7ffab00ccd00 legacy=0x7ffab015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00cc880 con 0x7ffab00adea0 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ffa98001670 con 0x7ffab00adea0 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 >> v1:172.21.15.97:6789/0 conn(0x7ffab00adea0 legacy=0x7ffab00b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 0x7ffa900039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90003f20 0x7ffa900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa90004d00 0x7ffa90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab015e210 con 0x7ffa90004d00 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9000dd20 con 0x7ffa90003f20 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.317+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9000dee0 con 0x7ffab01599a0 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaafe00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa90004d00 0x7ffa90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaaf400640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 0x7ffa900039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaa7e00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90003f20 0x7ffa900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaafe00640 1 -- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 msgr2=0x7ffa900039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaafe00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab01599a0 0x7ffa900039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaafe00640 1 -- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90003f20 msgr2=0x7ffa900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaafe00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90003f20 0x7ffa900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaafe00640 1 -- 172.21.15.97:0/4250255553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa90003520 con 0x7ffa90004d00 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.318+0000 7ffaa7e00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90003f20 0x7ffa900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.319+0000 7ffaafe00640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa90004d00 0x7ffa90005180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8001e50 tx=0x7ffa900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864c1c8344999c2a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:40.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.319+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8003b70 con 0x7ffa90004d00 2024-01-31T05:03:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.319+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8003cd0 con 0x7ffa90004d00 2024-01-31T05:03:40.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.319+0000 7ffaaea00640 1 -- 172.21.15.97:0/4250255553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8005310 con 0x7ffa90004d00 2024-01-31T05:03:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffab6400640 1 -- 172.21.15.97:0/4250255553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa90004d00 msgr2=0x7ffa90005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffab6400640 1 --2- 172.21.15.97:0/4250255553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa90004d00 0x7ffa90005180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8001e50 tx=0x7ffa900057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffaafe00640 1 -- 172.21.15.97:0/4250255553 reap_dead start 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffab6400640 1 -- 172.21.15.97:0/4250255553 shutdown_connections 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffab6400640 1 -- 172.21.15.97:0/4250255553 >> 172.21.15.97:0/4250255553 conn(0x7ffab00b0a10 msgr2=0x7ffab00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffab6400640 1 -- 172.21.15.97:0/4250255553 shutdown_connections 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.320+0000 7ffab6400640 1 -- 172.21.15.97:0/4250255553 wait complete. 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.322+0000 7ffab6400640 1 Processor -- start 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.322+0000 7ffab6400640 1 -- start start 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffab6400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90004d00 0x7ffab01f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffab6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab00ca1c0 0x7ffab00ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffab6400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 0x7ffab0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffab6400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00705c0 con 0x7ffab0071660 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffab6400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00708a0 con 0x7ffa90004d00 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffab6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0070b80 con 0x7ffab00ca1c0 2024-01-31T05:03:40.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 0x7ffab0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 0x7ffab0154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42238/0 (socket says 172.21.15.97:42238) 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 -- 172.21.15.97:0/293072747 learned_addr learned my addr 172.21.15.97:0/293072747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaa7e00640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab00ca1c0 0x7ffab00ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 -- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab00ca1c0 msgr2=0x7ffab00ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab00ca1c0 0x7ffab00ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 -- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90004d00 msgr2=0x7ffab01f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90004d00 0x7ffab01f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaafe00640 1 -- 172.21.15.97:0/293072747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0070e60 con 0x7ffab0071660 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.323+0000 7ffaa7e00640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab00ca1c0 0x7ffab00ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.324+0000 7ffaafe00640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 0x7ffab0154ab0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffaa802c060 tx=0x7ffaa80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.324+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa803e070 con 0x7ffab0071660 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.324+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8003e40 con 0x7ffab0071660 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.324+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa802f660 con 0x7ffab0071660 2024-01-31T05:03:40.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.324+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab0071140 con 0x7ffab0071660 2024-01-31T05:03:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.324+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab0154ff0 con 0x7ffab0071660 2024-01-31T05:03:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.325+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab0073080 con 0x7ffab0071660 2024-01-31T05:03:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.326+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ffaa802f7c0 con 0x7ffab0071660 2024-01-31T05:03:40.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.327+0000 7ffaa7400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa8806aed0 0x7ffa8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:40.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.327+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffaa80aaa10 con 0x7ffab0071660 2024-01-31T05:03:40.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.327+0000 7ffaaf400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa8806aed0 0x7ffa8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:40.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.328+0000 7ffaaf400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa8806aed0 0x7ffa8806d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000aa00 tx=0x7ffaa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:40.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.331+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ffaa8075790 con 0x7ffab0071660 2024-01-31T05:03:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:40.565+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffab0071330 con 0x7ffab0071660 2024-01-31T05:03:41.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.255+0000 7ffaa7400640 1 -- 172.21.15.97:0/293072747 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffaa803b020 con 0x7ffab0071660 2024-01-31T05:03:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa8806aed0 msgr2=0x7ffa8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa8806aed0 0x7ffa8806d390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000aa00 tx=0x7ffaa0008040 comp rx=0 tx=0).stop 2024-01-31T05:03:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 msgr2=0x7ffab0154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:41.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 0x7ffab0154ab0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffaa802c060 tx=0x7ffaa80014a0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 shutdown_connections 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ffa8806aed0 0x7ffa8806d390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab0071660 0x7ffab0154ab0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab00ca1c0 0x7ffab00ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 --2- 172.21.15.97:0/293072747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa90004d00 0x7ffab01f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 >> 172.21.15.97:0/293072747 conn(0x7ffab00b0a10 msgr2=0x7ffab00cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 shutdown_connections 2024-01-31T05:03:41.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.259+0000 7ffab6400640 1 -- 172.21.15.97:0/293072747 wait complete. 2024-01-31T05:03:41.279 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-31T05:03:41.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-31T05:03:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.495+0000 7f5ed7e00640 1 Processor -- start 2024-01-31T05:03:41.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.496+0000 7f5ed7e00640 1 -- start start 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.497+0000 7f5ed7e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ed00cfae0 0x7f5ed00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.497+0000 7f5ed7e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed015ec50 con 0x7f5ed015d590 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.497+0000 7f5ed7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00d0460 con 0x7f5ed00d0de0 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.497+0000 7f5ed7e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed00cded0 con 0x7f5ed00cfae0 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.498+0000 7f5ed4a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ed00cfae0 0x7f5ed00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.498+0000 7f5ed5400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5ed015d590 0x7f5ed015e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55352/0 (socket says 172.21.15.97:55352) 2024-01-31T05:03:41.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.498+0000 7f5ed5400640 1 -- 172.21.15.97:0/4139303196 learned_addr learned my addr 172.21.15.97:0/4139303196 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.498+0000 7f5ed4a00640 1 -- 172.21.15.97:0/4139303196 >> v1:172.21.15.188:6789/0 conn(0x7f5ed00d0de0 legacy=0x7f5ed00d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.498+0000 7f5ed4a00640 1 -- 172.21.15.97:0/4139303196 >> v1:172.21.15.97:6789/0 conn(0x7f5ed015d590 legacy=0x7f5ed015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:41.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.498+0000 7f5ed4a00640 1 -- 172.21.15.97:0/4139303196 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed015d970 con 0x7f5ed00cfae0 2024-01-31T05:03:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.500+0000 7f5ed4a00640 1 --2- 172.21.15.97:0/4139303196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed00cff20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009d90 tx=0x7f5ec002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6705fdc9d89b2aaa server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.500+0000 7f5ecfe00640 1 -- 172.21.15.97:0/4139303196 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec002cd00 con 0x7f5ed00cfae0 2024-01-31T05:03:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.500+0000 7f5ecfe00640 1 -- 172.21.15.97:0/4139303196 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec002ce60 con 0x7f5ed00cfae0 2024-01-31T05:03:41.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.500+0000 7f5ecfe00640 1 -- 172.21.15.97:0/4139303196 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0034660 con 0x7f5ed00cfae0 2024-01-31T05:03:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.500+0000 7f5ed7e00640 1 -- 172.21.15.97:0/4139303196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 msgr2=0x7f5ed00cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.501+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/4139303196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed00cff20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009d90 tx=0x7f5ec002bf50 comp rx=0 tx=0).stop 2024-01-31T05:03:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.501+0000 7f5ed7e00640 1 -- 172.21.15.97:0/4139303196 shutdown_connections 2024-01-31T05:03:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.501+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/4139303196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed00cff20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.501+0000 7f5ed7e00640 1 -- 172.21.15.97:0/4139303196 >> 172.21.15.97:0/4139303196 conn(0x7f5ed009c560 msgr2=0x7f5ed00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:41.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.501+0000 7f5ed7e00640 1 -- 172.21.15.97:0/4139303196 shutdown_connections 2024-01-31T05:03:41.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.501+0000 7f5ed7e00640 1 -- 172.21.15.97:0/4139303196 wait complete. 2024-01-31T05:03:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.503+0000 7f5ed7e00640 1 Processor -- start 2024-01-31T05:03:41.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.503+0000 7f5ed7e00640 1 -- start start 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.504+0000 7f5ed7e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed009ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.504+0000 7f5ed7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed00d0de0 0x7f5ed0152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.504+0000 7f5ed7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed015d590 0x7f5ed00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.504+0000 7f5ed7e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00cbcb0 con 0x7f5ed00cfae0 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.504+0000 7f5ed7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0152b70 con 0x7f5ed00d0de0 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.504+0000 7f5ed7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed00cd530 con 0x7f5ed015d590 2024-01-31T05:03:41.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed009ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed009ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42268/0 (socket says 172.21.15.97:42268) 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 -- 172.21.15.97:0/3886290008 learned_addr learned my addr 172.21.15.97:0/3886290008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 -- 172.21.15.97:0/3886290008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed015d590 msgr2=0x7f5ed00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed015d590 0x7f5ed00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 -- 172.21.15.97:0/3886290008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed00d0de0 msgr2=0x7f5ed0152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed00d0de0 0x7f5ed0152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 -- 172.21.15.97:0/3886290008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009a70 con 0x7f5ed00cfae0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.505+0000 7f5ed5400640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed009ced0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c3c0 tx=0x7f5eb800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.506+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8005ce0 con 0x7f5ed00cfae0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.506+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8005e40 con 0x7f5ed00cfae0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.506+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80072d0 con 0x7f5ed00cfae0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.506+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed00cb090 con 0x7f5ed00cfae0 2024-01-31T05:03:41.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.506+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed00cb5f0 con 0x7f5ed00cfae0 2024-01-31T05:03:41.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.507+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed0067830 con 0x7f5ed00cfae0 2024-01-31T05:03:41.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.508+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5eb8007430 con 0x7f5ed00cfae0 2024-01-31T05:03:41.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.509+0000 7f5ecd600640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ea806af50 0x7f5ea806d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:41.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.509+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5eb8088cc0 con 0x7f5ed00cfae0 2024-01-31T05:03:41.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.510+0000 7f5ed4a00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ea806af50 0x7f5ea806d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:41.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.511+0000 7f5ed4a00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ea806af50 0x7f5ea806d410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0036000 tx=0x7f5ec002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:41.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.514+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5eb808f080 con 0x7f5ed00cfae0 2024-01-31T05:03:41.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:41.745+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5ed006c400 con 0x7f5ed00cfae0 2024-01-31T05:03:42.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.259+0000 7f5ecd600640 1 -- 172.21.15.97:0/3886290008 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5eb8017020 con 0x7f5ed00cfae0 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.262+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ea806af50 msgr2=0x7f5ea806d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.262+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ea806af50 0x7f5ea806d410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0036000 tx=0x7f5ec002e000 comp rx=0 tx=0).stop 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.262+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 msgr2=0x7f5ed009ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.262+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed009ced0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5eb800c3c0 tx=0x7f5eb800c880 comp rx=0 tx=0).stop 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 shutdown_connections 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ed015d590 0x7f5ed00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ed00d0de0 0x7f5ed0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5ea806af50 0x7f5ea806d410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 --2- 172.21.15.97:0/3886290008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ed00cfae0 0x7f5ed009ced0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 >> 172.21.15.97:0/3886290008 conn(0x7f5ed009c560 msgr2=0x7f5ed00b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 shutdown_connections 2024-01-31T05:03:42.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.263+0000 7f5ed7e00640 1 -- 172.21.15.97:0/3886290008 wait complete. 2024-01-31T05:03:42.285 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-31T05:03:42.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:42.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.485+0000 7f23a6000640 1 Processor -- start 2024-01-31T05:03:42.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.485+0000 7f23a6000640 1 -- start start 2024-01-31T05:03:42.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.486+0000 7f23a6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 0x7f23a015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f23a6000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a006c8b0 con 0x7f23a006d090 2024-01-31T05:03:42.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f23a6000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a015b120 con 0x7f23a00b1d10 2024-01-31T05:03:42.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f23a6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a015b400 con 0x7f23a015a600 2024-01-31T05:03:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f239fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f23a006d090 0x7f23a006d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55368/0 (socket says 172.21.15.97:55368) 2024-01-31T05:03:42.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f239fe00640 1 -- 172.21.15.97:0/2332178297 learned_addr learned my addr 172.21.15.97:0/2332178297 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f2396800640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 0x7f23a015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.487+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583802490 0 0) 0x7f23a006c8b0 con 0x7f23a006d090 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003520 con 0x7f23a006d090 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3558705764 0 0) 0x7f2380003520 con 0x7f23a006d090 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 msgr2=0x7f23a015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 0x7f23a015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 >> v1:172.21.15.188:6789/0 conn(0x7f23a00b1d10 legacy=0x7f23a00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a006cb70 con 0x7f23a006d090 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f23980019f0 con 0x7f23a006d090 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 >> v1:172.21.15.97:6789/0 conn(0x7f23a006d090 legacy=0x7f23a006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 0x7f23800039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380003f70 0x7f23800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2380004d00 0x7f2380005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a015b400 con 0x7f2380004d00 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238000bac0 con 0x7f2380003f70 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.488+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238000de90 con 0x7f23a015a600 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239fe00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2380004d00 0x7f2380005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239f400640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 0x7f23800039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239fe00640 1 -- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 msgr2=0x7f23800039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239fe00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a015a600 0x7f23800039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f2396800640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380003f70 0x7f23800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239fe00640 1 -- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380003f70 msgr2=0x7f23800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239fe00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380003f70 0x7f23800047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.489+0000 7f239fe00640 1 -- 172.21.15.97:0/2332178297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2380003520 con 0x7f2380004d00 2024-01-31T05:03:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.490+0000 7f239fe00640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2380004d00 0x7f2380005180 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f239800a760 tx=0x7f23800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ca4a0c925f7ab1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.490+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398034630 con 0x7f2380004d00 2024-01-31T05:03:42.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.490+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239802f070 con 0x7f2380004d00 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.490+0000 7f239ea00640 1 -- 172.21.15.97:0/2332178297 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23980349f0 con 0x7f2380004d00 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f23a6000640 1 -- 172.21.15.97:0/2332178297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2380004d00 msgr2=0x7f2380005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f23a6000640 1 --2- 172.21.15.97:0/2332178297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2380004d00 0x7f2380005180 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f239800a760 tx=0x7f23800057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f239fe00640 1 -- 172.21.15.97:0/2332178297 reap_dead start 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f23a6000640 1 -- 172.21.15.97:0/2332178297 shutdown_connections 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f23a6000640 1 -- 172.21.15.97:0/2332178297 >> 172.21.15.97:0/2332178297 conn(0x7f23a00b0a10 msgr2=0x7f23a00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f23a6000640 1 -- 172.21.15.97:0/2332178297 shutdown_connections 2024-01-31T05:03:42.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.491+0000 7f23a6000640 1 -- 172.21.15.97:0/2332178297 wait complete. 2024-01-31T05:03:42.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.493+0000 7f23a6000640 1 Processor -- start 2024-01-31T05:03:42.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.493+0000 7f23a6000640 1 -- start start 2024-01-31T05:03:42.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f23a6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380004d00 0x7f23a0146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f23a6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a0153b10 0x7f23a015b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f23a6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 0x7f23a0155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f23a6000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0155f40 con 0x7f23a0155550 2024-01-31T05:03:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f23a6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00704b0 con 0x7f2380004d00 2024-01-31T05:03:42.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f23a6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0146f60 con 0x7f23a0153b10 2024-01-31T05:03:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.494+0000 7f239fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 0x7f23a0155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 0x7f23a0155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42282/0 (socket says 172.21.15.97:42282) 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 -- 172.21.15.97:0/2693229967 learned_addr learned my addr 172.21.15.97:0/2693229967 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239f400640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380004d00 0x7f23a0146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f2396800640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a0153b10 0x7f23a015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 -- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a0153b10 msgr2=0x7f23a015b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a0153b10 0x7f23a015b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 -- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380004d00 msgr2=0x7f23a0146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380004d00 0x7f23a0146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 -- 172.21.15.97:0/2693229967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0155020 con 0x7f23a0155550 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.495+0000 7f239fe00640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 0x7f23a0155a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f239800a760 tx=0x7f2398001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.496+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398034630 con 0x7f23a0155550 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.496+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398034790 con 0x7f23a0155550 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.496+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a0153fa0 con 0x7f23a0155550 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.496+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398034b00 con 0x7f23a0155550 2024-01-31T05:03:42.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.496+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0154560 con 0x7f23a0155550 2024-01-31T05:03:42.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.497+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0070620 con 0x7f23a0155550 2024-01-31T05:03:42.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.499+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2398031030 con 0x7f23a0155550 2024-01-31T05:03:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.500+0000 7f2397e00640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f237806aed0 0x7f237806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.500+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f23980aba20 con 0x7f23a0155550 2024-01-31T05:03:42.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.500+0000 7f239f400640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f237806aed0 0x7f237806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.501+0000 7f239f400640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f237806aed0 0x7f237806d390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2390007fd0 tx=0x7f2390009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:42.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.505+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f23a0070620 con 0x7f23a0155550 2024-01-31T05:03:42.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.737+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a0070620 con 0x7f23a0155550 2024-01-31T05:03:42.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.738+0000 7f2397e00640 1 -- 172.21.15.97:0/2693229967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f23a0070620 con 0x7f23a0155550 2024-01-31T05:03:42.738 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:42.738 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:03:38.646144+0000","modified":"2024-01-31T05:03:41.747278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3986947126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3986947126},{"type":"v1","addr":"172.21.15.97:6837","nonce":3986947126}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-31T05:03:42.738 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-01-31T05:03:42.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.740+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f237806aed0 msgr2=0x7f237806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.740+0000 7f23a6000640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f237806aed0 0x7f237806d390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2390007fd0 tx=0x7f2390009040 comp rx=0 tx=0).stop 2024-01-31T05:03:42.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.740+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 msgr2=0x7f23a0155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.740+0000 7f23a6000640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 0x7f23a0155a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f239800a760 tx=0x7f2398001cb0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 shutdown_connections 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f237806aed0 0x7f237806d390 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23a0155550 0x7f23a0155a00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23a0153b10 0x7f23a015b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 --2- 172.21.15.97:0/2693229967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2380004d00 0x7f23a0146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 >> 172.21.15.97:0/2693229967 conn(0x7f23a00b0a10 msgr2=0x7f23a00cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 shutdown_connections 2024-01-31T05:03:42.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.741+0000 7f23a6000640 1 -- 172.21.15.97:0/2693229967 wait complete. 2024-01-31T05:03:42.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.948+0000 7f08f5a00640 1 Processor -- start 2024-01-31T05:03:42.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.948+0000 7f08f5a00640 1 -- start start 2024-01-31T05:03:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08f5a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f08f00cb7d0 0x7f08f00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08f5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 0x7f08f00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08f5a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f006e0d0 con 0x7f08f006dc40 2024-01-31T05:03:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08f5a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f006cba0 con 0x7f08f00cb7d0 2024-01-31T05:03:42.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08f5a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f006ce80 con 0x7f08f006c350 2024-01-31T05:03:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08ee000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f08f006dc40 0x7f08f00cbbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55370/0 (socket says 172.21.15.97:55370) 2024-01-31T05:03:42.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08ee000640 1 -- 172.21.15.97:0/2918816443 learned_addr learned my addr 172.21.15.97:0/2918816443 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:42.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08ef400640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 0x7f08f00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08eea00640 1 --2- 172.21.15.97:0/2918816443 >> v2:172.21.15.188:3300/0 conn(0x7f08f00cb7d0 0x7f08f00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292922597 0 0) 0x7f08f006e0d0 con 0x7f08f006dc40 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.950+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d0003520 con 0x7f08f006dc40 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2807667457 0 0) 0x7f08d0003520 con 0x7f08f006dc40 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 msgr2=0x7f08f00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 0x7f08f00ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 >> v2:172.21.15.188:3300/0 conn(0x7f08f00cb7d0 msgr2=0x7f08f00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 --2- 172.21.15.97:0/2918816443 >> v2:172.21.15.188:3300/0 conn(0x7f08f00cb7d0 0x7f08f00b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f006d160 con 0x7f08f006dc40 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f08d8001670 con 0x7f08f006dc40 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ef400640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 0x7f08f00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08eea00640 1 --2- 172.21.15.97:0/2918816443 >> v2:172.21.15.188:3300/0 conn(0x7f08f00cb7d0 0x7f08f00b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:42.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 >> v1:172.21.15.97:6789/0 conn(0x7f08f006dc40 legacy=0x7f08f00cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d0003970 0x7f08d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d00047c0 0x7f08d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 0x7f08d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f006cba0 con 0x7f08d0003970 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f006ce80 con 0x7f08d00047c0 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d000ba80 con 0x7f08f006c350 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.951+0000 7f08eea00640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d0003970 0x7f08d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.952+0000 7f08ee000640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d00047c0 0x7f08d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.952+0000 7f08eea00640 1 -- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 msgr2=0x7f08d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.952+0000 7f08eea00640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f006c350 0x7f08d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.952+0000 7f08eea00640 1 -- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d00047c0 msgr2=0x7f08d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.952+0000 7f08eea00640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d00047c0 0x7f08d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.952+0000 7f08eea00640 1 -- 172.21.15.97:0/2918816443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0003520 con 0x7f08d0003970 2024-01-31T05:03:42.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.953+0000 7f08eea00640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d0003970 0x7f08d0003d90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f08e00098a0 tx=0x7f08d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7578fd0cde788d80 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.953+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004230 con 0x7f08d0003970 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.953+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0004390 con 0x7f08d0003970 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.953+0000 7f08ed600640 1 -- 172.21.15.97:0/2918816443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003830 con 0x7f08d0003970 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08f5a00640 1 -- 172.21.15.97:0/2918816443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d0003970 msgr2=0x7f08d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08f5a00640 1 --2- 172.21.15.97:0/2918816443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d0003970 0x7f08d0003d90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f08e00098a0 tx=0x7f08d00057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08ef400640 1 -- 172.21.15.97:0/2918816443 reap_dead start 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08f5a00640 1 -- 172.21.15.97:0/2918816443 shutdown_connections 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08f5a00640 1 -- 172.21.15.97:0/2918816443 >> 172.21.15.97:0/2918816443 conn(0x7f08f00b0a10 msgr2=0x7f08f00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08f5a00640 1 -- 172.21.15.97:0/2918816443 shutdown_connections 2024-01-31T05:03:42.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.954+0000 7f08f5a00640 1 -- 172.21.15.97:0/2918816443 wait complete. 2024-01-31T05:03:42.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.956+0000 7f08f5a00640 1 Processor -- start 2024-01-31T05:03:42.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.956+0000 7f08f5a00640 1 -- start start 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08f5a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d0003970 0x7f08f01f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08f5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 0x7f08f01604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08f5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f0161b50 0x7f08f0161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08f5a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f0160a30 con 0x7f08f0160070 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08f5a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f009a260 con 0x7f08d0003970 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08f5a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f0162880 con 0x7f08f0161b50 2024-01-31T05:03:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08ee000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 0x7f08f01604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08ee000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 0x7f08f01604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42298/0 (socket says 172.21.15.97:42298) 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.957+0000 7f08ee000640 1 -- 172.21.15.97:0/612296167 learned_addr learned my addr 172.21.15.97:0/612296167 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08eea00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d0003970 0x7f08f01f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ee000640 1 -- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f0161b50 msgr2=0x7f08f0161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ef400640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f0161b50 0x7f08f0161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ee000640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f0161b50 0x7f08f0161fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ee000640 1 -- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d0003970 msgr2=0x7f08f01f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ee000640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d0003970 0x7f08f01f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ee000640 1 -- 172.21.15.97:0/612296167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f0162b60 con 0x7f08f0160070 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08eea00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d0003970 0x7f08f01f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08ee000640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 0x7f08f01604f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08d800e760 tx=0x7f08d800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d800ee70 con 0x7f08f0160070 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d800c8d0 con 0x7f08f0160070 2024-01-31T05:03:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d800cc80 con 0x7f08f0160070 2024-01-31T05:03:42.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f0099c10 con 0x7f08f0160070 2024-01-31T05:03:42.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.958+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f0162510 con 0x7f08f0160070 2024-01-31T05:03:42.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.959+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f0099a70 con 0x7f08f0160070 2024-01-31T05:03:42.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.961+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f08d801d460 con 0x7f08f0160070 2024-01-31T05:03:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.961+0000 7f08e6a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f08c806aed0 0x7f08c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.962+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f08d80897d0 con 0x7f08f0160070 2024-01-31T05:03:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.962+0000 7f08eea00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f08c806aed0 0x7f08c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.963+0000 7f08eea00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f08c806aed0 0x7f08c806d390 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08e0035000 tx=0x7f08e003a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:42.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:42.966+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f08f0099a70 con 0x7f08f0160070 2024-01-31T05:03:43.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:03:43.163 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:03:43.163 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.161+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08f0099a70 con 0x7f08f0160070 2024-01-31T05:03:43.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.163+0000 7f08e6a00640 1 -- 172.21.15.97:0/612296167 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f08f0099a70 con 0x7f08f0160070 2024-01-31T05:03:43.164 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f08c806aed0 msgr2=0x7f08c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f08c806aed0 0x7f08c806d390 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08e0035000 tx=0x7f08e003a040 comp rx=0 tx=0).stop 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 msgr2=0x7f08f01604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 0x7f08f01604f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08d800e760 tx=0x7f08d800ec20 comp rx=0 tx=0).stop 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 shutdown_connections 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08f0161b50 0x7f08f0161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f08c806aed0 0x7f08c806d390 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0160070 0x7f08f01604f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 --2- 172.21.15.97:0/612296167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08d0003970 0x7f08f01f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.165+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 >> 172.21.15.97:0/612296167 conn(0x7f08f00b0a10 msgr2=0x7f08f0073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.166+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 shutdown_connections 2024-01-31T05:03:43.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.166+0000 7f08f5a00640 1 -- 172.21.15.97:0/612296167 wait complete. 2024-01-31T05:03:43.182 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-31T05:03:43.183 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:03:43.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.417+0000 7fb9fbe00640 1 Processor -- start 2024-01-31T05:03:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.417+0000 7fb9fbe00640 1 -- start start 2024-01-31T05:03:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.418+0000 7fb9fbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 0x7fb9fc0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.418+0000 7fb9fbe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb9fc0d0d60 0x7fb9fc0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9fbe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9fc15d8c0 con 0x7fb9fc0d1560 2024-01-31T05:03:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9fbe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9fc0b0b20 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9fbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0b22f0 con 0x7fb9fc15d4e0 2024-01-31T05:03:43.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9faa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 0x7fb9fc0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9faa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 0x7fb9fc0b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:43640/0 (socket says 172.21.15.97:43640) 2024-01-31T05:03:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9faa00640 1 -- 172.21.15.97:0/2950876804 learned_addr learned my addr 172.21.15.97:0/2950876804 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.419+0000 7fb9fa000640 1 --2- 172.21.15.97:0/2950876804 >> v2:172.21.15.188:3300/0 conn(0x7fb9fc0d0d60 0x7fb9fc0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.420+0000 7fb9faa00640 1 -- 172.21.15.97:0/2950876804 >> v2:172.21.15.188:3300/0 conn(0x7fb9fc0d0d60 msgr2=0x7fb9fc0b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.420+0000 7fb9faa00640 1 --2- 172.21.15.97:0/2950876804 >> v2:172.21.15.188:3300/0 conn(0x7fb9fc0d0d60 0x7fb9fc0b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.420+0000 7fb9faa00640 1 -- 172.21.15.97:0/2950876804 >> v1:172.21.15.188:6789/0 conn(0x7fb9fc0d1560 legacy=0x7fb9fc0b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.420+0000 7fb9faa00640 1 -- 172.21.15.97:0/2950876804 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0b1920 con 0x7fb9fc15d4e0 2024-01-31T05:03:43.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.421+0000 7fb9faa00640 1 --2- 172.21.15.97:0/2950876804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 0x7fb9fc0b05e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec02ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5059b35722912c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:43.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.422+0000 7fb9f9600640 1 -- 172.21.15.97:0/2950876804 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec02f840 con 0x7fb9fc15d4e0 2024-01-31T05:03:43.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.422+0000 7fb9f9600640 1 -- 172.21.15.97:0/2950876804 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec02f9a0 con 0x7fb9fc15d4e0 2024-01-31T05:03:43.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.422+0000 7fb9f9600640 1 -- 172.21.15.97:0/2950876804 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec02fd30 con 0x7fb9fc15d4e0 2024-01-31T05:03:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.422+0000 7fb9fbe00640 1 -- 172.21.15.97:0/2950876804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 msgr2=0x7fb9fc0b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.422+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/2950876804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 0x7fb9fc0b05e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009870 tx=0x7fb9ec02ba20 comp rx=0 tx=0).stop 2024-01-31T05:03:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.423+0000 7fb9fbe00640 1 -- 172.21.15.97:0/2950876804 shutdown_connections 2024-01-31T05:03:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.423+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/2950876804 >> v2:172.21.15.188:3300/0 conn(0x7fb9fc0d0d60 0x7fb9fc0b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.423+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/2950876804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc15d4e0 0x7fb9fc0b05e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.423+0000 7fb9fbe00640 1 -- 172.21.15.97:0/2950876804 >> 172.21.15.97:0/2950876804 conn(0x7fb9fc0d3010 msgr2=0x7fb9fc0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.423+0000 7fb9fbe00640 1 -- 172.21.15.97:0/2950876804 shutdown_connections 2024-01-31T05:03:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.423+0000 7fb9fbe00640 1 -- 172.21.15.97:0/2950876804 wait complete. 2024-01-31T05:03:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.424+0000 7fb9fbe00640 1 Processor -- start 2024-01-31T05:03:43.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.424+0000 7fb9fbe00640 1 -- start start 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9fbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 0x7fb9fc15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9fbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc0d1560 0x7fb9fc0b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9fbe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9fc15d4e0 0x7fb9fc15da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9fbe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc15f5a0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9fbe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0cb7e0 con 0x7fb9fc15d4e0 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9fbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc0cbac0 con 0x7fb9fc0d1560 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.425+0000 7fb9faa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 0x7fb9fc15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 0x7fb9fc15f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42306/0 (socket says 172.21.15.97:42306) 2024-01-31T05:03:43.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 -- 172.21.15.97:0/1954618329 learned_addr learned my addr 172.21.15.97:0/1954618329 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9fa000640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc0d1560 0x7fb9fc0b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 -- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc0d1560 msgr2=0x7fb9fc0b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc0d1560 0x7fb9fc0b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 -- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9fc15d4e0 msgr2=0x7fb9fc15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9fc15d4e0 0x7fb9fc15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 -- 172.21.15.97:0/1954618329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0094d0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9faa00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 0x7fb9fc15f060 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec02ba60 tx=0x7fb9ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.426+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec03c040 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.427+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec037be0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.427+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0423f0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.427+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc0cbda0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.427+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc0cc360 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.428+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc0676b0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.429+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb9ec02d3d0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.429+0000 7fb9f2a00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9d406aed0 0x7fb9d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.430+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb9ec0ab890 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.430+0000 7fb9fa000640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9d406aed0 0x7fb9d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.430+0000 7fb9fa000640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9d406aed0 0x7fb9d406d390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c69e0 tx=0x7fb9e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:43.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.434+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb9ec0766a0 con 0x7fb9fc0d0d60 2024-01-31T05:03:43.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.619+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9fc0b0ff0 con 0x7fb9d406aed0 2024-01-31T05:03:43.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.628+0000 7fb9f2a00640 1 -- 172.21.15.97:0/1954618329 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284616 (secure 0 0 0) 0x7fb9e4010840 con 0x7fb9d406aed0 2024-01-31T05:03:43.632 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:43.633 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9d406aed0 msgr2=0x7fb9d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9d406aed0 0x7fb9d406d390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c69e0 tx=0x7fb9e4008040 comp rx=0 tx=0).stop 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 msgr2=0x7fb9fc15f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 0x7fb9fc15f060 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec02ba60 tx=0x7fb9ec004730 comp rx=0 tx=0).stop 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 shutdown_connections 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9fc15d4e0 0x7fb9fc15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9fc0d1560 0x7fb9fc0b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb9d406aed0 0x7fb9d406d390 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 --2- 172.21.15.97:0/1954618329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9fc0d0d60 0x7fb9fc15f060 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 >> 172.21.15.97:0/1954618329 conn(0x7fb9fc0d3010 msgr2=0x7fb9fc0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 shutdown_connections 2024-01-31T05:03:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.633+0000 7fb9fbe00640 1 -- 172.21.15.97:0/1954618329 wait complete. 2024-01-31T05:03:43.650 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-01-31T05:03:43.610324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":188,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222300,"kb_used_data":6116,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752420,"statfs":{"total":773094113280,"available":772866478080,"internally_reserved":0,"allocated":6262784,"data_stored":3400346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001696"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231612+0000","last_change":"2024-01-31T05:03:38.231612+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231612+0000","last_undegraded":"2024-01-31T05:03:38.231612+0000","last_fullsized":"2024-01-31T05:03:38.231612+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.254203+0000","last_change":"2024-01-31T05:03:39.252312+0000","last_active":"2024-01-31T05:03:39.254203+0000","last_peered":"2024-01-31T05:03:39.254203+0000","last_clean":"2024-01-31T05:03:39.254203+0000","last_became_active":"2024-01-31T05:03:39.252042+0000","last_became_peered":"2024-01-31T05:03:39.252042+0000","last_unstale":"2024-01-31T05:03:39.254203+0000","last_undegraded":"2024-01-31T05:03:39.254203+0000","last_fullsized":"2024-01-31T05:03:39.254203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259236+0000","last_change":"2024-01-31T05:03:39.243310+0000","last_active":"2024-01-31T05:03:39.259236+0000","last_peered":"2024-01-31T05:03:39.259236+0000","last_clean":"2024-01-31T05:03:39.259236+0000","last_became_active":"2024-01-31T05:03:39.242309+0000","last_became_peered":"2024-01-31T05:03:39.242309+0000","last_unstale":"2024-01-31T05:03:39.259236+0000","last_undegraded":"2024-01-31T05:03:39.259236+0000","last_fullsized":"2024-01-31T05:03:39.259236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.261556+0000","last_change":"2024-01-31T05:03:39.254633+0000","last_active":"2024-01-31T05:03:39.261556+0000","last_peered":"2024-01-31T05:03:39.261556+0000","last_clean":"2024-01-31T05:03:39.261556+0000","last_became_active":"2024-01-31T05:03:39.254325+0000","last_became_peered":"2024-01-31T05:03:39.254325+0000","last_unstale":"2024-01-31T05:03:39.261556+0000","last_undegraded":"2024-01-31T05:03:39.261556+0000","last_fullsized":"2024-01-31T05:03:39.261556+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:03:40.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222392+0000","last_change":"2024-01-31T05:03:38.222392+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222392+0000","last_undegraded":"2024-01-31T05:03:38.222392+0000","last_fullsized":"2024-01-31T05:03:38.222392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226676+0000","last_change":"2024-01-31T05:03:38.226676+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226676+0000","last_undegraded":"2024-01-31T05:03:38.226676+0000","last_fullsized":"2024-01-31T05:03:38.226676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260398+0000","last_change":"2024-01-31T05:03:39.247885+0000","last_active":"2024-01-31T05:03:39.260398+0000","last_peered":"2024-01-31T05:03:39.260398+0000","last_clean":"2024-01-31T05:03:39.260398+0000","last_became_active":"2024-01-31T05:03:39.247556+0000","last_became_peered":"2024-01-31T05:03:39.247556+0000","last_unstale":"2024-01-31T05:03:39.260398+0000","last_undegraded":"2024-01-31T05:03:39.260398+0000","last_fullsized":"2024-01-31T05:03:39.260398+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:59.282060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.215147+0000","last_change":"2024-01-31T05:03:38.215147+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.215147+0000","last_undegraded":"2024-01-31T05:03:38.215147+0000","last_fullsized":"2024-01-31T05:03:38.215147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.270024+0000","last_change":"2024-01-31T05:03:39.256441+0000","last_active":"2024-01-31T05:03:39.270024+0000","last_peered":"2024-01-31T05:03:39.270024+0000","last_clean":"2024-01-31T05:03:39.270024+0000","last_became_active":"2024-01-31T05:03:39.256105+0000","last_became_peered":"2024-01-31T05:03:39.256105+0000","last_unstale":"2024-01-31T05:03:39.270024+0000","last_undegraded":"2024-01-31T05:03:39.270024+0000","last_fullsized":"2024-01-31T05:03:39.270024+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226447+0000","last_change":"2024-01-31T05:03:38.226447+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226447+0000","last_undegraded":"2024-01-31T05:03:38.226447+0000","last_fullsized":"2024-01-31T05:03:38.226447+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259313+0000","last_change":"2024-01-31T05:03:39.251020+0000","last_active":"2024-01-31T05:03:39.259313+0000","last_peered":"2024-01-31T05:03:39.259313+0000","last_clean":"2024-01-31T05:03:39.259313+0000","last_became_active":"2024-01-31T05:03:39.250684+0000","last_became_peered":"2024-01-31T05:03:39.250684+0000","last_unstale":"2024-01-31T05:03:39.259313+0000","last_undegraded":"2024-01-31T05:03:39.259313+0000","last_fullsized":"2024-01-31T05:03:39.259313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:39:06.883661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269332+0000","last_change":"2024-01-31T05:03:39.258617+0000","last_active":"2024-01-31T05:03:39.269332+0000","last_peered":"2024-01-31T05:03:39.269332+0000","last_clean":"2024-01-31T05:03:39.269332+0000","last_became_active":"2024-01-31T05:03:39.258317+0000","last_became_peered":"2024-01-31T05:03:39.258317+0000","last_unstale":"2024-01-31T05:03:39.269332+0000","last_undegraded":"2024-01-31T05:03:39.269332+0000","last_fullsized":"2024-01-31T05:03:39.269332+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218606+0000","last_change":"2024-01-31T05:03:38.218606+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218606+0000","last_undegraded":"2024-01-31T05:03:38.218606+0000","last_fullsized":"2024-01-31T05:03:38.218606+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267681+0000","last_change":"2024-01-31T05:03:39.251453+0000","last_active":"2024-01-31T05:03:39.267681+0000","last_peered":"2024-01-31T05:03:39.267681+0000","last_clean":"2024-01-31T05:03:39.267681+0000","last_became_active":"2024-01-31T05:03:39.248059+0000","last_became_peered":"2024-01-31T05:03:39.248059+0000","last_unstale":"2024-01-31T05:03:39.267681+0000","last_undegraded":"2024-01-31T05:03:39.267681+0000","last_fullsized":"2024-01-31T05:03:39.267681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.908110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222879+0000","last_change":"2024-01-31T05:03:38.222879+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222879+0000","last_undegraded":"2024-01-31T05:03:38.222879+0000","last_fullsized":"2024-01-31T05:03:38.222879+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220373+0000","last_change":"2024-01-31T05:03:38.220373+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220373+0000","last_undegraded":"2024-01-31T05:03:38.220373+0000","last_fullsized":"2024-01-31T05:03:38.220373+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231217+0000","last_change":"2024-01-31T05:03:38.231217+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231217+0000","last_undegraded":"2024-01-31T05:03:38.231217+0000","last_fullsized":"2024-01-31T05:03:38.231217+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266404+0000","last_change":"2024-01-31T05:03:39.249229+0000","last_active":"2024-01-31T05:03:39.266404+0000","last_peered":"2024-01-31T05:03:39.266404+0000","last_clean":"2024-01-31T05:03:39.266404+0000","last_became_active":"2024-01-31T05:03:39.248929+0000","last_became_peered":"2024-01-31T05:03:39.248929+0000","last_unstale":"2024-01-31T05:03:39.266404+0000","last_undegraded":"2024-01-31T05:03:39.266404+0000","last_fullsized":"2024-01-31T05:03:39.266404+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259739+0000","last_change":"2024-01-31T05:03:39.255701+0000","last_active":"2024-01-31T05:03:39.259739+0000","last_peered":"2024-01-31T05:03:39.259739+0000","last_clean":"2024-01-31T05:03:39.259739+0000","last_became_active":"2024-01-31T05:03:39.253975+0000","last_became_peered":"2024-01-31T05:03:39.253975+0000","last_unstale":"2024-01-31T05:03:39.259739+0000","last_undegraded":"2024-01-31T05:03:39.259739+0000","last_fullsized":"2024-01-31T05:03:39.259739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.263250+0000","last_change":"2024-01-31T05:03:39.258999+0000","last_active":"2024-01-31T05:03:39.263250+0000","last_peered":"2024-01-31T05:03:39.263250+0000","last_clean":"2024-01-31T05:03:39.263250+0000","last_became_active":"2024-01-31T05:03:39.257976+0000","last_became_peered":"2024-01-31T05:03:39.257976+0000","last_unstale":"2024-01-31T05:03:39.263250+0000","last_undegraded":"2024-01-31T05:03:39.263250+0000","last_fullsized":"2024-01-31T05:03:39.263250+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:24:32.083418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218199+0000","last_change":"2024-01-31T05:03:38.218199+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218199+0000","last_undegraded":"2024-01-31T05:03:38.218199+0000","last_fullsized":"2024-01-31T05:03:38.218199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.250048+0000","last_change":"2024-01-31T05:03:39.242603+0000","last_active":"2024-01-31T05:03:39.250048+0000","last_peered":"2024-01-31T05:03:39.250048+0000","last_clean":"2024-01-31T05:03:39.250048+0000","last_became_active":"2024-01-31T05:03:39.241443+0000","last_became_peered":"2024-01-31T05:03:39.241443+0000","last_unstale":"2024-01-31T05:03:39.250048+0000","last_undegraded":"2024-01-31T05:03:39.250048+0000","last_fullsized":"2024-01-31T05:03:39.250048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:28:05.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260437+0000","last_change":"2024-01-31T05:03:39.253595+0000","last_active":"2024-01-31T05:03:39.260437+0000","last_peered":"2024-01-31T05:03:39.260437+0000","last_clean":"2024-01-31T05:03:39.260437+0000","last_became_active":"2024-01-31T05:03:39.253076+0000","last_became_peered":"2024-01-31T05:03:39.253076+0000","last_unstale":"2024-01-31T05:03:39.260437+0000","last_undegraded":"2024-01-31T05:03:39.260437+0000","last_fullsized":"2024-01-31T05:03:39.260437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:57:48.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272382+0000","last_change":"2024-01-31T05:03:39.260142+0000","last_active":"2024-01-31T05:03:39.272382+0000","last_peered":"2024-01-31T05:03:39.272382+0000","last_clean":"2024-01-31T05:03:39.272382+0000","last_became_active":"2024-01-31T05:03:39.255918+0000","last_became_peered":"2024-01-31T05:03:39.255918+0000","last_unstale":"2024-01-31T05:03:39.272382+0000","last_undegraded":"2024-01-31T05:03:39.272382+0000","last_fullsized":"2024-01-31T05:03:39.272382+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.257215+0000","last_change":"2024-01-31T05:03:39.253163+0000","last_active":"2024-01-31T05:03:39.257215+0000","last_peered":"2024-01-31T05:03:39.257215+0000","last_clean":"2024-01-31T05:03:39.257215+0000","last_became_active":"2024-01-31T05:03:39.252895+0000","last_became_peered":"2024-01-31T05:03:39.252895+0000","last_unstale":"2024-01-31T05:03:39.257215+0000","last_undegraded":"2024-01-31T05:03:39.257215+0000","last_fullsized":"2024-01-31T05:03:39.257215+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218748+0000","last_change":"2024-01-31T05:03:38.218748+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218748+0000","last_undegraded":"2024-01-31T05:03:38.218748+0000","last_fullsized":"2024-01-31T05:03:38.218748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.252464+0000","last_change":"2024-01-31T05:03:39.245654+0000","last_active":"2024-01-31T05:03:39.252464+0000","last_peered":"2024-01-31T05:03:39.252464+0000","last_clean":"2024-01-31T05:03:39.252464+0000","last_became_active":"2024-01-31T05:03:39.242973+0000","last_became_peered":"2024-01-31T05:03:39.242973+0000","last_unstale":"2024-01-31T05:03:39.252464+0000","last_undegraded":"2024-01-31T05:03:39.252464+0000","last_fullsized":"2024-01-31T05:03:39.252464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.227220+0000","last_change":"2024-01-31T05:03:38.227220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.227220+0000","last_undegraded":"2024-01-31T05:03:38.227220+0000","last_fullsized":"2024-01-31T05:03:38.227220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219977+0000","last_change":"2024-01-31T05:03:38.219977+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219977+0000","last_undegraded":"2024-01-31T05:03:38.219977+0000","last_fullsized":"2024-01-31T05:03:38.219977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262925+0000","last_change":"2024-01-31T05:03:39.258792+0000","last_active":"2024-01-31T05:03:39.262925+0000","last_peered":"2024-01-31T05:03:39.262925+0000","last_clean":"2024-01-31T05:03:39.262925+0000","last_became_active":"2024-01-31T05:03:39.258531+0000","last_became_peered":"2024-01-31T05:03:39.258531+0000","last_unstale":"2024-01-31T05:03:39.262925+0000","last_undegraded":"2024-01-31T05:03:39.262925+0000","last_fullsized":"2024-01-31T05:03:39.262925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:02:30.641509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243207+0000","last_change":"2024-01-31T05:03:39.236368+0000","last_active":"2024-01-31T05:03:39.243207+0000","last_peered":"2024-01-31T05:03:39.243207+0000","last_clean":"2024-01-31T05:03:39.243207+0000","last_became_active":"2024-01-31T05:03:39.236055+0000","last_became_peered":"2024-01-31T05:03:39.236055+0000","last_unstale":"2024-01-31T05:03:39.243207+0000","last_undegraded":"2024-01-31T05:03:39.243207+0000","last_fullsized":"2024-01-31T05:03:39.243207+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.901754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.241385+0000","last_change":"2024-01-31T05:03:39.239349+0000","last_active":"2024-01-31T05:03:39.241385+0000","last_peered":"2024-01-31T05:03:39.241385+0000","last_clean":"2024-01-31T05:03:39.241385+0000","last_became_active":"2024-01-31T05:03:39.238868+0000","last_became_peered":"2024-01-31T05:03:39.238868+0000","last_unstale":"2024-01-31T05:03:39.241385+0000","last_undegraded":"2024-01-31T05:03:39.241385+0000","last_fullsized":"2024-01-31T05:03:39.241385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:55:11.110405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.225923+0000","last_change":"2024-01-31T05:03:38.225923+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.225923+0000","last_undegraded":"2024-01-31T05:03:38.225923+0000","last_fullsized":"2024-01-31T05:03:38.225923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.276730+0000","last_change":"2024-01-31T05:03:38.235515+0000","last_active":"2024-01-31T05:03:39.276730+0000","last_peered":"2024-01-31T05:03:39.276730+0000","last_clean":"2024-01-31T05:03:39.276730+0000","last_became_active":"2024-01-31T05:03:38.235228+0000","last_became_peered":"2024-01-31T05:03:38.235228+0000","last_unstale":"2024-01-31T05:03:39.276730+0000","last_undegraded":"2024-01-31T05:03:39.276730+0000","last_fullsized":"2024-01-31T05:03:39.276730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:42.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.230764+0000","last_change":"2024-01-31T05:03:38.230764+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.230764+0000","last_undegraded":"2024-01-31T05:03:38.230764+0000","last_fullsized":"2024-01-31T05:03:38.230764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269887+0000","last_change":"2024-01-31T05:03:38.224888+0000","last_active":"2024-01-31T05:03:39.269887+0000","last_peered":"2024-01-31T05:03:39.269887+0000","last_clean":"2024-01-31T05:03:39.269887+0000","last_became_active":"2024-01-31T05:03:38.223355+0000","last_became_peered":"2024-01-31T05:03:38.223355+0000","last_unstale":"2024-01-31T05:03:39.269887+0000","last_undegraded":"2024-01-31T05:03:39.269887+0000","last_fullsized":"2024-01-31T05:03:39.269887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226684+0000","last_change":"2024-01-31T05:03:38.226684+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226684+0000","last_undegraded":"2024-01-31T05:03:38.226684+0000","last_fullsized":"2024-01-31T05:03:38.226684+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.239333+0000","last_change":"2024-01-31T05:03:38.234777+0000","last_active":"2024-01-31T05:03:38.239333+0000","last_peered":"2024-01-31T05:03:38.239333+0000","last_clean":"2024-01-31T05:03:38.239333+0000","last_became_active":"2024-01-31T05:03:38.234493+0000","last_became_peered":"2024-01-31T05:03:38.234493+0000","last_unstale":"2024-01-31T05:03:38.239333+0000","last_undegraded":"2024-01-31T05:03:38.239333+0000","last_fullsized":"2024-01-31T05:03:38.239333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:48:31.432282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224220+0000","last_change":"2024-01-31T05:03:38.224220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224220+0000","last_undegraded":"2024-01-31T05:03:38.224220+0000","last_fullsized":"2024-01-31T05:03:38.224220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277804+0000","last_change":"2024-01-31T05:03:38.221765+0000","last_active":"2024-01-31T05:03:39.277804+0000","last_peered":"2024-01-31T05:03:39.277804+0000","last_clean":"2024-01-31T05:03:39.277804+0000","last_became_active":"2024-01-31T05:03:38.221534+0000","last_became_peered":"2024-01-31T05:03:38.221534+0000","last_unstale":"2024-01-31T05:03:39.277804+0000","last_undegraded":"2024-01-31T05:03:39.277804+0000","last_fullsized":"2024-01-31T05:03:39.277804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:22.374866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.210993+0000","last_change":"2024-01-31T05:03:38.210993+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.210993+0000","last_undegraded":"2024-01-31T05:03:38.210993+0000","last_fullsized":"2024-01-31T05:03:38.210993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277781+0000","last_change":"2024-01-31T05:03:38.226696+0000","last_active":"2024-01-31T05:03:39.277781+0000","last_peered":"2024-01-31T05:03:39.277781+0000","last_clean":"2024-01-31T05:03:39.277781+0000","last_became_active":"2024-01-31T05:03:38.226440+0000","last_became_peered":"2024-01-31T05:03:38.226440+0000","last_unstale":"2024-01-31T05:03:39.277781+0000","last_undegraded":"2024-01-31T05:03:39.277781+0000","last_fullsized":"2024-01-31T05:03:39.277781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243539+0000","last_change":"2024-01-31T05:03:39.240337+0000","last_active":"2024-01-31T05:03:39.243539+0000","last_peered":"2024-01-31T05:03:39.243539+0000","last_clean":"2024-01-31T05:03:39.243539+0000","last_became_active":"2024-01-31T05:03:39.240029+0000","last_became_peered":"2024-01-31T05:03:39.240029+0000","last_unstale":"2024-01-31T05:03:39.243539+0000","last_undegraded":"2024-01-31T05:03:39.243539+0000","last_fullsized":"2024-01-31T05:03:39.243539+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:24:32.372349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240709+0000","last_change":"2024-01-31T05:03:38.239050+0000","last_active":"2024-01-31T05:03:38.240709+0000","last_peered":"2024-01-31T05:03:38.240709+0000","last_clean":"2024-01-31T05:03:38.240709+0000","last_became_active":"2024-01-31T05:03:38.238746+0000","last_became_peered":"2024-01-31T05:03:38.238746+0000","last_unstale":"2024-01-31T05:03:38.240709+0000","last_undegraded":"2024-01-31T05:03:38.240709+0000","last_fullsized":"2024-01-31T05:03:38.240709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:37:59.345660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260998+0000","last_change":"2024-01-31T05:03:39.244622+0000","last_active":"2024-01-31T05:03:39.260998+0000","last_peered":"2024-01-31T05:03:39.260998+0000","last_clean":"2024-01-31T05:03:39.260998+0000","last_became_active":"2024-01-31T05:03:39.243824+0000","last_became_peered":"2024-01-31T05:03:39.243824+0000","last_unstale":"2024-01-31T05:03:39.260998+0000","last_undegraded":"2024-01-31T05:03:39.260998+0000","last_fullsized":"2024-01-31T05:03:39.260998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234692+0000","last_change":"2024-01-31T05:03:38.230982+0000","last_active":"2024-01-31T05:03:39.234692+0000","last_peered":"2024-01-31T05:03:39.234692+0000","last_clean":"2024-01-31T05:03:39.234692+0000","last_became_active":"2024-01-31T05:03:38.230687+0000","last_became_peered":"2024-01-31T05:03:38.230687+0000","last_unstale":"2024-01-31T05:03:39.234692+0000","last_undegraded":"2024-01-31T05:03:39.234692+0000","last_fullsized":"2024-01-31T05:03:39.234692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220574+0000","last_change":"2024-01-31T05:03:38.220574+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220574+0000","last_undegraded":"2024-01-31T05:03:38.220574+0000","last_fullsized":"2024-01-31T05:03:38.220574+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240721+0000","last_change":"2024-01-31T05:03:38.236445+0000","last_active":"2024-01-31T05:03:38.240721+0000","last_peered":"2024-01-31T05:03:38.240721+0000","last_clean":"2024-01-31T05:03:38.240721+0000","last_became_active":"2024-01-31T05:03:38.236193+0000","last_became_peered":"2024-01-31T05:03:38.236193+0000","last_unstale":"2024-01-31T05:03:38.240721+0000","last_undegraded":"2024-01-31T05:03:38.240721+0000","last_fullsized":"2024-01-31T05:03:38.240721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218290+0000","last_change":"2024-01-31T05:03:38.218290+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218290+0000","last_undegraded":"2024-01-31T05:03:38.218290+0000","last_fullsized":"2024-01-31T05:03:38.218290+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.236414+0000","last_change":"2024-01-31T05:03:38.223052+0000","last_active":"2024-01-31T05:03:39.236414+0000","last_peered":"2024-01-31T05:03:39.236414+0000","last_clean":"2024-01-31T05:03:39.236414+0000","last_became_active":"2024-01-31T05:03:38.222742+0000","last_became_peered":"2024-01-31T05:03:38.222742+0000","last_unstale":"2024-01-31T05:03:39.236414+0000","last_undegraded":"2024-01-31T05:03:39.236414+0000","last_fullsized":"2024-01-31T05:03:39.236414+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267542+0000","last_change":"2024-01-31T05:03:39.252433+0000","last_active":"2024-01-31T05:03:39.267542+0000","last_peered":"2024-01-31T05:03:39.267542+0000","last_clean":"2024-01-31T05:03:39.267542+0000","last_became_active":"2024-01-31T05:03:39.249986+0000","last_became_peered":"2024-01-31T05:03:39.249986+0000","last_unstale":"2024-01-31T05:03:39.267542+0000","last_undegraded":"2024-01-31T05:03:39.267542+0000","last_fullsized":"2024-01-31T05:03:39.267542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:06.790339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240724+0000","last_change":"2024-01-31T05:03:38.237692+0000","last_active":"2024-01-31T05:03:38.240724+0000","last_peered":"2024-01-31T05:03:38.240724+0000","last_clean":"2024-01-31T05:03:38.240724+0000","last_became_active":"2024-01-31T05:03:38.236259+0000","last_became_peered":"2024-01-31T05:03:38.236259+0000","last_unstale":"2024-01-31T05:03:38.240724+0000","last_undegraded":"2024-01-31T05:03:38.240724+0000","last_fullsized":"2024-01-31T05:03:38.240724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:06:57.676616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219356+0000","last_change":"2024-01-31T05:03:38.219356+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219356+0000","last_undegraded":"2024-01-31T05:03:38.219356+0000","last_fullsized":"2024-01-31T05:03:38.219356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278537+0000","last_change":"2024-01-31T05:03:38.225581+0000","last_active":"2024-01-31T05:03:39.278537+0000","last_peered":"2024-01-31T05:03:39.278537+0000","last_clean":"2024-01-31T05:03:39.278537+0000","last_became_active":"2024-01-31T05:03:38.225059+0000","last_became_peered":"2024-01-31T05:03:38.225059+0000","last_unstale":"2024-01-31T05:03:39.278537+0000","last_undegraded":"2024-01-31T05:03:39.278537+0000","last_fullsized":"2024-01-31T05:03:39.278537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:03:20.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259585+0000","last_change":"2024-01-31T05:03:39.250182+0000","last_active":"2024-01-31T05:03:39.259585+0000","last_peered":"2024-01-31T05:03:39.259585+0000","last_clean":"2024-01-31T05:03:39.259585+0000","last_became_active":"2024-01-31T05:03:39.249917+0000","last_became_peered":"2024-01-31T05:03:39.249917+0000","last_unstale":"2024-01-31T05:03:39.259585+0000","last_undegraded":"2024-01-31T05:03:39.259585+0000","last_fullsized":"2024-01-31T05:03:39.259585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:42:20.611592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.235944+0000","last_change":"2024-01-31T05:03:38.230550+0000","last_active":"2024-01-31T05:03:38.235944+0000","last_peered":"2024-01-31T05:03:38.235944+0000","last_clean":"2024-01-31T05:03:38.235944+0000","last_became_active":"2024-01-31T05:03:38.230284+0000","last_became_peered":"2024-01-31T05:03:38.230284+0000","last_unstale":"2024-01-31T05:03:38.235944+0000","last_undegraded":"2024-01-31T05:03:38.235944+0000","last_fullsized":"2024-01-31T05:03:38.235944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.249515+0000","last_change":"2024-01-31T05:03:39.246221+0000","last_active":"2024-01-31T05:03:39.249515+0000","last_peered":"2024-01-31T05:03:39.249515+0000","last_clean":"2024-01-31T05:03:39.249515+0000","last_became_active":"2024-01-31T05:03:39.244574+0000","last_became_peered":"2024-01-31T05:03:39.244574+0000","last_unstale":"2024-01-31T05:03:39.249515+0000","last_undegraded":"2024-01-31T05:03:39.249515+0000","last_fullsized":"2024-01-31T05:03:39.249515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240559+0000","last_change":"2024-01-31T05:03:38.237263+0000","last_active":"2024-01-31T05:03:38.240559+0000","last_peered":"2024-01-31T05:03:38.240559+0000","last_clean":"2024-01-31T05:03:38.240559+0000","last_became_active":"2024-01-31T05:03:38.236868+0000","last_became_peered":"2024-01-31T05:03:38.236868+0000","last_unstale":"2024-01-31T05:03:38.240559+0000","last_undegraded":"2024-01-31T05:03:38.240559+0000","last_fullsized":"2024-01-31T05:03:38.240559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259275+0000","last_change":"2024-01-31T05:03:39.242937+0000","last_active":"2024-01-31T05:03:39.259275+0000","last_peered":"2024-01-31T05:03:39.259275+0000","last_clean":"2024-01-31T05:03:39.259275+0000","last_became_active":"2024-01-31T05:03:39.241602+0000","last_became_peered":"2024-01-31T05:03:39.241602+0000","last_unstale":"2024-01-31T05:03:39.259275+0000","last_undegraded":"2024-01-31T05:03:39.259275+0000","last_fullsized":"2024-01-31T05:03:39.259275+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233462+0000","last_change":"2024-01-31T05:03:38.225322+0000","last_active":"2024-01-31T05:03:39.233462+0000","last_peered":"2024-01-31T05:03:39.233462+0000","last_clean":"2024-01-31T05:03:39.233462+0000","last_became_active":"2024-01-31T05:03:38.224422+0000","last_became_peered":"2024-01-31T05:03:38.224422+0000","last_unstale":"2024-01-31T05:03:39.233462+0000","last_undegraded":"2024-01-31T05:03:39.233462+0000","last_fullsized":"2024-01-31T05:03:39.233462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:44.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269779+0000","last_change":"2024-01-31T05:03:39.258087+0000","last_active":"2024-01-31T05:03:39.269779+0000","last_peered":"2024-01-31T05:03:39.269779+0000","last_clean":"2024-01-31T05:03:39.269779+0000","last_became_active":"2024-01-31T05:03:39.257724+0000","last_became_peered":"2024-01-31T05:03:39.257724+0000","last_unstale":"2024-01-31T05:03:39.269779+0000","last_undegraded":"2024-01-31T05:03:39.269779+0000","last_fullsized":"2024-01-31T05:03:39.269779+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.231884+0000","last_change":"2024-01-31T05:03:38.219525+0000","last_active":"2024-01-31T05:03:39.231884+0000","last_peered":"2024-01-31T05:03:39.231884+0000","last_clean":"2024-01-31T05:03:39.231884+0000","last_became_active":"2024-01-31T05:03:38.219302+0000","last_became_peered":"2024-01-31T05:03:38.219302+0000","last_unstale":"2024-01-31T05:03:39.231884+0000","last_undegraded":"2024-01-31T05:03:39.231884+0000","last_fullsized":"2024-01-31T05:03:39.231884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:08.160317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260419+0000","last_change":"2024-01-31T05:03:39.253326+0000","last_active":"2024-01-31T05:03:39.260419+0000","last_peered":"2024-01-31T05:03:39.260419+0000","last_clean":"2024-01-31T05:03:39.260419+0000","last_became_active":"2024-01-31T05:03:39.252687+0000","last_became_peered":"2024-01-31T05:03:39.252687+0000","last_unstale":"2024-01-31T05:03:39.260419+0000","last_undegraded":"2024-01-31T05:03:39.260419+0000","last_fullsized":"2024-01-31T05:03:39.260419+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:23:14.519189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278703+0000","last_change":"2024-01-31T05:03:38.224819+0000","last_active":"2024-01-31T05:03:39.278703+0000","last_peered":"2024-01-31T05:03:39.278703+0000","last_clean":"2024-01-31T05:03:39.278703+0000","last_became_active":"2024-01-31T05:03:38.224555+0000","last_became_peered":"2024-01-31T05:03:38.224555+0000","last_unstale":"2024-01-31T05:03:39.278703+0000","last_undegraded":"2024-01-31T05:03:39.278703+0000","last_fullsized":"2024-01-31T05:03:39.278703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:15:12.301634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266298+0000","last_change":"2024-01-31T05:03:39.249020+0000","last_active":"2024-01-31T05:03:39.266298+0000","last_peered":"2024-01-31T05:03:39.266298+0000","last_clean":"2024-01-31T05:03:39.266298+0000","last_became_active":"2024-01-31T05:03:39.246977+0000","last_became_peered":"2024-01-31T05:03:39.246977+0000","last_unstale":"2024-01-31T05:03:39.266298+0000","last_undegraded":"2024-01-31T05:03:39.266298+0000","last_fullsized":"2024-01-31T05:03:39.266298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272054+0000","last_change":"2024-01-31T05:03:38.239609+0000","last_active":"2024-01-31T05:03:39.272054+0000","last_peered":"2024-01-31T05:03:39.272054+0000","last_clean":"2024-01-31T05:03:39.272054+0000","last_became_active":"2024-01-31T05:03:38.238339+0000","last_became_peered":"2024-01-31T05:03:38.238339+0000","last_unstale":"2024-01-31T05:03:39.272054+0000","last_undegraded":"2024-01-31T05:03:39.272054+0000","last_fullsized":"2024-01-31T05:03:39.272054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267510+0000","last_change":"2024-01-31T05:03:39.250705+0000","last_active":"2024-01-31T05:03:39.267510+0000","last_peered":"2024-01-31T05:03:39.267510+0000","last_clean":"2024-01-31T05:03:39.267510+0000","last_became_active":"2024-01-31T05:03:39.248093+0000","last_became_peered":"2024-01-31T05:03:39.248093+0000","last_unstale":"2024-01-31T05:03:39.267510+0000","last_undegraded":"2024-01-31T05:03:39.267510+0000","last_fullsized":"2024-01-31T05:03:39.267510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240743+0000","last_change":"2024-01-31T05:03:38.237901+0000","last_active":"2024-01-31T05:03:38.240743+0000","last_peered":"2024-01-31T05:03:38.240743+0000","last_clean":"2024-01-31T05:03:38.240743+0000","last_became_active":"2024-01-31T05:03:38.237413+0000","last_became_peered":"2024-01-31T05:03:38.237413+0000","last_unstale":"2024-01-31T05:03:38.240743+0000","last_undegraded":"2024-01-31T05:03:38.240743+0000","last_fullsized":"2024-01-31T05:03:38.240743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:58:23.675405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262915+0000","last_change":"2024-01-31T05:03:39.257158+0000","last_active":"2024-01-31T05:03:39.262915+0000","last_peered":"2024-01-31T05:03:39.262915+0000","last_clean":"2024-01-31T05:03:39.262915+0000","last_became_active":"2024-01-31T05:03:39.256870+0000","last_became_peered":"2024-01-31T05:03:39.256870+0000","last_unstale":"2024-01-31T05:03:39.262915+0000","last_undegraded":"2024-01-31T05:03:39.262915+0000","last_fullsized":"2024-01-31T05:03:39.262915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277453+0000","last_change":"2024-01-31T05:03:38.230887+0000","last_active":"2024-01-31T05:03:39.277453+0000","last_peered":"2024-01-31T05:03:39.277453+0000","last_clean":"2024-01-31T05:03:39.277453+0000","last_became_active":"2024-01-31T05:03:38.229073+0000","last_became_peered":"2024-01-31T05:03:38.229073+0000","last_unstale":"2024-01-31T05:03:39.277453+0000","last_undegraded":"2024-01-31T05:03:39.277453+0000","last_fullsized":"2024-01-31T05:03:39.277453+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259604+0000","last_change":"2024-01-31T05:03:39.254309+0000","last_active":"2024-01-31T05:03:39.259604+0000","last_peered":"2024-01-31T05:03:39.259604+0000","last_clean":"2024-01-31T05:03:39.259604+0000","last_became_active":"2024-01-31T05:03:39.254044+0000","last_became_peered":"2024-01-31T05:03:39.254044+0000","last_unstale":"2024-01-31T05:03:39.259604+0000","last_undegraded":"2024-01-31T05:03:39.259604+0000","last_fullsized":"2024-01-31T05:03:39.259604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:33:55.111654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240233+0000","last_change":"2024-01-31T05:03:38.234624+0000","last_active":"2024-01-31T05:03:38.240233+0000","last_peered":"2024-01-31T05:03:38.240233+0000","last_clean":"2024-01-31T05:03:38.240233+0000","last_became_active":"2024-01-31T05:03:38.234296+0000","last_became_peered":"2024-01-31T05:03:38.234296+0000","last_unstale":"2024-01-31T05:03:38.240233+0000","last_undegraded":"2024-01-31T05:03:38.240233+0000","last_fullsized":"2024-01-31T05:03:38.240233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269815+0000","last_change":"2024-01-31T05:03:39.260082+0000","last_active":"2024-01-31T05:03:39.269815+0000","last_peered":"2024-01-31T05:03:39.269815+0000","last_clean":"2024-01-31T05:03:39.269815+0000","last_became_active":"2024-01-31T05:03:39.256821+0000","last_became_peered":"2024-01-31T05:03:39.256821+0000","last_unstale":"2024-01-31T05:03:39.269815+0000","last_undegraded":"2024-01-31T05:03:39.269815+0000","last_fullsized":"2024-01-31T05:03:39.269815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.274034+0000","last_change":"2024-01-31T05:03:38.234981+0000","last_active":"2024-01-31T05:03:39.274034+0000","last_peered":"2024-01-31T05:03:39.274034+0000","last_clean":"2024-01-31T05:03:39.274034+0000","last_became_active":"2024-01-31T05:03:38.234678+0000","last_became_peered":"2024-01-31T05:03:38.234678+0000","last_unstale":"2024-01-31T05:03:39.274034+0000","last_undegraded":"2024-01-31T05:03:39.274034+0000","last_fullsized":"2024-01-31T05:03:39.274034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:08:18.852860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218772+0000","last_change":"2024-01-31T05:03:38.218772+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218772+0000","last_undegraded":"2024-01-31T05:03:38.218772+0000","last_fullsized":"2024-01-31T05:03:38.218772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238837+0000","last_change":"2024-01-31T05:03:38.230732+0000","last_active":"2024-01-31T05:03:38.238837+0000","last_peered":"2024-01-31T05:03:38.238837+0000","last_clean":"2024-01-31T05:03:38.238837+0000","last_became_active":"2024-01-31T05:03:38.229403+0000","last_became_peered":"2024-01-31T05:03:38.229403+0000","last_unstale":"2024-01-31T05:03:38.238837+0000","last_undegraded":"2024-01-31T05:03:38.238837+0000","last_fullsized":"2024-01-31T05:03:38.238837+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:00:00.029942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.216599+0000","last_change":"2024-01-31T05:03:38.216599+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.216599+0000","last_undegraded":"2024-01-31T05:03:38.216599+0000","last_fullsized":"2024-01-31T05:03:38.216599+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232247+0000","last_change":"2024-01-31T05:03:38.230238+0000","last_active":"2024-01-31T05:03:39.232247+0000","last_peered":"2024-01-31T05:03:39.232247+0000","last_clean":"2024-01-31T05:03:39.232247+0000","last_became_active":"2024-01-31T05:03:38.229961+0000","last_became_peered":"2024-01-31T05:03:38.229961+0000","last_unstale":"2024-01-31T05:03:39.232247+0000","last_undegraded":"2024-01-31T05:03:39.232247+0000","last_fullsized":"2024-01-31T05:03:39.232247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266259+0000","last_change":"2024-01-31T05:03:39.248707+0000","last_active":"2024-01-31T05:03:39.266259+0000","last_peered":"2024-01-31T05:03:39.266259+0000","last_clean":"2024-01-31T05:03:39.266259+0000","last_became_active":"2024-01-31T05:03:39.246545+0000","last_became_peered":"2024-01-31T05:03:39.246545+0000","last_unstale":"2024-01-31T05:03:39.266259+0000","last_undegraded":"2024-01-31T05:03:39.266259+0000","last_fullsized":"2024-01-31T05:03:39.266259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.271942+0000","last_change":"2024-01-31T05:03:38.232128+0000","last_active":"2024-01-31T05:03:39.271942+0000","last_peered":"2024-01-31T05:03:39.271942+0000","last_clean":"2024-01-31T05:03:39.271942+0000","last_became_active":"2024-01-31T05:03:38.231862+0000","last_became_peered":"2024-01-31T05:03:38.231862+0000","last_unstale":"2024-01-31T05:03:39.271942+0000","last_undegraded":"2024-01-31T05:03:39.271942+0000","last_fullsized":"2024-01-31T05:03:39.271942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267560+0000","last_change":"2024-01-31T05:03:39.252646+0000","last_active":"2024-01-31T05:03:39.267560+0000","last_peered":"2024-01-31T05:03:39.267560+0000","last_clean":"2024-01-31T05:03:39.267560+0000","last_became_active":"2024-01-31T05:03:39.244259+0000","last_became_peered":"2024-01-31T05:03:39.244259+0000","last_unstale":"2024-01-31T05:03:39.267560+0000","last_undegraded":"2024-01-31T05:03:39.267560+0000","last_fullsized":"2024-01-31T05:03:39.267560+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:01:30.708807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240725+0000","last_change":"2024-01-31T05:03:38.239097+0000","last_active":"2024-01-31T05:03:38.240725+0000","last_peered":"2024-01-31T05:03:38.240725+0000","last_clean":"2024-01-31T05:03:38.240725+0000","last_became_active":"2024-01-31T05:03:38.238757+0000","last_became_peered":"2024-01-31T05:03:38.238757+0000","last_unstale":"2024-01-31T05:03:38.240725+0000","last_undegraded":"2024-01-31T05:03:38.240725+0000","last_fullsized":"2024-01-31T05:03:38.240725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:18:04.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246623+0000","last_change":"2024-01-31T05:03:39.244177+0000","last_active":"2024-01-31T05:03:39.246623+0000","last_peered":"2024-01-31T05:03:39.246623+0000","last_clean":"2024-01-31T05:03:39.246623+0000","last_became_active":"2024-01-31T05:03:39.243841+0000","last_became_peered":"2024-01-31T05:03:39.243841+0000","last_unstale":"2024-01-31T05:03:39.246623+0000","last_undegraded":"2024-01-31T05:03:39.246623+0000","last_fullsized":"2024-01-31T05:03:39.246623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234411+0000","last_change":"2024-01-31T05:03:38.224802+0000","last_active":"2024-01-31T05:03:39.234411+0000","last_peered":"2024-01-31T05:03:39.234411+0000","last_clean":"2024-01-31T05:03:39.234411+0000","last_became_active":"2024-01-31T05:03:38.223989+0000","last_became_peered":"2024-01-31T05:03:38.223989+0000","last_unstale":"2024-01-31T05:03:39.234411+0000","last_undegraded":"2024-01-31T05:03:39.234411+0000","last_fullsized":"2024-01-31T05:03:39.234411+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219620+0000","last_change":"2024-01-31T05:03:38.219620+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219620+0000","last_undegraded":"2024-01-31T05:03:38.219620+0000","last_fullsized":"2024-01-31T05:03:38.219620+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.230405+0000","last_change":"2024-01-31T05:03:38.224866+0000","last_active":"2024-01-31T05:03:39.230405+0000","last_peered":"2024-01-31T05:03:39.230405+0000","last_clean":"2024-01-31T05:03:39.230405+0000","last_became_active":"2024-01-31T05:03:38.224437+0000","last_became_peered":"2024-01-31T05:03:38.224437+0000","last_unstale":"2024-01-31T05:03:39.230405+0000","last_undegraded":"2024-01-31T05:03:39.230405+0000","last_fullsized":"2024-01-31T05:03:39.230405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224396+0000","last_change":"2024-01-31T05:03:38.224396+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224396+0000","last_undegraded":"2024-01-31T05:03:38.224396+0000","last_fullsized":"2024-01-31T05:03:38.224396+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.225219+0000","last_change":"2024-01-31T05:03:38.236959+0000","last_active":"2024-01-31T05:03:39.225219+0000","last_peered":"2024-01-31T05:03:39.225219+0000","last_clean":"2024-01-31T05:03:39.225219+0000","last_became_active":"2024-01-31T05:03:38.235737+0000","last_became_peered":"2024-01-31T05:03:38.235737+0000","last_unstale":"2024-01-31T05:03:39.225219+0000","last_undegraded":"2024-01-31T05:03:39.225219+0000","last_fullsized":"2024-01-31T05:03:39.225219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:44.258646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.217657+0000","last_change":"2024-01-31T05:03:38.217657+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.217657+0000","last_undegraded":"2024-01-31T05:03:38.217657+0000","last_fullsized":"2024-01-31T05:03:38.217657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233182+0000","last_change":"2024-01-31T05:03:38.231189+0000","last_active":"2024-01-31T05:03:39.233182+0000","last_peered":"2024-01-31T05:03:39.233182+0000","last_clean":"2024-01-31T05:03:39.233182+0000","last_became_active":"2024-01-31T05:03:38.230998+0000","last_became_peered":"2024-01-31T05:03:38.230998+0000","last_unstale":"2024-01-31T05:03:39.233182+0000","last_undegraded":"2024-01-31T05:03:39.233182+0000","last_fullsized":"2024-01-31T05:03:39.233182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.213769+0000","last_change":"2024-01-31T05:03:38.213769+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.213769+0000","last_undegraded":"2024-01-31T05:03:38.213769+0000","last_fullsized":"2024-01-31T05:03:38.213769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238725+0000","last_change":"2024-01-31T05:03:38.235803+0000","last_active":"2024-01-31T05:03:38.238725+0000","last_peered":"2024-01-31T05:03:38.238725+0000","last_clean":"2024-01-31T05:03:38.238725+0000","last_became_active":"2024-01-31T05:03:38.234927+0000","last_became_peered":"2024-01-31T05:03:38.234927+0000","last_unstale":"2024-01-31T05:03:38.238725+0000","last_undegraded":"2024-01-31T05:03:38.238725+0000","last_fullsized":"2024-01-31T05:03:38.238725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:14.881712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246903+0000","last_change":"2024-01-31T05:03:39.243770+0000","last_active":"2024-01-31T05:03:39.246903+0000","last_peered":"2024-01-31T05:03:39.246903+0000","last_clean":"2024-01-31T05:03:39.246903+0000","last_became_active":"2024-01-31T05:03:39.243396+0000","last_became_peered":"2024-01-31T05:03:39.243396+0000","last_unstale":"2024-01-31T05:03:39.246903+0000","last_undegraded":"2024-01-31T05:03:39.246903+0000","last_fullsized":"2024-01-31T05:03:39.246903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240286+0000","last_change":"2024-01-31T05:03:38.235205+0000","last_active":"2024-01-31T05:03:38.240286+0000","last_peered":"2024-01-31T05:03:38.240286+0000","last_clean":"2024-01-31T05:03:38.240286+0000","last_became_active":"2024-01-31T05:03:38.234627+0000","last_became_peered":"2024-01-31T05:03:38.234627+0000","last_unstale":"2024-01-31T05:03:38.240286+0000","last_undegraded":"2024-01-31T05:03:38.240286+0000","last_fullsized":"2024-01-31T05:03:38.240286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.212184+0000","last_change":"2024-01-31T05:03:38.212184+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.212184+0000","last_undegraded":"2024-01-31T05:03:38.212184+0000","last_fullsized":"2024-01-31T05:03:38.212184+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.236723+0000","last_change":"2024-01-31T05:03:38.233879+0000","last_active":"2024-01-31T05:03:38.236723+0000","last_peered":"2024-01-31T05:03:38.236723+0000","last_clean":"2024-01-31T05:03:38.236723+0000","last_became_active":"2024-01-31T05:03:38.233495+0000","last_became_peered":"2024-01-31T05:03:38.233495+0000","last_unstale":"2024-01-31T05:03:38.236723+0000","last_undegraded":"2024-01-31T05:03:38.236723+0000","last_fullsized":"2024-01-31T05:03:38.236723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232380+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:39.232380+0000","last_peered":"2024-01-31T05:03:39.232380+0000","last_clean":"2024-01-31T05:03:39.232380+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:39.232380+0000","last_undegraded":"2024-01-31T05:03:39.232380+0000","last_fullsized":"2024-01-31T05:03:39.232380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":275170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0149999999999999,"15min":1.0009999999999999},"min":{"1min":0.68200000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5369999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.099,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6719999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.244,"15min":1.256},"min":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6259999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.872}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.175,"15min":1.1799999999999999},"min":{"1min":0.747,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4930000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.29,"15min":1.292},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.512,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.016}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2230000000000001,"15min":1.2150000000000001},"min":{"1min":0.91900000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.609,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6120000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.179}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":0.98499999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.609,"5min":2.25,"15min":2.25},"last":1.349},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3580000000000001,"15min":1.3740000000000001},"min":{"1min":1.004,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.573,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.472,"15min":1.4750000000000001},"min":{"1min":1.0109999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.762,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.85999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":1.0089999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.1240000000000001,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5529999999999999,"15min":1.5609999999999999},"min":{"1min":1.2709999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.726,"5min":2.629,"15min":2.629},"last":1.6359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":282881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0329999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3240000000000001,"5min":1.476,"15min":1.476},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.05,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.876}]},{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.78600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.587,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1200000000000001,"15min":1.131},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5369999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.48}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.6950000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.569},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1799999999999999,"15min":1.1839999999999999},"min":{"1min":0.80400000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.421,"5min":1.883,"15min":1.883},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.381,"15min":1.395},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.72,"5min":1.73,"15min":1.73},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.95599999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0830000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6259999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.878,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.611,"5min":2.032,"15min":2.032},"last":1.012},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.585},"min":{"1min":1.0780000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.984,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.165}]},{"osd":6,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5209999999999999,"15min":1.5329999999999999},"min":{"1min":0.96899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.972,"5min":2.335,"15min":2.335},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5860000000000001,"15min":1.6060000000000001},"min":{"1min":1.2609999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.768,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.115}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.599,"15min":1.6160000000000001},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8620000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.071},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.637,"15min":1.659},"min":{"1min":1.0920000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.891,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.7350000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":282355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.151},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1919999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":1.032,"15min":1.0840000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1439999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.121,"15min":1.1779999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4910000000000001,"5min":5.617,"15min":5.617},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.2330000000000001,"15min":1.2989999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.94599999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.292,"15min":1.3680000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.484,"15min":1.5720000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4930000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.6220000000000001,"15min":1.7230000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.9990000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.921},{"interface":"front","average":{"1min":1.141,"5min":1.621,"15min":1.7010000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.696,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.2130000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.639,"15min":1.7230000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9339999999999999,"5min":6.048,"15min":6.048},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.7430000000000001,"15min":1.831},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.952,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.804}]},{"osd":6,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.6739999999999999,"15min":1.744},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.024,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.657,"15min":1.748},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5700000000000001,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.6539999999999999,"15min":1.74},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.802,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.8380000000000001,"15min":1.946},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.73,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.5609999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":284037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.667,"5min":1.73,"15min":1.73},"last":1.4390000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.88900000000000001,"15min":0.90300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.472,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.046},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.60499999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.099}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93000000000000005,"15min":0.92300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.23},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0780000000000001,"15min":1.087},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":1.798,"15min":1.798},"last":1.387}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5780000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3220000000000001,"15min":1.331},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0059999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.321}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.232,"15min":1.2410000000000001},"min":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.585,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.472},{"interface":"front","average":{"1min":1.26,"5min":1.3100000000000001,"15min":1.3180000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8220000000000001,"5min":2.52,"15min":2.52},"last":1.494}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.419},"min":{"1min":1.0189999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1320000000000001,"5min":2.992,"15min":2.992},"last":1.663},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3939999999999999,"15min":1.4079999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.6879999999999999,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.397,"15min":1.403},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.8700000000000001,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.869,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.619}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1236992,"data_stored":873215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":1.026,"15min":1.0820000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.272,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.149,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4890000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.98399999999999999,"15min":1.0289999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0229999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.131,"15min":1.1839999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4099999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":1.0109999999999999,"15min":1.0449999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4419999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1639999999999999,"15min":1.222},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.294,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.028}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.4550000000000001,"15min":1.5089999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5940000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.345,"15min":1.379},"min":{"1min":0.80900000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4630000000000001,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.4059999999999999,"15min":1.4470000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.4650000000000001,"5min":2.722,"15min":2.722},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.409,"15min":1.4550000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.587,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.5189999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4790000000000001,"15min":1.538},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6439999999999999,"5min":2.577,"15min":2.577},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.415,"15min":1.4510000000000001},"min":{"1min":0.85699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5609999999999999,"5min":2.794,"15min":2.794},"last":1.651}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.4930000000000001,"15min":1.5449999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.647,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.5640000000000001,"15min":1.615},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5489999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.581}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.135,"15min":1.151},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.091},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.552,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1339999999999999,"15min":1.131},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5589999999999999,"5min":1.573,"15min":1.573},"last":1.242},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.73799999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5349999999999999,"5min":1.976,"15min":1.976},"last":1.1459999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7470000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.266,"15min":1.28},"min":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.544,"5min":1.873,"15min":1.873},"last":1.321}]},{"osd":3,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3580000000000001,"15min":1.375},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.7050000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.335,"15min":1.339},"min":{"1min":0.95899999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.4399999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.254,"15min":1.2529999999999999},"min":{"1min":0.79300000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6890000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":0.94399999999999995,"5min":0.628,"15min":0.628},"max":{"1min":2.0499999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4430000000000001,"15min":1.454},"min":{"1min":1.0049999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.679,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.379},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.593,"15min":1.6040000000000001},"min":{"1min":1.1990000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0990000000000002,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.462}]},{"osd":7,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5620000000000001,"15min":1.5820000000000001},"min":{"1min":0.89400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9530000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6719999999999999,"15min":1.6839999999999999},"min":{"1min":1.246,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9990000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.409}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":865538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.038,"15min":1.022},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.792}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1279999999999999,"15min":1.141},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.2030000000000001,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5720000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.3320000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.206,"15min":1.222},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.663,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.29},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.296,"15min":1.3089999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.647,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.49}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.304,"15min":1.306},"min":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.704,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.389,"15min":1.399},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.712,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.5309999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":0.70399999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.0139999999999998,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.91400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.792,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4259999999999999,"15min":1.4099999999999999},"min":{"1min":0.93500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":1.494},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.851,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5860000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.948,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.0029999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5219999999999998,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.6240000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0389999999999999,"15min":1.0489999999999999},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.0680000000000001,"5min":3.282,"15min":3.282},"last":0.627}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.075,"5min":1.774,"15min":1.774},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.091,"15min":1.089},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.2250000000000001},"min":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4370000000000001,"15min":1.4399999999999999},"min":{"1min":0.93999999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.8500000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.097}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.492,"15min":1.482},"min":{"1min":0.85399999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":5.75,"5min":5.75,"15min":5.75},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.468,"15min":1.4750000000000001},"min":{"1min":0.86499999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":4.8799999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.169}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.355,"15min":1.3300000000000001},"min":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":7.415,"5min":7.415,"15min":7.415},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.704,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":0.88900000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":7.5650000000000004,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.298}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.877,"5min":0.751,"15min":0.751},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:03:43.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:03:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.887+0000 7f31a9000640 1 Processor -- start 2024-01-31T05:03:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.887+0000 7f31a9000640 1 -- start start 2024-01-31T05:03:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.888+0000 7f31a9000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f31a415d4e0 0x7f31a40b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a9000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a415d8c0 con 0x7f31a40d1c70 2024-01-31T05:03:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a9000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a40b1920 con 0x7f31a40d0d60 2024-01-31T05:03:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a9000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a40b0b20 con 0x7f31a415d4e0 2024-01-31T05:03:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31a40d1c70 0x7f31a40b5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55376/0 (socket says 172.21.15.97:55376) 2024-01-31T05:03:43.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a3e00640 1 -- 172.21.15.97:0/3818367172 learned_addr learned my addr 172.21.15.97:0/3818367172 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380889765 0 0) 0x7f319c001490 con 0x7f31a40d1c70 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3184003520 con 0x7f31a40d1c70 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.889+0000 7f31a3400640 1 --2- 172.21.15.97:0/3818367172 >> v2:172.21.15.188:3300/0 conn(0x7f31a415d4e0 0x7f31a40b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1729237558 0 0) 0x7f3184003520 con 0x7f31a40d1c70 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 >> v2:172.21.15.188:3300/0 conn(0x7f31a415d4e0 msgr2=0x7f31a40b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 --2- 172.21.15.97:0/3818367172 >> v2:172.21.15.188:3300/0 conn(0x7f31a415d4e0 0x7f31a40b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 >> v1:172.21.15.188:6789/0 conn(0x7f31a40d0d60 legacy=0x7f31a40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40cef50 con 0x7f31a40d1c70 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f319c001b80 con 0x7f31a40d1c70 2024-01-31T05:03:43.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 >> v1:172.21.15.97:6789/0 conn(0x7f31a40d1c70 legacy=0x7f31a40b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3184003970 0x7f3184003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31840047c0 0x7f3184004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184005180 0x7f318400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318400c060 con 0x7f31840047c0 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318400c1d0 con 0x7f3184003970 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318400c390 con 0x7f3184005180 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2a00640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31840047c0 0x7f3184004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a3400640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3184003970 0x7f3184003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a3e00640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184005180 0x7f318400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2a00640 1 -- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184005180 msgr2=0x7f318400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2a00640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3184005180 0x7f318400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2a00640 1 -- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3184003970 msgr2=0x7f3184003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.890+0000 7f31a2a00640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3184003970 0x7f3184003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.891+0000 7f31a2a00640 1 -- 172.21.15.97:0/3818367172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184003520 con 0x7f31840047c0 2024-01-31T05:03:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.891+0000 7f31a3e00640 1 -- 172.21.15.97:0/3818367172 reap_dead start 2024-01-31T05:03:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a2a00640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31840047c0 0x7f3184004c40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f318c003db0 tx=0x7f31840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e38ff57a75fd76 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c032600 con 0x7f31840047c0 2024-01-31T05:03:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c02e050 con 0x7f31840047c0 2024-01-31T05:03:43.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a2000640 1 -- 172.21.15.97:0/3818367172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c0329e0 con 0x7f31840047c0 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a9000640 1 -- 172.21.15.97:0/3818367172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31840047c0 msgr2=0x7f3184004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a9000640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31840047c0 0x7f3184004c40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f318c003db0 tx=0x7f31840057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a9000640 1 -- 172.21.15.97:0/3818367172 shutdown_connections 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a9000640 1 --2- 172.21.15.97:0/3818367172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31840047c0 0x7f3184004c40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a9000640 1 -- 172.21.15.97:0/3818367172 >> 172.21.15.97:0/3818367172 conn(0x7f31a40d3010 msgr2=0x7f31a40d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.892+0000 7f31a9000640 1 -- 172.21.15.97:0/3818367172 shutdown_connections 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.893+0000 7f31a9000640 1 -- 172.21.15.97:0/3818367172 wait complete. 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.894+0000 7f31a9000640 1 Processor -- start 2024-01-31T05:03:43.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.894+0000 7f31a9000640 1 -- start start 2024-01-31T05:03:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840047c0 0x7f31a415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a9000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 0x7f31a40c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a9000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a406b200 0x7f31a406b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a9000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a406bbc0 con 0x7f31a4151690 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a9000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40c79f0 con 0x7f31840047c0 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a9000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40c8560 con 0x7f31a406b200 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 0x7f31a40c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 0x7f31a40c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42328/0 (socket says 172.21.15.97:42328) 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.895+0000 7f31a2a00640 1 -- 172.21.15.97:0/361448782 learned_addr learned my addr 172.21.15.97:0/361448782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a2a00640 1 -- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a406b200 msgr2=0x7f31a406b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a3e00640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a406b200 0x7f31a406b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a3400640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840047c0 0x7f31a415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a2a00640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a406b200 0x7f31a406b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a2a00640 1 -- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840047c0 msgr2=0x7f31a415f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a2a00640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840047c0 0x7f31a415f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:43.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a2a00640 1 -- 172.21.15.97:0/361448782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40c6c30 con 0x7f31a4151690 2024-01-31T05:03:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a2a00640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 0x7f31a40c8020 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f318c02bbe0 tx=0x7f318c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c03c410 con 0x7f31a4151690 2024-01-31T05:03:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c03c570 con 0x7f31a4151690 2024-01-31T05:03:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c03c920 con 0x7f31a4151690 2024-01-31T05:03:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.896+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a40c6f10 con 0x7f31a4151690 2024-01-31T05:03:43.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.897+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a40c74d0 con 0x7f31a4151690 2024-01-31T05:03:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.898+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a40c7660 con 0x7f31a4151690 2024-01-31T05:03:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.898+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f318c039050 con 0x7f31a4151690 2024-01-31T05:03:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.899+0000 7f319b400640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f317c06aed0 0x7f317c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.899+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f318c0aabf0 con 0x7f31a4151690 2024-01-31T05:03:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.899+0000 7f31a3400640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f317c06aed0 0x7f317c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:43.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.900+0000 7f31a3400640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f317c06aed0 0x7f317c06d390 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31a4050230 tx=0x7f3194008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:43.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:43.903+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f318c075b50 con 0x7f31a4151690 2024-01-31T05:03:44.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.094+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31a40b12d0 con 0x7f317c06aed0 2024-01-31T05:03:44.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.116+0000 7f319b400640 1 -- 172.21.15.97:0/361448782 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284616 (secure 0 0 0) 0x7f319400bd90 con 0x7f317c06aed0 2024-01-31T05:03:44.119 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f317c06aed0 msgr2=0x7f317c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f317c06aed0 0x7f317c06d390 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f31a4050230 tx=0x7f3194008040 comp rx=0 tx=0).stop 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 msgr2=0x7f31a40c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 0x7f31a40c8020 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f318c02bbe0 tx=0x7f318c0036f0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 shutdown_connections 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31a406b200 0x7f31a406b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f317c06aed0 0x7f317c06d390 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a4151690 0x7f31a40c8020 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 --2- 172.21.15.97:0/361448782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31840047c0 0x7f31a415f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 >> 172.21.15.97:0/361448782 conn(0x7f31a40d3010 msgr2=0x7f31a406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 shutdown_connections 2024-01-31T05:03:44.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.122+0000 7f31a9000640 1 -- 172.21.15.97:0/361448782 wait complete. 2024-01-31T05:03:44.140 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-01-31T05:03:43.610324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":188,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222300,"kb_used_data":6116,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752420,"statfs":{"total":773094113280,"available":772866478080,"internally_reserved":0,"allocated":6262784,"data_stored":3400346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001696"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231612+0000","last_change":"2024-01-31T05:03:38.231612+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231612+0000","last_undegraded":"2024-01-31T05:03:38.231612+0000","last_fullsized":"2024-01-31T05:03:38.231612+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.254203+0000","last_change":"2024-01-31T05:03:39.252312+0000","last_active":"2024-01-31T05:03:39.254203+0000","last_peered":"2024-01-31T05:03:39.254203+0000","last_clean":"2024-01-31T05:03:39.254203+0000","last_became_active":"2024-01-31T05:03:39.252042+0000","last_became_peered":"2024-01-31T05:03:39.252042+0000","last_unstale":"2024-01-31T05:03:39.254203+0000","last_undegraded":"2024-01-31T05:03:39.254203+0000","last_fullsized":"2024-01-31T05:03:39.254203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259236+0000","last_change":"2024-01-31T05:03:39.243310+0000","last_active":"2024-01-31T05:03:39.259236+0000","last_peered":"2024-01-31T05:03:39.259236+0000","last_clean":"2024-01-31T05:03:39.259236+0000","last_became_active":"2024-01-31T05:03:39.242309+0000","last_became_peered":"2024-01-31T05:03:39.242309+0000","last_unstale":"2024-01-31T05:03:39.259236+0000","last_undegraded":"2024-01-31T05:03:39.259236+0000","last_fullsized":"2024-01-31T05:03:39.259236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.261556+0000","last_change":"2024-01-31T05:03:39.254633+0000","last_active":"2024-01-31T05:03:39.261556+0000","last_peered":"2024-01-31T05:03:39.261556+0000","last_clean":"2024-01-31T05:03:39.261556+0000","last_became_active":"2024-01-31T05:03:39.254325+0000","last_became_peered":"2024-01-31T05:03:39.254325+0000","last_unstale":"2024-01-31T05:03:39.261556+0000","last_undegraded":"2024-01-31T05:03:39.261556+0000","last_fullsized":"2024-01-31T05:03:39.261556+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:03:40.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222392+0000","last_change":"2024-01-31T05:03:38.222392+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222392+0000","last_undegraded":"2024-01-31T05:03:38.222392+0000","last_fullsized":"2024-01-31T05:03:38.222392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226676+0000","last_change":"2024-01-31T05:03:38.226676+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226676+0000","last_undegraded":"2024-01-31T05:03:38.226676+0000","last_fullsized":"2024-01-31T05:03:38.226676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260398+0000","last_change":"2024-01-31T05:03:39.247885+0000","last_active":"2024-01-31T05:03:39.260398+0000","last_peered":"2024-01-31T05:03:39.260398+0000","last_clean":"2024-01-31T05:03:39.260398+0000","last_became_active":"2024-01-31T05:03:39.247556+0000","last_became_peered":"2024-01-31T05:03:39.247556+0000","last_unstale":"2024-01-31T05:03:39.260398+0000","last_undegraded":"2024-01-31T05:03:39.260398+0000","last_fullsized":"2024-01-31T05:03:39.260398+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:59.282060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.215147+0000","last_change":"2024-01-31T05:03:38.215147+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.215147+0000","last_undegraded":"2024-01-31T05:03:38.215147+0000","last_fullsized":"2024-01-31T05:03:38.215147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.270024+0000","last_change":"2024-01-31T05:03:39.256441+0000","last_active":"2024-01-31T05:03:39.270024+0000","last_peered":"2024-01-31T05:03:39.270024+0000","last_clean":"2024-01-31T05:03:39.270024+0000","last_became_active":"2024-01-31T05:03:39.256105+0000","last_became_peered":"2024-01-31T05:03:39.256105+0000","last_unstale":"2024-01-31T05:03:39.270024+0000","last_undegraded":"2024-01-31T05:03:39.270024+0000","last_fullsized":"2024-01-31T05:03:39.270024+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226447+0000","last_change":"2024-01-31T05:03:38.226447+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226447+0000","last_undegraded":"2024-01-31T05:03:38.226447+0000","last_fullsized":"2024-01-31T05:03:38.226447+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259313+0000","last_change":"2024-01-31T05:03:39.251020+0000","last_active":"2024-01-31T05:03:39.259313+0000","last_peered":"2024-01-31T05:03:39.259313+0000","last_clean":"2024-01-31T05:03:39.259313+0000","last_became_active":"2024-01-31T05:03:39.250684+0000","last_became_peered":"2024-01-31T05:03:39.250684+0000","last_unstale":"2024-01-31T05:03:39.259313+0000","last_undegraded":"2024-01-31T05:03:39.259313+0000","last_fullsized":"2024-01-31T05:03:39.259313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:39:06.883661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269332+0000","last_change":"2024-01-31T05:03:39.258617+0000","last_active":"2024-01-31T05:03:39.269332+0000","last_peered":"2024-01-31T05:03:39.269332+0000","last_clean":"2024-01-31T05:03:39.269332+0000","last_became_active":"2024-01-31T05:03:39.258317+0000","last_became_peered":"2024-01-31T05:03:39.258317+0000","last_unstale":"2024-01-31T05:03:39.269332+0000","last_undegraded":"2024-01-31T05:03:39.269332+0000","last_fullsized":"2024-01-31T05:03:39.269332+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218606+0000","last_change":"2024-01-31T05:03:38.218606+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218606+0000","last_undegraded":"2024-01-31T05:03:38.218606+0000","last_fullsized":"2024-01-31T05:03:38.218606+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267681+0000","last_change":"2024-01-31T05:03:39.251453+0000","last_active":"2024-01-31T05:03:39.267681+0000","last_peered":"2024-01-31T05:03:39.267681+0000","last_clean":"2024-01-31T05:03:39.267681+0000","last_became_active":"2024-01-31T05:03:39.248059+0000","last_became_peered":"2024-01-31T05:03:39.248059+0000","last_unstale":"2024-01-31T05:03:39.267681+0000","last_undegraded":"2024-01-31T05:03:39.267681+0000","last_fullsized":"2024-01-31T05:03:39.267681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.908110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222879+0000","last_change":"2024-01-31T05:03:38.222879+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222879+0000","last_undegraded":"2024-01-31T05:03:38.222879+0000","last_fullsized":"2024-01-31T05:03:38.222879+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220373+0000","last_change":"2024-01-31T05:03:38.220373+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220373+0000","last_undegraded":"2024-01-31T05:03:38.220373+0000","last_fullsized":"2024-01-31T05:03:38.220373+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231217+0000","last_change":"2024-01-31T05:03:38.231217+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231217+0000","last_undegraded":"2024-01-31T05:03:38.231217+0000","last_fullsized":"2024-01-31T05:03:38.231217+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266404+0000","last_change":"2024-01-31T05:03:39.249229+0000","last_active":"2024-01-31T05:03:39.266404+0000","last_peered":"2024-01-31T05:03:39.266404+0000","last_clean":"2024-01-31T05:03:39.266404+0000","last_became_active":"2024-01-31T05:03:39.248929+0000","last_became_peered":"2024-01-31T05:03:39.248929+0000","last_unstale":"2024-01-31T05:03:39.266404+0000","last_undegraded":"2024-01-31T05:03:39.266404+0000","last_fullsized":"2024-01-31T05:03:39.266404+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259739+0000","last_change":"2024-01-31T05:03:39.255701+0000","last_active":"2024-01-31T05:03:39.259739+0000","last_peered":"2024-01-31T05:03:39.259739+0000","last_clean":"2024-01-31T05:03:39.259739+0000","last_became_active":"2024-01-31T05:03:39.253975+0000","last_became_peered":"2024-01-31T05:03:39.253975+0000","last_unstale":"2024-01-31T05:03:39.259739+0000","last_undegraded":"2024-01-31T05:03:39.259739+0000","last_fullsized":"2024-01-31T05:03:39.259739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.263250+0000","last_change":"2024-01-31T05:03:39.258999+0000","last_active":"2024-01-31T05:03:39.263250+0000","last_peered":"2024-01-31T05:03:39.263250+0000","last_clean":"2024-01-31T05:03:39.263250+0000","last_became_active":"2024-01-31T05:03:39.257976+0000","last_became_peered":"2024-01-31T05:03:39.257976+0000","last_unstale":"2024-01-31T05:03:39.263250+0000","last_undegraded":"2024-01-31T05:03:39.263250+0000","last_fullsized":"2024-01-31T05:03:39.263250+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:24:32.083418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218199+0000","last_change":"2024-01-31T05:03:38.218199+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218199+0000","last_undegraded":"2024-01-31T05:03:38.218199+0000","last_fullsized":"2024-01-31T05:03:38.218199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.250048+0000","last_change":"2024-01-31T05:03:39.242603+0000","last_active":"2024-01-31T05:03:39.250048+0000","last_peered":"2024-01-31T05:03:39.250048+0000","last_clean":"2024-01-31T05:03:39.250048+0000","last_became_active":"2024-01-31T05:03:39.241443+0000","last_became_peered":"2024-01-31T05:03:39.241443+0000","last_unstale":"2024-01-31T05:03:39.250048+0000","last_undegraded":"2024-01-31T05:03:39.250048+0000","last_fullsized":"2024-01-31T05:03:39.250048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:28:05.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260437+0000","last_change":"2024-01-31T05:03:39.253595+0000","last_active":"2024-01-31T05:03:39.260437+0000","last_peered":"2024-01-31T05:03:39.260437+0000","last_clean":"2024-01-31T05:03:39.260437+0000","last_became_active":"2024-01-31T05:03:39.253076+0000","last_became_peered":"2024-01-31T05:03:39.253076+0000","last_unstale":"2024-01-31T05:03:39.260437+0000","last_undegraded":"2024-01-31T05:03:39.260437+0000","last_fullsized":"2024-01-31T05:03:39.260437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:57:48.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272382+0000","last_change":"2024-01-31T05:03:39.260142+0000","last_active":"2024-01-31T05:03:39.272382+0000","last_peered":"2024-01-31T05:03:39.272382+0000","last_clean":"2024-01-31T05:03:39.272382+0000","last_became_active":"2024-01-31T05:03:39.255918+0000","last_became_peered":"2024-01-31T05:03:39.255918+0000","last_unstale":"2024-01-31T05:03:39.272382+0000","last_undegraded":"2024-01-31T05:03:39.272382+0000","last_fullsized":"2024-01-31T05:03:39.272382+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.257215+0000","last_change":"2024-01-31T05:03:39.253163+0000","last_active":"2024-01-31T05:03:39.257215+0000","last_peered":"2024-01-31T05:03:39.257215+0000","last_clean":"2024-01-31T05:03:39.257215+0000","last_became_active":"2024-01-31T05:03:39.252895+0000","last_became_peered":"2024-01-31T05:03:39.252895+0000","last_unstale":"2024-01-31T05:03:39.257215+0000","last_undegraded":"2024-01-31T05:03:39.257215+0000","last_fullsized":"2024-01-31T05:03:39.257215+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218748+0000","last_change":"2024-01-31T05:03:38.218748+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218748+0000","last_undegraded":"2024-01-31T05:03:38.218748+0000","last_fullsized":"2024-01-31T05:03:38.218748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.252464+0000","last_change":"2024-01-31T05:03:39.245654+0000","last_active":"2024-01-31T05:03:39.252464+0000","last_peered":"2024-01-31T05:03:39.252464+0000","last_clean":"2024-01-31T05:03:39.252464+0000","last_became_active":"2024-01-31T05:03:39.242973+0000","last_became_peered":"2024-01-31T05:03:39.242973+0000","last_unstale":"2024-01-31T05:03:39.252464+0000","last_undegraded":"2024-01-31T05:03:39.252464+0000","last_fullsized":"2024-01-31T05:03:39.252464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.227220+0000","last_change":"2024-01-31T05:03:38.227220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.227220+0000","last_undegraded":"2024-01-31T05:03:38.227220+0000","last_fullsized":"2024-01-31T05:03:38.227220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219977+0000","last_change":"2024-01-31T05:03:38.219977+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219977+0000","last_undegraded":"2024-01-31T05:03:38.219977+0000","last_fullsized":"2024-01-31T05:03:38.219977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262925+0000","last_change":"2024-01-31T05:03:39.258792+0000","last_active":"2024-01-31T05:03:39.262925+0000","last_peered":"2024-01-31T05:03:39.262925+0000","last_clean":"2024-01-31T05:03:39.262925+0000","last_became_active":"2024-01-31T05:03:39.258531+0000","last_became_peered":"2024-01-31T05:03:39.258531+0000","last_unstale":"2024-01-31T05:03:39.262925+0000","last_undegraded":"2024-01-31T05:03:39.262925+0000","last_fullsized":"2024-01-31T05:03:39.262925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:02:30.641509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243207+0000","last_change":"2024-01-31T05:03:39.236368+0000","last_active":"2024-01-31T05:03:39.243207+0000","last_peered":"2024-01-31T05:03:39.243207+0000","last_clean":"2024-01-31T05:03:39.243207+0000","last_became_active":"2024-01-31T05:03:39.236055+0000","last_became_peered":"2024-01-31T05:03:39.236055+0000","last_unstale":"2024-01-31T05:03:39.243207+0000","last_undegraded":"2024-01-31T05:03:39.243207+0000","last_fullsized":"2024-01-31T05:03:39.243207+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.901754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.241385+0000","last_change":"2024-01-31T05:03:39.239349+0000","last_active":"2024-01-31T05:03:39.241385+0000","last_peered":"2024-01-31T05:03:39.241385+0000","last_clean":"2024-01-31T05:03:39.241385+0000","last_became_active":"2024-01-31T05:03:39.238868+0000","last_became_peered":"2024-01-31T05:03:39.238868+0000","last_unstale":"2024-01-31T05:03:39.241385+0000","last_undegraded":"2024-01-31T05:03:39.241385+0000","last_fullsized":"2024-01-31T05:03:39.241385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:55:11.110405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.225923+0000","last_change":"2024-01-31T05:03:38.225923+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.225923+0000","last_undegraded":"2024-01-31T05:03:38.225923+0000","last_fullsized":"2024-01-31T05:03:38.225923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.276730+0000","last_change":"2024-01-31T05:03:38.235515+0000","last_active":"2024-01-31T05:03:39.276730+0000","last_peered":"2024-01-31T05:03:39.276730+0000","last_clean":"2024-01-31T05:03:39.276730+0000","last_became_active":"2024-01-31T05:03:38.235228+0000","last_became_peered":"2024-01-31T05:03:38.235228+0000","last_unstale":"2024-01-31T05:03:39.276730+0000","last_undegraded":"2024-01-31T05:03:39.276730+0000","last_fullsized":"2024-01-31T05:03:39.276730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:42.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.230764+0000","last_change":"2024-01-31T05:03:38.230764+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.230764+0000","last_undegraded":"2024-01-31T05:03:38.230764+0000","last_fullsized":"2024-01-31T05:03:38.230764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269887+0000","last_change":"2024-01-31T05:03:38.224888+0000","last_active":"2024-01-31T05:03:39.269887+0000","last_peered":"2024-01-31T05:03:39.269887+0000","last_clean":"2024-01-31T05:03:39.269887+0000","last_became_active":"2024-01-31T05:03:38.223355+0000","last_became_peered":"2024-01-31T05:03:38.223355+0000","last_unstale":"2024-01-31T05:03:39.269887+0000","last_undegraded":"2024-01-31T05:03:39.269887+0000","last_fullsized":"2024-01-31T05:03:39.269887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226684+0000","last_change":"2024-01-31T05:03:38.226684+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226684+0000","last_undegraded":"2024-01-31T05:03:38.226684+0000","last_fullsized":"2024-01-31T05:03:38.226684+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.239333+0000","last_change":"2024-01-31T05:03:38.234777+0000","last_active":"2024-01-31T05:03:38.239333+0000","last_peered":"2024-01-31T05:03:38.239333+0000","last_clean":"2024-01-31T05:03:38.239333+0000","last_became_active":"2024-01-31T05:03:38.234493+0000","last_became_peered":"2024-01-31T05:03:38.234493+0000","last_unstale":"2024-01-31T05:03:38.239333+0000","last_undegraded":"2024-01-31T05:03:38.239333+0000","last_fullsized":"2024-01-31T05:03:38.239333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:48:31.432282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224220+0000","last_change":"2024-01-31T05:03:38.224220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224220+0000","last_undegraded":"2024-01-31T05:03:38.224220+0000","last_fullsized":"2024-01-31T05:03:38.224220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277804+0000","last_change":"2024-01-31T05:03:38.221765+0000","last_active":"2024-01-31T05:03:39.277804+0000","last_peered":"2024-01-31T05:03:39.277804+0000","last_clean":"2024-01-31T05:03:39.277804+0000","last_became_active":"2024-01-31T05:03:38.221534+0000","last_became_peered":"2024-01-31T05:03:38.221534+0000","last_unstale":"2024-01-31T05:03:39.277804+0000","last_undegraded":"2024-01-31T05:03:39.277804+0000","last_fullsized":"2024-01-31T05:03:39.277804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:22.374866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.210993+0000","last_change":"2024-01-31T05:03:38.210993+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.210993+0000","last_undegraded":"2024-01-31T05:03:38.210993+0000","last_fullsized":"2024-01-31T05:03:38.210993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277781+0000","last_change":"2024-01-31T05:03:38.226696+0000","last_active":"2024-01-31T05:03:39.277781+0000","last_peered":"2024-01-31T05:03:39.277781+0000","last_clean":"2024-01-31T05:03:39.277781+0000","last_became_active":"2024-01-31T05:03:38.226440+0000","last_became_peered":"2024-01-31T05:03:38.226440+0000","last_unstale":"2024-01-31T05:03:39.277781+0000","last_undegraded":"2024-01-31T05:03:39.277781+0000","last_fullsized":"2024-01-31T05:03:39.277781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243539+0000","last_change":"2024-01-31T05:03:39.240337+0000","last_active":"2024-01-31T05:03:39.243539+0000","last_peered":"2024-01-31T05:03:39.243539+0000","last_clean":"2024-01-31T05:03:39.243539+0000","last_became_active":"2024-01-31T05:03:39.240029+0000","last_became_peered":"2024-01-31T05:03:39.240029+0000","last_unstale":"2024-01-31T05:03:39.243539+0000","last_undegraded":"2024-01-31T05:03:39.243539+0000","last_fullsized":"2024-01-31T05:03:39.243539+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:24:32.372349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240709+0000","last_change":"2024-01-31T05:03:38.239050+0000","last_active":"2024-01-31T05:03:38.240709+0000","last_peered":"2024-01-31T05:03:38.240709+0000","last_clean":"2024-01-31T05:03:38.240709+0000","last_became_active":"2024-01-31T05:03:38.238746+0000","last_became_peered":"2024-01-31T05:03:38.238746+0000","last_unstale":"2024-01-31T05:03:38.240709+0000","last_undegraded":"2024-01-31T05:03:38.240709+0000","last_fullsized":"2024-01-31T05:03:38.240709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:37:59.345660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260998+0000","last_change":"2024-01-31T05:03:39.244622+0000","last_active":"2024-01-31T05:03:39.260998+0000","last_peered":"2024-01-31T05:03:39.260998+0000","last_clean":"2024-01-31T05:03:39.260998+0000","last_became_active":"2024-01-31T05:03:39.243824+0000","last_became_peered":"2024-01-31T05:03:39.243824+0000","last_unstale":"2024-01-31T05:03:39.260998+0000","last_undegraded":"2024-01-31T05:03:39.260998+0000","last_fullsized":"2024-01-31T05:03:39.260998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234692+0000","last_change":"2024-01-31T05:03:38.230982+0000","last_active":"2024-01-31T05:03:39.234692+0000","last_peered":"2024-01-31T05:03:39.234692+0000","last_clean":"2024-01-31T05:03:39.234692+0000","last_became_active":"2024-01-31T05:03:38.230687+0000","last_became_peered":"2024-01-31T05:03:38.230687+0000","last_unstale":"2024-01-31T05:03:39.234692+0000","last_undegraded":"2024-01-31T05:03:39.234692+0000","last_fullsized":"2024-01-31T05:03:39.234692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220574+0000","last_change":"2024-01-31T05:03:38.220574+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220574+0000","last_undegraded":"2024-01-31T05:03:38.220574+0000","last_fullsized":"2024-01-31T05:03:38.220574+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240721+0000","last_change":"2024-01-31T05:03:38.236445+0000","last_active":"2024-01-31T05:03:38.240721+0000","last_peered":"2024-01-31T05:03:38.240721+0000","last_clean":"2024-01-31T05:03:38.240721+0000","last_became_active":"2024-01-31T05:03:38.236193+0000","last_became_peered":"2024-01-31T05:03:38.236193+0000","last_unstale":"2024-01-31T05:03:38.240721+0000","last_undegraded":"2024-01-31T05:03:38.240721+0000","last_fullsized":"2024-01-31T05:03:38.240721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218290+0000","last_change":"2024-01-31T05:03:38.218290+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218290+0000","last_undegraded":"2024-01-31T05:03:38.218290+0000","last_fullsized":"2024-01-31T05:03:38.218290+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.236414+0000","last_change":"2024-01-31T05:03:38.223052+0000","last_active":"2024-01-31T05:03:39.236414+0000","last_peered":"2024-01-31T05:03:39.236414+0000","last_clean":"2024-01-31T05:03:39.236414+0000","last_became_active":"2024-01-31T05:03:38.222742+0000","last_became_peered":"2024-01-31T05:03:38.222742+0000","last_unstale":"2024-01-31T05:03:39.236414+0000","last_undegraded":"2024-01-31T05:03:39.236414+0000","last_fullsized":"2024-01-31T05:03:39.236414+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267542+0000","last_change":"2024-01-31T05:03:39.252433+0000","last_active":"2024-01-31T05:03:39.267542+0000","last_peered":"2024-01-31T05:03:39.267542+0000","last_clean":"2024-01-31T05:03:39.267542+0000","last_became_active":"2024-01-31T05:03:39.249986+0000","last_became_peered":"2024-01-31T05:03:39.249986+0000","last_unstale":"2024-01-31T05:03:39.267542+0000","last_undegraded":"2024-01-31T05:03:39.267542+0000","last_fullsized":"2024-01-31T05:03:39.267542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:06.790339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240724+0000","last_change":"2024-01-31T05:03:38.237692+0000","last_active":"2024-01-31T05:03:38.240724+0000","last_peered":"2024-01-31T05:03:38.240724+0000","last_clean":"2024-01-31T05:03:38.240724+0000","last_became_active":"2024-01-31T05:03:38.236259+0000","last_became_peered":"2024-01-31T05:03:38.236259+0000","last_unstale":"2024-01-31T05:03:38.240724+0000","last_undegraded":"2024-01-31T05:03:38.240724+0000","last_fullsized":"2024-01-31T05:03:38.240724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:06:57.676616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219356+0000","last_change":"2024-01-31T05:03:38.219356+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219356+0000","last_undegraded":"2024-01-31T05:03:38.219356+0000","last_fullsized":"2024-01-31T05:03:38.219356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278537+0000","last_change":"2024-01-31T05:03:38.225581+0000","last_active":"2024-01-31T05:03:39.278537+0000","last_peered":"2024-01-31T05:03:39.278537+0000","last_clean":"2024-01-31T05:03:39.278537+0000","last_became_active":"2024-01-31T05:03:38.225059+0000","last_became_peered":"2024-01-31T05:03:38.225059+0000","last_unstale":"2024-01-31T05:03:39.278537+0000","last_undegraded":"2024-01-31T05:03:39.278537+0000","last_fullsized":"2024-01-31T05:03:39.278537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:03:20.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259585+0000","last_change":"2024-01-31T05:03:39.250182+0000","last_active":"2024-01-31T05:03:39.259585+0000","last_peered":"2024-01-31T05:03:39.259585+0000","last_clean":"2024-01-31T05:03:39.259585+0000","last_became_active":"2024-01-31T05:03:39.249917+0000","last_became_peered":"2024-01-31T05:03:39.249917+0000","last_unstale":"2024-01-31T05:03:39.259585+0000","last_undegraded":"2024-01-31T05:03:39.259585+0000","last_fullsized":"2024-01-31T05:03:39.259585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:42:20.611592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.235944+0000","last_change":"2024-01-31T05:03:38.230550+0000","last_active":"2024-01-31T05:03:38.235944+0000","last_peered":"2024-01-31T05:03:38.235944+0000","last_clean":"2024-01-31T05:03:38.235944+0000","last_became_active":"2024-01-31T05:03:38.230284+0000","last_became_peered":"2024-01-31T05:03:38.230284+0000","last_unstale":"2024-01-31T05:03:38.235944+0000","last_undegraded":"2024-01-31T05:03:38.235944+0000","last_fullsized":"2024-01-31T05:03:38.235944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.249515+0000","last_change":"2024-01-31T05:03:39.246221+0000","last_active":"2024-01-31T05:03:39.249515+0000","last_peered":"2024-01-31T05:03:39.249515+0000","last_clean":"2024-01-31T05:03:39.249515+0000","last_became_active":"2024-01-31T05:03:39.244574+0000","last_became_peered":"2024-01-31T05:03:39.244574+0000","last_unstale":"2024-01-31T05:03:39.249515+0000","last_undegraded":"2024-01-31T05:03:39.249515+0000","last_fullsized":"2024-01-31T05:03:39.249515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240559+0000","last_change":"2024-01-31T05:03:38.237263+0000","last_active":"2024-01-31T05:03:38.240559+0000","last_peered":"2024-01-31T05:03:38.240559+0000","last_clean":"2024-01-31T05:03:38.240559+0000","last_became_active":"2024-01-31T05:03:38.236868+0000","last_became_peered":"2024-01-31T05:03:38.236868+0000","last_unstale":"2024-01-31T05:03:38.240559+0000","last_undegraded":"2024-01-31T05:03:38.240559+0000","last_fullsized":"2024-01-31T05:03:38.240559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259275+0000","last_change":"2024-01-31T05:03:39.242937+0000","last_active":"2024-01-31T05:03:39.259275+0000","last_peered":"2024-01-31T05:03:39.259275+0000","last_clean":"2024-01-31T05:03:39.259275+0000","last_became_active":"2024-01-31T05:03:39.241602+0000","last_became_peered":"2024-01-31T05:03:39.241602+0000","last_unstale":"2024-01-31T05:03:39.259275+0000","last_undegraded":"2024-01-31T05:03:39.259275+0000","last_fullsized":"2024-01-31T05:03:39.259275+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233462+0000","last_change":"2024-01-31T05:03:38.225322+0000","last_active":"2024-01-31T05:03:39.233462+0000","last_peered":"2024-01-31T05:03:39.233462+0000","last_clean":"2024-01-31T05:03:39.233462+0000","last_became_active":"2024-01-31T05:03:38.224422+0000","last_became_peered":"2024-01-31T05:03:38.224422+0000","last_unstale":"2024-01-31T05:03:39.233462+0000","last_undegraded":"2024-01-31T05:03:39.233462+0000","last_fullsized":"2024-01-31T05:03:39.233462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:44.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269779+0000","last_change":"2024-01-31T05:03:39.258087+0000","last_active":"2024-01-31T05:03:39.269779+0000","last_peered":"2024-01-31T05:03:39.269779+0000","last_clean":"2024-01-31T05:03:39.269779+0000","last_became_active":"2024-01-31T05:03:39.257724+0000","last_became_peered":"2024-01-31T05:03:39.257724+0000","last_unstale":"2024-01-31T05:03:39.269779+0000","last_undegraded":"2024-01-31T05:03:39.269779+0000","last_fullsized":"2024-01-31T05:03:39.269779+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.231884+0000","last_change":"2024-01-31T05:03:38.219525+0000","last_active":"2024-01-31T05:03:39.231884+0000","last_peered":"2024-01-31T05:03:39.231884+0000","last_clean":"2024-01-31T05:03:39.231884+0000","last_became_active":"2024-01-31T05:03:38.219302+0000","last_became_peered":"2024-01-31T05:03:38.219302+0000","last_unstale":"2024-01-31T05:03:39.231884+0000","last_undegraded":"2024-01-31T05:03:39.231884+0000","last_fullsized":"2024-01-31T05:03:39.231884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:08.160317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260419+0000","last_change":"2024-01-31T05:03:39.253326+0000","last_active":"2024-01-31T05:03:39.260419+0000","last_peered":"2024-01-31T05:03:39.260419+0000","last_clean":"2024-01-31T05:03:39.260419+0000","last_became_active":"2024-01-31T05:03:39.252687+0000","last_became_peered":"2024-01-31T05:03:39.252687+0000","last_unstale":"2024-01-31T05:03:39.260419+0000","last_undegraded":"2024-01-31T05:03:39.260419+0000","last_fullsized":"2024-01-31T05:03:39.260419+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:23:14.519189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278703+0000","last_change":"2024-01-31T05:03:38.224819+0000","last_active":"2024-01-31T05:03:39.278703+0000","last_peered":"2024-01-31T05:03:39.278703+0000","last_clean":"2024-01-31T05:03:39.278703+0000","last_became_active":"2024-01-31T05:03:38.224555+0000","last_became_peered":"2024-01-31T05:03:38.224555+0000","last_unstale":"2024-01-31T05:03:39.278703+0000","last_undegraded":"2024-01-31T05:03:39.278703+0000","last_fullsized":"2024-01-31T05:03:39.278703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:15:12.301634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266298+0000","last_change":"2024-01-31T05:03:39.249020+0000","last_active":"2024-01-31T05:03:39.266298+0000","last_peered":"2024-01-31T05:03:39.266298+0000","last_clean":"2024-01-31T05:03:39.266298+0000","last_became_active":"2024-01-31T05:03:39.246977+0000","last_became_peered":"2024-01-31T05:03:39.246977+0000","last_unstale":"2024-01-31T05:03:39.266298+0000","last_undegraded":"2024-01-31T05:03:39.266298+0000","last_fullsized":"2024-01-31T05:03:39.266298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272054+0000","last_change":"2024-01-31T05:03:38.239609+0000","last_active":"2024-01-31T05:03:39.272054+0000","last_peered":"2024-01-31T05:03:39.272054+0000","last_clean":"2024-01-31T05:03:39.272054+0000","last_became_active":"2024-01-31T05:03:38.238339+0000","last_became_peered":"2024-01-31T05:03:38.238339+0000","last_unstale":"2024-01-31T05:03:39.272054+0000","last_undegraded":"2024-01-31T05:03:39.272054+0000","last_fullsized":"2024-01-31T05:03:39.272054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267510+0000","last_change":"2024-01-31T05:03:39.250705+0000","last_active":"2024-01-31T05:03:39.267510+0000","last_peered":"2024-01-31T05:03:39.267510+0000","last_clean":"2024-01-31T05:03:39.267510+0000","last_became_active":"2024-01-31T05:03:39.248093+0000","last_became_peered":"2024-01-31T05:03:39.248093+0000","last_unstale":"2024-01-31T05:03:39.267510+0000","last_undegraded":"2024-01-31T05:03:39.267510+0000","last_fullsized":"2024-01-31T05:03:39.267510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240743+0000","last_change":"2024-01-31T05:03:38.237901+0000","last_active":"2024-01-31T05:03:38.240743+0000","last_peered":"2024-01-31T05:03:38.240743+0000","last_clean":"2024-01-31T05:03:38.240743+0000","last_became_active":"2024-01-31T05:03:38.237413+0000","last_became_peered":"2024-01-31T05:03:38.237413+0000","last_unstale":"2024-01-31T05:03:38.240743+0000","last_undegraded":"2024-01-31T05:03:38.240743+0000","last_fullsized":"2024-01-31T05:03:38.240743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:58:23.675405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262915+0000","last_change":"2024-01-31T05:03:39.257158+0000","last_active":"2024-01-31T05:03:39.262915+0000","last_peered":"2024-01-31T05:03:39.262915+0000","last_clean":"2024-01-31T05:03:39.262915+0000","last_became_active":"2024-01-31T05:03:39.256870+0000","last_became_peered":"2024-01-31T05:03:39.256870+0000","last_unstale":"2024-01-31T05:03:39.262915+0000","last_undegraded":"2024-01-31T05:03:39.262915+0000","last_fullsized":"2024-01-31T05:03:39.262915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277453+0000","last_change":"2024-01-31T05:03:38.230887+0000","last_active":"2024-01-31T05:03:39.277453+0000","last_peered":"2024-01-31T05:03:39.277453+0000","last_clean":"2024-01-31T05:03:39.277453+0000","last_became_active":"2024-01-31T05:03:38.229073+0000","last_became_peered":"2024-01-31T05:03:38.229073+0000","last_unstale":"2024-01-31T05:03:39.277453+0000","last_undegraded":"2024-01-31T05:03:39.277453+0000","last_fullsized":"2024-01-31T05:03:39.277453+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259604+0000","last_change":"2024-01-31T05:03:39.254309+0000","last_active":"2024-01-31T05:03:39.259604+0000","last_peered":"2024-01-31T05:03:39.259604+0000","last_clean":"2024-01-31T05:03:39.259604+0000","last_became_active":"2024-01-31T05:03:39.254044+0000","last_became_peered":"2024-01-31T05:03:39.254044+0000","last_unstale":"2024-01-31T05:03:39.259604+0000","last_undegraded":"2024-01-31T05:03:39.259604+0000","last_fullsized":"2024-01-31T05:03:39.259604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:33:55.111654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240233+0000","last_change":"2024-01-31T05:03:38.234624+0000","last_active":"2024-01-31T05:03:38.240233+0000","last_peered":"2024-01-31T05:03:38.240233+0000","last_clean":"2024-01-31T05:03:38.240233+0000","last_became_active":"2024-01-31T05:03:38.234296+0000","last_became_peered":"2024-01-31T05:03:38.234296+0000","last_unstale":"2024-01-31T05:03:38.240233+0000","last_undegraded":"2024-01-31T05:03:38.240233+0000","last_fullsized":"2024-01-31T05:03:38.240233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269815+0000","last_change":"2024-01-31T05:03:39.260082+0000","last_active":"2024-01-31T05:03:39.269815+0000","last_peered":"2024-01-31T05:03:39.269815+0000","last_clean":"2024-01-31T05:03:39.269815+0000","last_became_active":"2024-01-31T05:03:39.256821+0000","last_became_peered":"2024-01-31T05:03:39.256821+0000","last_unstale":"2024-01-31T05:03:39.269815+0000","last_undegraded":"2024-01-31T05:03:39.269815+0000","last_fullsized":"2024-01-31T05:03:39.269815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.274034+0000","last_change":"2024-01-31T05:03:38.234981+0000","last_active":"2024-01-31T05:03:39.274034+0000","last_peered":"2024-01-31T05:03:39.274034+0000","last_clean":"2024-01-31T05:03:39.274034+0000","last_became_active":"2024-01-31T05:03:38.234678+0000","last_became_peered":"2024-01-31T05:03:38.234678+0000","last_unstale":"2024-01-31T05:03:39.274034+0000","last_undegraded":"2024-01-31T05:03:39.274034+0000","last_fullsized":"2024-01-31T05:03:39.274034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:08:18.852860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218772+0000","last_change":"2024-01-31T05:03:38.218772+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218772+0000","last_undegraded":"2024-01-31T05:03:38.218772+0000","last_fullsized":"2024-01-31T05:03:38.218772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238837+0000","last_change":"2024-01-31T05:03:38.230732+0000","last_active":"2024-01-31T05:03:38.238837+0000","last_peered":"2024-01-31T05:03:38.238837+0000","last_clean":"2024-01-31T05:03:38.238837+0000","last_became_active":"2024-01-31T05:03:38.229403+0000","last_became_peered":"2024-01-31T05:03:38.229403+0000","last_unstale":"2024-01-31T05:03:38.238837+0000","last_undegraded":"2024-01-31T05:03:38.238837+0000","last_fullsized":"2024-01-31T05:03:38.238837+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:00:00.029942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.216599+0000","last_change":"2024-01-31T05:03:38.216599+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.216599+0000","last_undegraded":"2024-01-31T05:03:38.216599+0000","last_fullsized":"2024-01-31T05:03:38.216599+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232247+0000","last_change":"2024-01-31T05:03:38.230238+0000","last_active":"2024-01-31T05:03:39.232247+0000","last_peered":"2024-01-31T05:03:39.232247+0000","last_clean":"2024-01-31T05:03:39.232247+0000","last_became_active":"2024-01-31T05:03:38.229961+0000","last_became_peered":"2024-01-31T05:03:38.229961+0000","last_unstale":"2024-01-31T05:03:39.232247+0000","last_undegraded":"2024-01-31T05:03:39.232247+0000","last_fullsized":"2024-01-31T05:03:39.232247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266259+0000","last_change":"2024-01-31T05:03:39.248707+0000","last_active":"2024-01-31T05:03:39.266259+0000","last_peered":"2024-01-31T05:03:39.266259+0000","last_clean":"2024-01-31T05:03:39.266259+0000","last_became_active":"2024-01-31T05:03:39.246545+0000","last_became_peered":"2024-01-31T05:03:39.246545+0000","last_unstale":"2024-01-31T05:03:39.266259+0000","last_undegraded":"2024-01-31T05:03:39.266259+0000","last_fullsized":"2024-01-31T05:03:39.266259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.271942+0000","last_change":"2024-01-31T05:03:38.232128+0000","last_active":"2024-01-31T05:03:39.271942+0000","last_peered":"2024-01-31T05:03:39.271942+0000","last_clean":"2024-01-31T05:03:39.271942+0000","last_became_active":"2024-01-31T05:03:38.231862+0000","last_became_peered":"2024-01-31T05:03:38.231862+0000","last_unstale":"2024-01-31T05:03:39.271942+0000","last_undegraded":"2024-01-31T05:03:39.271942+0000","last_fullsized":"2024-01-31T05:03:39.271942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267560+0000","last_change":"2024-01-31T05:03:39.252646+0000","last_active":"2024-01-31T05:03:39.267560+0000","last_peered":"2024-01-31T05:03:39.267560+0000","last_clean":"2024-01-31T05:03:39.267560+0000","last_became_active":"2024-01-31T05:03:39.244259+0000","last_became_peered":"2024-01-31T05:03:39.244259+0000","last_unstale":"2024-01-31T05:03:39.267560+0000","last_undegraded":"2024-01-31T05:03:39.267560+0000","last_fullsized":"2024-01-31T05:03:39.267560+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:01:30.708807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240725+0000","last_change":"2024-01-31T05:03:38.239097+0000","last_active":"2024-01-31T05:03:38.240725+0000","last_peered":"2024-01-31T05:03:38.240725+0000","last_clean":"2024-01-31T05:03:38.240725+0000","last_became_active":"2024-01-31T05:03:38.238757+0000","last_became_peered":"2024-01-31T05:03:38.238757+0000","last_unstale":"2024-01-31T05:03:38.240725+0000","last_undegraded":"2024-01-31T05:03:38.240725+0000","last_fullsized":"2024-01-31T05:03:38.240725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:18:04.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246623+0000","last_change":"2024-01-31T05:03:39.244177+0000","last_active":"2024-01-31T05:03:39.246623+0000","last_peered":"2024-01-31T05:03:39.246623+0000","last_clean":"2024-01-31T05:03:39.246623+0000","last_became_active":"2024-01-31T05:03:39.243841+0000","last_became_peered":"2024-01-31T05:03:39.243841+0000","last_unstale":"2024-01-31T05:03:39.246623+0000","last_undegraded":"2024-01-31T05:03:39.246623+0000","last_fullsized":"2024-01-31T05:03:39.246623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234411+0000","last_change":"2024-01-31T05:03:38.224802+0000","last_active":"2024-01-31T05:03:39.234411+0000","last_peered":"2024-01-31T05:03:39.234411+0000","last_clean":"2024-01-31T05:03:39.234411+0000","last_became_active":"2024-01-31T05:03:38.223989+0000","last_became_peered":"2024-01-31T05:03:38.223989+0000","last_unstale":"2024-01-31T05:03:39.234411+0000","last_undegraded":"2024-01-31T05:03:39.234411+0000","last_fullsized":"2024-01-31T05:03:39.234411+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219620+0000","last_change":"2024-01-31T05:03:38.219620+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219620+0000","last_undegraded":"2024-01-31T05:03:38.219620+0000","last_fullsized":"2024-01-31T05:03:38.219620+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.230405+0000","last_change":"2024-01-31T05:03:38.224866+0000","last_active":"2024-01-31T05:03:39.230405+0000","last_peered":"2024-01-31T05:03:39.230405+0000","last_clean":"2024-01-31T05:03:39.230405+0000","last_became_active":"2024-01-31T05:03:38.224437+0000","last_became_peered":"2024-01-31T05:03:38.224437+0000","last_unstale":"2024-01-31T05:03:39.230405+0000","last_undegraded":"2024-01-31T05:03:39.230405+0000","last_fullsized":"2024-01-31T05:03:39.230405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224396+0000","last_change":"2024-01-31T05:03:38.224396+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224396+0000","last_undegraded":"2024-01-31T05:03:38.224396+0000","last_fullsized":"2024-01-31T05:03:38.224396+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.225219+0000","last_change":"2024-01-31T05:03:38.236959+0000","last_active":"2024-01-31T05:03:39.225219+0000","last_peered":"2024-01-31T05:03:39.225219+0000","last_clean":"2024-01-31T05:03:39.225219+0000","last_became_active":"2024-01-31T05:03:38.235737+0000","last_became_peered":"2024-01-31T05:03:38.235737+0000","last_unstale":"2024-01-31T05:03:39.225219+0000","last_undegraded":"2024-01-31T05:03:39.225219+0000","last_fullsized":"2024-01-31T05:03:39.225219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:44.258646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.217657+0000","last_change":"2024-01-31T05:03:38.217657+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.217657+0000","last_undegraded":"2024-01-31T05:03:38.217657+0000","last_fullsized":"2024-01-31T05:03:38.217657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233182+0000","last_change":"2024-01-31T05:03:38.231189+0000","last_active":"2024-01-31T05:03:39.233182+0000","last_peered":"2024-01-31T05:03:39.233182+0000","last_clean":"2024-01-31T05:03:39.233182+0000","last_became_active":"2024-01-31T05:03:38.230998+0000","last_became_peered":"2024-01-31T05:03:38.230998+0000","last_unstale":"2024-01-31T05:03:39.233182+0000","last_undegraded":"2024-01-31T05:03:39.233182+0000","last_fullsized":"2024-01-31T05:03:39.233182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.213769+0000","last_change":"2024-01-31T05:03:38.213769+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.213769+0000","last_undegraded":"2024-01-31T05:03:38.213769+0000","last_fullsized":"2024-01-31T05:03:38.213769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238725+0000","last_change":"2024-01-31T05:03:38.235803+0000","last_active":"2024-01-31T05:03:38.238725+0000","last_peered":"2024-01-31T05:03:38.238725+0000","last_clean":"2024-01-31T05:03:38.238725+0000","last_became_active":"2024-01-31T05:03:38.234927+0000","last_became_peered":"2024-01-31T05:03:38.234927+0000","last_unstale":"2024-01-31T05:03:38.238725+0000","last_undegraded":"2024-01-31T05:03:38.238725+0000","last_fullsized":"2024-01-31T05:03:38.238725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:14.881712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246903+0000","last_change":"2024-01-31T05:03:39.243770+0000","last_active":"2024-01-31T05:03:39.246903+0000","last_peered":"2024-01-31T05:03:39.246903+0000","last_clean":"2024-01-31T05:03:39.246903+0000","last_became_active":"2024-01-31T05:03:39.243396+0000","last_became_peered":"2024-01-31T05:03:39.243396+0000","last_unstale":"2024-01-31T05:03:39.246903+0000","last_undegraded":"2024-01-31T05:03:39.246903+0000","last_fullsized":"2024-01-31T05:03:39.246903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240286+0000","last_change":"2024-01-31T05:03:38.235205+0000","last_active":"2024-01-31T05:03:38.240286+0000","last_peered":"2024-01-31T05:03:38.240286+0000","last_clean":"2024-01-31T05:03:38.240286+0000","last_became_active":"2024-01-31T05:03:38.234627+0000","last_became_peered":"2024-01-31T05:03:38.234627+0000","last_unstale":"2024-01-31T05:03:38.240286+0000","last_undegraded":"2024-01-31T05:03:38.240286+0000","last_fullsized":"2024-01-31T05:03:38.240286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.212184+0000","last_change":"2024-01-31T05:03:38.212184+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.212184+0000","last_undegraded":"2024-01-31T05:03:38.212184+0000","last_fullsized":"2024-01-31T05:03:38.212184+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.236723+0000","last_change":"2024-01-31T05:03:38.233879+0000","last_active":"2024-01-31T05:03:38.236723+0000","last_peered":"2024-01-31T05:03:38.236723+0000","last_clean":"2024-01-31T05:03:38.236723+0000","last_became_active":"2024-01-31T05:03:38.233495+0000","last_became_peered":"2024-01-31T05:03:38.233495+0000","last_unstale":"2024-01-31T05:03:38.236723+0000","last_undegraded":"2024-01-31T05:03:38.236723+0000","last_fullsized":"2024-01-31T05:03:38.236723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232380+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:39.232380+0000","last_peered":"2024-01-31T05:03:39.232380+0000","last_clean":"2024-01-31T05:03:39.232380+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:39.232380+0000","last_undegraded":"2024-01-31T05:03:39.232380+0000","last_fullsized":"2024-01-31T05:03:39.232380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":275170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0149999999999999,"15min":1.0009999999999999},"min":{"1min":0.68200000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5369999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.099,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6719999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.244,"15min":1.256},"min":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6259999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.872}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.175,"15min":1.1799999999999999},"min":{"1min":0.747,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4930000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.29,"15min":1.292},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.512,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.016}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2230000000000001,"15min":1.2150000000000001},"min":{"1min":0.91900000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.609,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6120000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.179}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":0.98499999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.609,"5min":2.25,"15min":2.25},"last":1.349},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3580000000000001,"15min":1.3740000000000001},"min":{"1min":1.004,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.573,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.472,"15min":1.4750000000000001},"min":{"1min":1.0109999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.762,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.85999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":1.0089999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.1240000000000001,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5529999999999999,"15min":1.5609999999999999},"min":{"1min":1.2709999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.726,"5min":2.629,"15min":2.629},"last":1.6359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":282881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0329999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3240000000000001,"5min":1.476,"15min":1.476},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.05,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.876}]},{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.78600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.587,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1200000000000001,"15min":1.131},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5369999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.48}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.6950000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.569},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1799999999999999,"15min":1.1839999999999999},"min":{"1min":0.80400000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.421,"5min":1.883,"15min":1.883},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.381,"15min":1.395},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.72,"5min":1.73,"15min":1.73},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.95599999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0830000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6259999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.878,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.611,"5min":2.032,"15min":2.032},"last":1.012},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.585},"min":{"1min":1.0780000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.984,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.165}]},{"osd":6,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5209999999999999,"15min":1.5329999999999999},"min":{"1min":0.96899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.972,"5min":2.335,"15min":2.335},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5860000000000001,"15min":1.6060000000000001},"min":{"1min":1.2609999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.768,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.115}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.599,"15min":1.6160000000000001},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8620000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.071},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.637,"15min":1.659},"min":{"1min":1.0920000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.891,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.7350000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":282355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.151},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1919999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":1.032,"15min":1.0840000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1439999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.121,"15min":1.1779999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4910000000000001,"5min":5.617,"15min":5.617},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.2330000000000001,"15min":1.2989999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.94599999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.292,"15min":1.3680000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.484,"15min":1.5720000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4930000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.6220000000000001,"15min":1.7230000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.9990000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.921},{"interface":"front","average":{"1min":1.141,"5min":1.621,"15min":1.7010000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.696,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.2130000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.639,"15min":1.7230000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9339999999999999,"5min":6.048,"15min":6.048},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.7430000000000001,"15min":1.831},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.952,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.804}]},{"osd":6,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.6739999999999999,"15min":1.744},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.024,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.657,"15min":1.748},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5700000000000001,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.6539999999999999,"15min":1.74},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.802,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.8380000000000001,"15min":1.946},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.73,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.5609999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":284037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.667,"5min":1.73,"15min":1.73},"last":1.4390000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.88900000000000001,"15min":0.90300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.472,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.046},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.60499999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.099}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93000000000000005,"15min":0.92300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.23},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0780000000000001,"15min":1.087},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":1.798,"15min":1.798},"last":1.387}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5780000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3220000000000001,"15min":1.331},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0059999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.321}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.232,"15min":1.2410000000000001},"min":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.585,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.472},{"interface":"front","average":{"1min":1.26,"5min":1.3100000000000001,"15min":1.3180000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8220000000000001,"5min":2.52,"15min":2.52},"last":1.494}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.419},"min":{"1min":1.0189999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1320000000000001,"5min":2.992,"15min":2.992},"last":1.663},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3939999999999999,"15min":1.4079999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.6879999999999999,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.397,"15min":1.403},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.8700000000000001,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.869,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.619}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1236992,"data_stored":873215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":1.026,"15min":1.0820000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.272,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.149,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4890000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.98399999999999999,"15min":1.0289999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0229999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.131,"15min":1.1839999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4099999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":1.0109999999999999,"15min":1.0449999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4419999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1639999999999999,"15min":1.222},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.294,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.028}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.4550000000000001,"15min":1.5089999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5940000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.345,"15min":1.379},"min":{"1min":0.80900000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4630000000000001,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.4059999999999999,"15min":1.4470000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.4650000000000001,"5min":2.722,"15min":2.722},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.409,"15min":1.4550000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.587,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.5189999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4790000000000001,"15min":1.538},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6439999999999999,"5min":2.577,"15min":2.577},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.415,"15min":1.4510000000000001},"min":{"1min":0.85699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5609999999999999,"5min":2.794,"15min":2.794},"last":1.651}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.4930000000000001,"15min":1.5449999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.647,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.5640000000000001,"15min":1.615},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5489999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.581}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.135,"15min":1.151},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.091},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.552,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1339999999999999,"15min":1.131},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5589999999999999,"5min":1.573,"15min":1.573},"last":1.242},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.73799999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5349999999999999,"5min":1.976,"15min":1.976},"last":1.1459999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7470000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.266,"15min":1.28},"min":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.544,"5min":1.873,"15min":1.873},"last":1.321}]},{"osd":3,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3580000000000001,"15min":1.375},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.7050000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.335,"15min":1.339},"min":{"1min":0.95899999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.4399999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.254,"15min":1.2529999999999999},"min":{"1min":0.79300000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6890000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":0.94399999999999995,"5min":0.628,"15min":0.628},"max":{"1min":2.0499999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4430000000000001,"15min":1.454},"min":{"1min":1.0049999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.679,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.379},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.593,"15min":1.6040000000000001},"min":{"1min":1.1990000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0990000000000002,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.462}]},{"osd":7,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5620000000000001,"15min":1.5820000000000001},"min":{"1min":0.89400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9530000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6719999999999999,"15min":1.6839999999999999},"min":{"1min":1.246,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9990000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.409}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":865538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.038,"15min":1.022},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.792}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1279999999999999,"15min":1.141},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.2030000000000001,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5720000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.3320000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.206,"15min":1.222},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.663,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.29},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.296,"15min":1.3089999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.647,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.49}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.304,"15min":1.306},"min":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.704,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.389,"15min":1.399},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.712,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.5309999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":0.70399999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.0139999999999998,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.91400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.792,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4259999999999999,"15min":1.4099999999999999},"min":{"1min":0.93500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":1.494},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.851,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5860000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.948,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.0029999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5219999999999998,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.6240000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0389999999999999,"15min":1.0489999999999999},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.0680000000000001,"5min":3.282,"15min":3.282},"last":0.627}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.075,"5min":1.774,"15min":1.774},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.091,"15min":1.089},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.2250000000000001},"min":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4370000000000001,"15min":1.4399999999999999},"min":{"1min":0.93999999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.8500000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.097}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.492,"15min":1.482},"min":{"1min":0.85399999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":5.75,"5min":5.75,"15min":5.75},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.468,"15min":1.4750000000000001},"min":{"1min":0.86499999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":4.8799999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.169}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.355,"15min":1.3300000000000001},"min":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":7.415,"5min":7.415,"15min":7.415},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.704,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":0.88900000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":7.5650000000000004,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.298}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.877,"5min":0.751,"15min":0.751},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:03:44.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:03:44.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.389+0000 7f6e84000640 1 Processor -- start 2024-01-31T05:03:44.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.389+0000 7f6e84000640 1 -- start start 2024-01-31T05:03:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.390+0000 7f6e84000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c15b400 0x7f6e7c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.390+0000 7f6e84000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e7c06db70 0x7f6e7c0ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.390+0000 7f6e84000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c06d1a0 con 0x7f6e7c0cb810 2024-01-31T05:03:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.390+0000 7f6e84000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0cc960 con 0x7f6e7c06db70 2024-01-31T05:03:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.390+0000 7f6e84000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c06c600 con 0x7f6e7c15b400 2024-01-31T05:03:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.390+0000 7f6e82000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e7c06db70 0x7f6e7c0ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e81600640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6e7c0cb810 0x7f6e7c0cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55392/0 (socket says 172.21.15.97:55392) 2024-01-31T05:03:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e81600640 1 -- 172.21.15.97:0/671195733 learned_addr learned my addr 172.21.15.97:0/671195733 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e7be00640 1 -- 172.21.15.97:0/671195733 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896587057 0 0) 0x7f6e7c06d1a0 con 0x7f6e7c0cb810 2024-01-31T05:03:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e80c00640 1 --2- 172.21.15.97:0/671195733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c15b400 0x7f6e7c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e7be00640 1 -- 172.21.15.97:0/671195733 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e5c003520 con 0x7f6e7c0cb810 2024-01-31T05:03:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e82000640 1 -- 172.21.15.97:0/671195733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c15b400 msgr2=0x7f6e7c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e82000640 1 --2- 172.21.15.97:0/671195733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c15b400 0x7f6e7c06d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e82000640 1 -- 172.21.15.97:0/671195733 >> v1:172.21.15.97:6789/0 conn(0x7f6e7c0cb810 legacy=0x7f6e7c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.391+0000 7f6e82000640 1 -- 172.21.15.97:0/671195733 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c06c870 con 0x7f6e7c06db70 2024-01-31T05:03:44.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.392+0000 7f6e82000640 1 --2- 172.21.15.97:0/671195733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c06db70 0x7f6e7c0ccd00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6e740097a0 tx=0x7f6e7402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fd1a333d9455dc server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:44.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e7be00640 1 -- 172.21.15.97:0/671195733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7402d040 con 0x7f6e7c06db70 2024-01-31T05:03:44.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e7be00640 1 -- 172.21.15.97:0/671195733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e74031810 con 0x7f6e7c06db70 2024-01-31T05:03:44.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e7be00640 1 -- 172.21.15.97:0/671195733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74031b80 con 0x7f6e7c06db70 2024-01-31T05:03:44.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 -- 172.21.15.97:0/671195733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c06db70 msgr2=0x7f6e7c0ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 --2- 172.21.15.97:0/671195733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c06db70 0x7f6e7c0ccd00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6e740097a0 tx=0x7f6e7402b8b0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 -- 172.21.15.97:0/671195733 shutdown_connections 2024-01-31T05:03:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 --2- 172.21.15.97:0/671195733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c06db70 0x7f6e7c0ccd00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 --2- 172.21.15.97:0/671195733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c15b400 0x7f6e7c06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 -- 172.21.15.97:0/671195733 >> 172.21.15.97:0/671195733 conn(0x7f6e7c0b0a10 msgr2=0x7f6e7c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:44.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 -- 172.21.15.97:0/671195733 shutdown_connections 2024-01-31T05:03:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.393+0000 7f6e84000640 1 -- 172.21.15.97:0/671195733 wait complete. 2024-01-31T05:03:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.395+0000 7f6e84000640 1 Processor -- start 2024-01-31T05:03:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.395+0000 7f6e84000640 1 -- start start 2024-01-31T05:03:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e84000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c06db70 0x7f6e7c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e84000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c0cb810 0x7f6e7c157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e84000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 0x7f6e7c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e84000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c09b000 con 0x7f6e7c15b400 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e84000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c157d50 con 0x7f6e7c06db70 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e84000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c157f80 con 0x7f6e7c0cb810 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e82000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 0x7f6e7c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e82000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 0x7f6e7c152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42348/0 (socket says 172.21.15.97:42348) 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.396+0000 7f6e82000640 1 -- 172.21.15.97:0/112110870 learned_addr learned my addr 172.21.15.97:0/112110870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:44.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e80c00640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c0cb810 0x7f6e7c157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e82000640 1 -- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c0cb810 msgr2=0x7f6e7c157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e82000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c0cb810 0x7f6e7c157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e82000640 1 -- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c06db70 msgr2=0x7f6e7c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e82000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c06db70 0x7f6e7c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e82000640 1 -- 172.21.15.97:0/112110870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e74009400 con 0x7f6e7c15b400 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e82000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 0x7f6e7c152500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6e740098c0 tx=0x7f6e740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:44.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74003820 con 0x7f6e7c15b400 2024-01-31T05:03:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7403f070 con 0x7f6e7c15b400 2024-01-31T05:03:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74003ba0 con 0x7f6e7c15b400 2024-01-31T05:03:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.397+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c158260 con 0x7f6e7c15b400 2024-01-31T05:03:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.398+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c152a40 con 0x7f6e7c15b400 2024-01-31T05:03:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.399+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c0677c0 con 0x7f6e7c15b400 2024-01-31T05:03:44.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.399+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6e74003d00 con 0x7f6e7c15b400 2024-01-31T05:03:44.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.400+0000 7f6e79600640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e5406aed0 0x7f6e5406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.400+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6e74041070 con 0x7f6e7c15b400 2024-01-31T05:03:44.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.400+0000 7f6e81600640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e5406aed0 0x7f6e5406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.401+0000 7f6e81600640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e5406aed0 0x7f6e5406d390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0012e0 tx=0x7f6e6c001650 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:44.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.404+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6e74076810 con 0x7f6e7c15b400 2024-01-31T05:03:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.596+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e7c158420 con 0x7f6e5406aed0 2024-01-31T05:03:44.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.608+0000 7f6e79600640 1 -- 172.21.15.97:0/112110870 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284616 (secure 0 0 0) 0x7f6e7c158420 con 0x7f6e5406aed0 2024-01-31T05:03:44.610 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:44.613 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e5406aed0 msgr2=0x7f6e5406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e5406aed0 0x7f6e5406d390 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0012e0 tx=0x7f6e6c001650 comp rx=0 tx=0).stop 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 msgr2=0x7f6e7c152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 0x7f6e7c152500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6e740098c0 tx=0x7f6e740046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 shutdown_connections 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6e5406aed0 0x7f6e5406d390 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e7c15b400 0x7f6e7c152500 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7c0cb810 0x7f6e7c157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 --2- 172.21.15.97:0/112110870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7c06db70 0x7f6e7c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.615+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 >> 172.21.15.97:0/112110870 conn(0x7f6e7c0b0a10 msgr2=0x7f6e7c0ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:44.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.616+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 shutdown_connections 2024-01-31T05:03:44.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.616+0000 7f6e84000640 1 -- 172.21.15.97:0/112110870 wait complete. 2024-01-31T05:03:44.633 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-01-31T05:03:43.610324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":188,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222300,"kb_used_data":6116,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752420,"statfs":{"total":773094113280,"available":772866478080,"internally_reserved":0,"allocated":6262784,"data_stored":3400346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001696"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231612+0000","last_change":"2024-01-31T05:03:38.231612+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231612+0000","last_undegraded":"2024-01-31T05:03:38.231612+0000","last_fullsized":"2024-01-31T05:03:38.231612+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.254203+0000","last_change":"2024-01-31T05:03:39.252312+0000","last_active":"2024-01-31T05:03:39.254203+0000","last_peered":"2024-01-31T05:03:39.254203+0000","last_clean":"2024-01-31T05:03:39.254203+0000","last_became_active":"2024-01-31T05:03:39.252042+0000","last_became_peered":"2024-01-31T05:03:39.252042+0000","last_unstale":"2024-01-31T05:03:39.254203+0000","last_undegraded":"2024-01-31T05:03:39.254203+0000","last_fullsized":"2024-01-31T05:03:39.254203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259236+0000","last_change":"2024-01-31T05:03:39.243310+0000","last_active":"2024-01-31T05:03:39.259236+0000","last_peered":"2024-01-31T05:03:39.259236+0000","last_clean":"2024-01-31T05:03:39.259236+0000","last_became_active":"2024-01-31T05:03:39.242309+0000","last_became_peered":"2024-01-31T05:03:39.242309+0000","last_unstale":"2024-01-31T05:03:39.259236+0000","last_undegraded":"2024-01-31T05:03:39.259236+0000","last_fullsized":"2024-01-31T05:03:39.259236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.261556+0000","last_change":"2024-01-31T05:03:39.254633+0000","last_active":"2024-01-31T05:03:39.261556+0000","last_peered":"2024-01-31T05:03:39.261556+0000","last_clean":"2024-01-31T05:03:39.261556+0000","last_became_active":"2024-01-31T05:03:39.254325+0000","last_became_peered":"2024-01-31T05:03:39.254325+0000","last_unstale":"2024-01-31T05:03:39.261556+0000","last_undegraded":"2024-01-31T05:03:39.261556+0000","last_fullsized":"2024-01-31T05:03:39.261556+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:03:40.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222392+0000","last_change":"2024-01-31T05:03:38.222392+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222392+0000","last_undegraded":"2024-01-31T05:03:38.222392+0000","last_fullsized":"2024-01-31T05:03:38.222392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226676+0000","last_change":"2024-01-31T05:03:38.226676+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226676+0000","last_undegraded":"2024-01-31T05:03:38.226676+0000","last_fullsized":"2024-01-31T05:03:38.226676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260398+0000","last_change":"2024-01-31T05:03:39.247885+0000","last_active":"2024-01-31T05:03:39.260398+0000","last_peered":"2024-01-31T05:03:39.260398+0000","last_clean":"2024-01-31T05:03:39.260398+0000","last_became_active":"2024-01-31T05:03:39.247556+0000","last_became_peered":"2024-01-31T05:03:39.247556+0000","last_unstale":"2024-01-31T05:03:39.260398+0000","last_undegraded":"2024-01-31T05:03:39.260398+0000","last_fullsized":"2024-01-31T05:03:39.260398+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:59.282060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.215147+0000","last_change":"2024-01-31T05:03:38.215147+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.215147+0000","last_undegraded":"2024-01-31T05:03:38.215147+0000","last_fullsized":"2024-01-31T05:03:38.215147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.270024+0000","last_change":"2024-01-31T05:03:39.256441+0000","last_active":"2024-01-31T05:03:39.270024+0000","last_peered":"2024-01-31T05:03:39.270024+0000","last_clean":"2024-01-31T05:03:39.270024+0000","last_became_active":"2024-01-31T05:03:39.256105+0000","last_became_peered":"2024-01-31T05:03:39.256105+0000","last_unstale":"2024-01-31T05:03:39.270024+0000","last_undegraded":"2024-01-31T05:03:39.270024+0000","last_fullsized":"2024-01-31T05:03:39.270024+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226447+0000","last_change":"2024-01-31T05:03:38.226447+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226447+0000","last_undegraded":"2024-01-31T05:03:38.226447+0000","last_fullsized":"2024-01-31T05:03:38.226447+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259313+0000","last_change":"2024-01-31T05:03:39.251020+0000","last_active":"2024-01-31T05:03:39.259313+0000","last_peered":"2024-01-31T05:03:39.259313+0000","last_clean":"2024-01-31T05:03:39.259313+0000","last_became_active":"2024-01-31T05:03:39.250684+0000","last_became_peered":"2024-01-31T05:03:39.250684+0000","last_unstale":"2024-01-31T05:03:39.259313+0000","last_undegraded":"2024-01-31T05:03:39.259313+0000","last_fullsized":"2024-01-31T05:03:39.259313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:39:06.883661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269332+0000","last_change":"2024-01-31T05:03:39.258617+0000","last_active":"2024-01-31T05:03:39.269332+0000","last_peered":"2024-01-31T05:03:39.269332+0000","last_clean":"2024-01-31T05:03:39.269332+0000","last_became_active":"2024-01-31T05:03:39.258317+0000","last_became_peered":"2024-01-31T05:03:39.258317+0000","last_unstale":"2024-01-31T05:03:39.269332+0000","last_undegraded":"2024-01-31T05:03:39.269332+0000","last_fullsized":"2024-01-31T05:03:39.269332+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218606+0000","last_change":"2024-01-31T05:03:38.218606+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218606+0000","last_undegraded":"2024-01-31T05:03:38.218606+0000","last_fullsized":"2024-01-31T05:03:38.218606+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267681+0000","last_change":"2024-01-31T05:03:39.251453+0000","last_active":"2024-01-31T05:03:39.267681+0000","last_peered":"2024-01-31T05:03:39.267681+0000","last_clean":"2024-01-31T05:03:39.267681+0000","last_became_active":"2024-01-31T05:03:39.248059+0000","last_became_peered":"2024-01-31T05:03:39.248059+0000","last_unstale":"2024-01-31T05:03:39.267681+0000","last_undegraded":"2024-01-31T05:03:39.267681+0000","last_fullsized":"2024-01-31T05:03:39.267681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.908110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222879+0000","last_change":"2024-01-31T05:03:38.222879+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222879+0000","last_undegraded":"2024-01-31T05:03:38.222879+0000","last_fullsized":"2024-01-31T05:03:38.222879+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220373+0000","last_change":"2024-01-31T05:03:38.220373+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220373+0000","last_undegraded":"2024-01-31T05:03:38.220373+0000","last_fullsized":"2024-01-31T05:03:38.220373+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231217+0000","last_change":"2024-01-31T05:03:38.231217+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231217+0000","last_undegraded":"2024-01-31T05:03:38.231217+0000","last_fullsized":"2024-01-31T05:03:38.231217+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266404+0000","last_change":"2024-01-31T05:03:39.249229+0000","last_active":"2024-01-31T05:03:39.266404+0000","last_peered":"2024-01-31T05:03:39.266404+0000","last_clean":"2024-01-31T05:03:39.266404+0000","last_became_active":"2024-01-31T05:03:39.248929+0000","last_became_peered":"2024-01-31T05:03:39.248929+0000","last_unstale":"2024-01-31T05:03:39.266404+0000","last_undegraded":"2024-01-31T05:03:39.266404+0000","last_fullsized":"2024-01-31T05:03:39.266404+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259739+0000","last_change":"2024-01-31T05:03:39.255701+0000","last_active":"2024-01-31T05:03:39.259739+0000","last_peered":"2024-01-31T05:03:39.259739+0000","last_clean":"2024-01-31T05:03:39.259739+0000","last_became_active":"2024-01-31T05:03:39.253975+0000","last_became_peered":"2024-01-31T05:03:39.253975+0000","last_unstale":"2024-01-31T05:03:39.259739+0000","last_undegraded":"2024-01-31T05:03:39.259739+0000","last_fullsized":"2024-01-31T05:03:39.259739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.263250+0000","last_change":"2024-01-31T05:03:39.258999+0000","last_active":"2024-01-31T05:03:39.263250+0000","last_peered":"2024-01-31T05:03:39.263250+0000","last_clean":"2024-01-31T05:03:39.263250+0000","last_became_active":"2024-01-31T05:03:39.257976+0000","last_became_peered":"2024-01-31T05:03:39.257976+0000","last_unstale":"2024-01-31T05:03:39.263250+0000","last_undegraded":"2024-01-31T05:03:39.263250+0000","last_fullsized":"2024-01-31T05:03:39.263250+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:24:32.083418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218199+0000","last_change":"2024-01-31T05:03:38.218199+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218199+0000","last_undegraded":"2024-01-31T05:03:38.218199+0000","last_fullsized":"2024-01-31T05:03:38.218199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.250048+0000","last_change":"2024-01-31T05:03:39.242603+0000","last_active":"2024-01-31T05:03:39.250048+0000","last_peered":"2024-01-31T05:03:39.250048+0000","last_clean":"2024-01-31T05:03:39.250048+0000","last_became_active":"2024-01-31T05:03:39.241443+0000","last_became_peered":"2024-01-31T05:03:39.241443+0000","last_unstale":"2024-01-31T05:03:39.250048+0000","last_undegraded":"2024-01-31T05:03:39.250048+0000","last_fullsized":"2024-01-31T05:03:39.250048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:28:05.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260437+0000","last_change":"2024-01-31T05:03:39.253595+0000","last_active":"2024-01-31T05:03:39.260437+0000","last_peered":"2024-01-31T05:03:39.260437+0000","last_clean":"2024-01-31T05:03:39.260437+0000","last_became_active":"2024-01-31T05:03:39.253076+0000","last_became_peered":"2024-01-31T05:03:39.253076+0000","last_unstale":"2024-01-31T05:03:39.260437+0000","last_undegraded":"2024-01-31T05:03:39.260437+0000","last_fullsized":"2024-01-31T05:03:39.260437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:57:48.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272382+0000","last_change":"2024-01-31T05:03:39.260142+0000","last_active":"2024-01-31T05:03:39.272382+0000","last_peered":"2024-01-31T05:03:39.272382+0000","last_clean":"2024-01-31T05:03:39.272382+0000","last_became_active":"2024-01-31T05:03:39.255918+0000","last_became_peered":"2024-01-31T05:03:39.255918+0000","last_unstale":"2024-01-31T05:03:39.272382+0000","last_undegraded":"2024-01-31T05:03:39.272382+0000","last_fullsized":"2024-01-31T05:03:39.272382+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.257215+0000","last_change":"2024-01-31T05:03:39.253163+0000","last_active":"2024-01-31T05:03:39.257215+0000","last_peered":"2024-01-31T05:03:39.257215+0000","last_clean":"2024-01-31T05:03:39.257215+0000","last_became_active":"2024-01-31T05:03:39.252895+0000","last_became_peered":"2024-01-31T05:03:39.252895+0000","last_unstale":"2024-01-31T05:03:39.257215+0000","last_undegraded":"2024-01-31T05:03:39.257215+0000","last_fullsized":"2024-01-31T05:03:39.257215+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218748+0000","last_change":"2024-01-31T05:03:38.218748+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218748+0000","last_undegraded":"2024-01-31T05:03:38.218748+0000","last_fullsized":"2024-01-31T05:03:38.218748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.252464+0000","last_change":"2024-01-31T05:03:39.245654+0000","last_active":"2024-01-31T05:03:39.252464+0000","last_peered":"2024-01-31T05:03:39.252464+0000","last_clean":"2024-01-31T05:03:39.252464+0000","last_became_active":"2024-01-31T05:03:39.242973+0000","last_became_peered":"2024-01-31T05:03:39.242973+0000","last_unstale":"2024-01-31T05:03:39.252464+0000","last_undegraded":"2024-01-31T05:03:39.252464+0000","last_fullsized":"2024-01-31T05:03:39.252464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.227220+0000","last_change":"2024-01-31T05:03:38.227220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.227220+0000","last_undegraded":"2024-01-31T05:03:38.227220+0000","last_fullsized":"2024-01-31T05:03:38.227220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219977+0000","last_change":"2024-01-31T05:03:38.219977+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219977+0000","last_undegraded":"2024-01-31T05:03:38.219977+0000","last_fullsized":"2024-01-31T05:03:38.219977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262925+0000","last_change":"2024-01-31T05:03:39.258792+0000","last_active":"2024-01-31T05:03:39.262925+0000","last_peered":"2024-01-31T05:03:39.262925+0000","last_clean":"2024-01-31T05:03:39.262925+0000","last_became_active":"2024-01-31T05:03:39.258531+0000","last_became_peered":"2024-01-31T05:03:39.258531+0000","last_unstale":"2024-01-31T05:03:39.262925+0000","last_undegraded":"2024-01-31T05:03:39.262925+0000","last_fullsized":"2024-01-31T05:03:39.262925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:02:30.641509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243207+0000","last_change":"2024-01-31T05:03:39.236368+0000","last_active":"2024-01-31T05:03:39.243207+0000","last_peered":"2024-01-31T05:03:39.243207+0000","last_clean":"2024-01-31T05:03:39.243207+0000","last_became_active":"2024-01-31T05:03:39.236055+0000","last_became_peered":"2024-01-31T05:03:39.236055+0000","last_unstale":"2024-01-31T05:03:39.243207+0000","last_undegraded":"2024-01-31T05:03:39.243207+0000","last_fullsized":"2024-01-31T05:03:39.243207+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.901754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.241385+0000","last_change":"2024-01-31T05:03:39.239349+0000","last_active":"2024-01-31T05:03:39.241385+0000","last_peered":"2024-01-31T05:03:39.241385+0000","last_clean":"2024-01-31T05:03:39.241385+0000","last_became_active":"2024-01-31T05:03:39.238868+0000","last_became_peered":"2024-01-31T05:03:39.238868+0000","last_unstale":"2024-01-31T05:03:39.241385+0000","last_undegraded":"2024-01-31T05:03:39.241385+0000","last_fullsized":"2024-01-31T05:03:39.241385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:55:11.110405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.225923+0000","last_change":"2024-01-31T05:03:38.225923+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.225923+0000","last_undegraded":"2024-01-31T05:03:38.225923+0000","last_fullsized":"2024-01-31T05:03:38.225923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.276730+0000","last_change":"2024-01-31T05:03:38.235515+0000","last_active":"2024-01-31T05:03:39.276730+0000","last_peered":"2024-01-31T05:03:39.276730+0000","last_clean":"2024-01-31T05:03:39.276730+0000","last_became_active":"2024-01-31T05:03:38.235228+0000","last_became_peered":"2024-01-31T05:03:38.235228+0000","last_unstale":"2024-01-31T05:03:39.276730+0000","last_undegraded":"2024-01-31T05:03:39.276730+0000","last_fullsized":"2024-01-31T05:03:39.276730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:42.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.230764+0000","last_change":"2024-01-31T05:03:38.230764+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.230764+0000","last_undegraded":"2024-01-31T05:03:38.230764+0000","last_fullsized":"2024-01-31T05:03:38.230764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269887+0000","last_change":"2024-01-31T05:03:38.224888+0000","last_active":"2024-01-31T05:03:39.269887+0000","last_peered":"2024-01-31T05:03:39.269887+0000","last_clean":"2024-01-31T05:03:39.269887+0000","last_became_active":"2024-01-31T05:03:38.223355+0000","last_became_peered":"2024-01-31T05:03:38.223355+0000","last_unstale":"2024-01-31T05:03:39.269887+0000","last_undegraded":"2024-01-31T05:03:39.269887+0000","last_fullsized":"2024-01-31T05:03:39.269887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226684+0000","last_change":"2024-01-31T05:03:38.226684+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226684+0000","last_undegraded":"2024-01-31T05:03:38.226684+0000","last_fullsized":"2024-01-31T05:03:38.226684+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.239333+0000","last_change":"2024-01-31T05:03:38.234777+0000","last_active":"2024-01-31T05:03:38.239333+0000","last_peered":"2024-01-31T05:03:38.239333+0000","last_clean":"2024-01-31T05:03:38.239333+0000","last_became_active":"2024-01-31T05:03:38.234493+0000","last_became_peered":"2024-01-31T05:03:38.234493+0000","last_unstale":"2024-01-31T05:03:38.239333+0000","last_undegraded":"2024-01-31T05:03:38.239333+0000","last_fullsized":"2024-01-31T05:03:38.239333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:48:31.432282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224220+0000","last_change":"2024-01-31T05:03:38.224220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224220+0000","last_undegraded":"2024-01-31T05:03:38.224220+0000","last_fullsized":"2024-01-31T05:03:38.224220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277804+0000","last_change":"2024-01-31T05:03:38.221765+0000","last_active":"2024-01-31T05:03:39.277804+0000","last_peered":"2024-01-31T05:03:39.277804+0000","last_clean":"2024-01-31T05:03:39.277804+0000","last_became_active":"2024-01-31T05:03:38.221534+0000","last_became_peered":"2024-01-31T05:03:38.221534+0000","last_unstale":"2024-01-31T05:03:39.277804+0000","last_undegraded":"2024-01-31T05:03:39.277804+0000","last_fullsized":"2024-01-31T05:03:39.277804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:22.374866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.210993+0000","last_change":"2024-01-31T05:03:38.210993+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.210993+0000","last_undegraded":"2024-01-31T05:03:38.210993+0000","last_fullsized":"2024-01-31T05:03:38.210993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277781+0000","last_change":"2024-01-31T05:03:38.226696+0000","last_active":"2024-01-31T05:03:39.277781+0000","last_peered":"2024-01-31T05:03:39.277781+0000","last_clean":"2024-01-31T05:03:39.277781+0000","last_became_active":"2024-01-31T05:03:38.226440+0000","last_became_peered":"2024-01-31T05:03:38.226440+0000","last_unstale":"2024-01-31T05:03:39.277781+0000","last_undegraded":"2024-01-31T05:03:39.277781+0000","last_fullsized":"2024-01-31T05:03:39.277781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243539+0000","last_change":"2024-01-31T05:03:39.240337+0000","last_active":"2024-01-31T05:03:39.243539+0000","last_peered":"2024-01-31T05:03:39.243539+0000","last_clean":"2024-01-31T05:03:39.243539+0000","last_became_active":"2024-01-31T05:03:39.240029+0000","last_became_peered":"2024-01-31T05:03:39.240029+0000","last_unstale":"2024-01-31T05:03:39.243539+0000","last_undegraded":"2024-01-31T05:03:39.243539+0000","last_fullsized":"2024-01-31T05:03:39.243539+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:24:32.372349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240709+0000","last_change":"2024-01-31T05:03:38.239050+0000","last_active":"2024-01-31T05:03:38.240709+0000","last_peered":"2024-01-31T05:03:38.240709+0000","last_clean":"2024-01-31T05:03:38.240709+0000","last_became_active":"2024-01-31T05:03:38.238746+0000","last_became_peered":"2024-01-31T05:03:38.238746+0000","last_unstale":"2024-01-31T05:03:38.240709+0000","last_undegraded":"2024-01-31T05:03:38.240709+0000","last_fullsized":"2024-01-31T05:03:38.240709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:37:59.345660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260998+0000","last_change":"2024-01-31T05:03:39.244622+0000","last_active":"2024-01-31T05:03:39.260998+0000","last_peered":"2024-01-31T05:03:39.260998+0000","last_clean":"2024-01-31T05:03:39.260998+0000","last_became_active":"2024-01-31T05:03:39.243824+0000","last_became_peered":"2024-01-31T05:03:39.243824+0000","last_unstale":"2024-01-31T05:03:39.260998+0000","last_undegraded":"2024-01-31T05:03:39.260998+0000","last_fullsized":"2024-01-31T05:03:39.260998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234692+0000","last_change":"2024-01-31T05:03:38.230982+0000","last_active":"2024-01-31T05:03:39.234692+0000","last_peered":"2024-01-31T05:03:39.234692+0000","last_clean":"2024-01-31T05:03:39.234692+0000","last_became_active":"2024-01-31T05:03:38.230687+0000","last_became_peered":"2024-01-31T05:03:38.230687+0000","last_unstale":"2024-01-31T05:03:39.234692+0000","last_undegraded":"2024-01-31T05:03:39.234692+0000","last_fullsized":"2024-01-31T05:03:39.234692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220574+0000","last_change":"2024-01-31T05:03:38.220574+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220574+0000","last_undegraded":"2024-01-31T05:03:38.220574+0000","last_fullsized":"2024-01-31T05:03:38.220574+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240721+0000","last_change":"2024-01-31T05:03:38.236445+0000","last_active":"2024-01-31T05:03:38.240721+0000","last_peered":"2024-01-31T05:03:38.240721+0000","last_clean":"2024-01-31T05:03:38.240721+0000","last_became_active":"2024-01-31T05:03:38.236193+0000","last_became_peered":"2024-01-31T05:03:38.236193+0000","last_unstale":"2024-01-31T05:03:38.240721+0000","last_undegraded":"2024-01-31T05:03:38.240721+0000","last_fullsized":"2024-01-31T05:03:38.240721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218290+0000","last_change":"2024-01-31T05:03:38.218290+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218290+0000","last_undegraded":"2024-01-31T05:03:38.218290+0000","last_fullsized":"2024-01-31T05:03:38.218290+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.236414+0000","last_change":"2024-01-31T05:03:38.223052+0000","last_active":"2024-01-31T05:03:39.236414+0000","last_peered":"2024-01-31T05:03:39.236414+0000","last_clean":"2024-01-31T05:03:39.236414+0000","last_became_active":"2024-01-31T05:03:38.222742+0000","last_became_peered":"2024-01-31T05:03:38.222742+0000","last_unstale":"2024-01-31T05:03:39.236414+0000","last_undegraded":"2024-01-31T05:03:39.236414+0000","last_fullsized":"2024-01-31T05:03:39.236414+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267542+0000","last_change":"2024-01-31T05:03:39.252433+0000","last_active":"2024-01-31T05:03:39.267542+0000","last_peered":"2024-01-31T05:03:39.267542+0000","last_clean":"2024-01-31T05:03:39.267542+0000","last_became_active":"2024-01-31T05:03:39.249986+0000","last_became_peered":"2024-01-31T05:03:39.249986+0000","last_unstale":"2024-01-31T05:03:39.267542+0000","last_undegraded":"2024-01-31T05:03:39.267542+0000","last_fullsized":"2024-01-31T05:03:39.267542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:06.790339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240724+0000","last_change":"2024-01-31T05:03:38.237692+0000","last_active":"2024-01-31T05:03:38.240724+0000","last_peered":"2024-01-31T05:03:38.240724+0000","last_clean":"2024-01-31T05:03:38.240724+0000","last_became_active":"2024-01-31T05:03:38.236259+0000","last_became_peered":"2024-01-31T05:03:38.236259+0000","last_unstale":"2024-01-31T05:03:38.240724+0000","last_undegraded":"2024-01-31T05:03:38.240724+0000","last_fullsized":"2024-01-31T05:03:38.240724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:06:57.676616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219356+0000","last_change":"2024-01-31T05:03:38.219356+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219356+0000","last_undegraded":"2024-01-31T05:03:38.219356+0000","last_fullsized":"2024-01-31T05:03:38.219356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278537+0000","last_change":"2024-01-31T05:03:38.225581+0000","last_active":"2024-01-31T05:03:39.278537+0000","last_peered":"2024-01-31T05:03:39.278537+0000","last_clean":"2024-01-31T05:03:39.278537+0000","last_became_active":"2024-01-31T05:03:38.225059+0000","last_became_peered":"2024-01-31T05:03:38.225059+0000","last_unstale":"2024-01-31T05:03:39.278537+0000","last_undegraded":"2024-01-31T05:03:39.278537+0000","last_fullsized":"2024-01-31T05:03:39.278537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:03:20.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259585+0000","last_change":"2024-01-31T05:03:39.250182+0000","last_active":"2024-01-31T05:03:39.259585+0000","last_peered":"2024-01-31T05:03:39.259585+0000","last_clean":"2024-01-31T05:03:39.259585+0000","last_became_active":"2024-01-31T05:03:39.249917+0000","last_became_peered":"2024-01-31T05:03:39.249917+0000","last_unstale":"2024-01-31T05:03:39.259585+0000","last_undegraded":"2024-01-31T05:03:39.259585+0000","last_fullsized":"2024-01-31T05:03:39.259585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:42:20.611592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.235944+0000","last_change":"2024-01-31T05:03:38.230550+0000","last_active":"2024-01-31T05:03:38.235944+0000","last_peered":"2024-01-31T05:03:38.235944+0000","last_clean":"2024-01-31T05:03:38.235944+0000","last_became_active":"2024-01-31T05:03:38.230284+0000","last_became_peered":"2024-01-31T05:03:38.230284+0000","last_unstale":"2024-01-31T05:03:38.235944+0000","last_undegraded":"2024-01-31T05:03:38.235944+0000","last_fullsized":"2024-01-31T05:03:38.235944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.249515+0000","last_change":"2024-01-31T05:03:39.246221+0000","last_active":"2024-01-31T05:03:39.249515+0000","last_peered":"2024-01-31T05:03:39.249515+0000","last_clean":"2024-01-31T05:03:39.249515+0000","last_became_active":"2024-01-31T05:03:39.244574+0000","last_became_peered":"2024-01-31T05:03:39.244574+0000","last_unstale":"2024-01-31T05:03:39.249515+0000","last_undegraded":"2024-01-31T05:03:39.249515+0000","last_fullsized":"2024-01-31T05:03:39.249515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240559+0000","last_change":"2024-01-31T05:03:38.237263+0000","last_active":"2024-01-31T05:03:38.240559+0000","last_peered":"2024-01-31T05:03:38.240559+0000","last_clean":"2024-01-31T05:03:38.240559+0000","last_became_active":"2024-01-31T05:03:38.236868+0000","last_became_peered":"2024-01-31T05:03:38.236868+0000","last_unstale":"2024-01-31T05:03:38.240559+0000","last_undegraded":"2024-01-31T05:03:38.240559+0000","last_fullsized":"2024-01-31T05:03:38.240559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259275+0000","last_change":"2024-01-31T05:03:39.242937+0000","last_active":"2024-01-31T05:03:39.259275+0000","last_peered":"2024-01-31T05:03:39.259275+0000","last_clean":"2024-01-31T05:03:39.259275+0000","last_became_active":"2024-01-31T05:03:39.241602+0000","last_became_peered":"2024-01-31T05:03:39.241602+0000","last_unstale":"2024-01-31T05:03:39.259275+0000","last_undegraded":"2024-01-31T05:03:39.259275+0000","last_fullsized":"2024-01-31T05:03:39.259275+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233462+0000","last_change":"2024-01-31T05:03:38.225322+0000","last_active":"2024-01-31T05:03:39.233462+0000","last_peered":"2024-01-31T05:03:39.233462+0000","last_clean":"2024-01-31T05:03:39.233462+0000","last_became_active":"2024-01-31T05:03:38.224422+0000","last_became_peered":"2024-01-31T05:03:38.224422+0000","last_unstale":"2024-01-31T05:03:39.233462+0000","last_undegraded":"2024-01-31T05:03:39.233462+0000","last_fullsized":"2024-01-31T05:03:39.233462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:44.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269779+0000","last_change":"2024-01-31T05:03:39.258087+0000","last_active":"2024-01-31T05:03:39.269779+0000","last_peered":"2024-01-31T05:03:39.269779+0000","last_clean":"2024-01-31T05:03:39.269779+0000","last_became_active":"2024-01-31T05:03:39.257724+0000","last_became_peered":"2024-01-31T05:03:39.257724+0000","last_unstale":"2024-01-31T05:03:39.269779+0000","last_undegraded":"2024-01-31T05:03:39.269779+0000","last_fullsized":"2024-01-31T05:03:39.269779+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.231884+0000","last_change":"2024-01-31T05:03:38.219525+0000","last_active":"2024-01-31T05:03:39.231884+0000","last_peered":"2024-01-31T05:03:39.231884+0000","last_clean":"2024-01-31T05:03:39.231884+0000","last_became_active":"2024-01-31T05:03:38.219302+0000","last_became_peered":"2024-01-31T05:03:38.219302+0000","last_unstale":"2024-01-31T05:03:39.231884+0000","last_undegraded":"2024-01-31T05:03:39.231884+0000","last_fullsized":"2024-01-31T05:03:39.231884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:08.160317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260419+0000","last_change":"2024-01-31T05:03:39.253326+0000","last_active":"2024-01-31T05:03:39.260419+0000","last_peered":"2024-01-31T05:03:39.260419+0000","last_clean":"2024-01-31T05:03:39.260419+0000","last_became_active":"2024-01-31T05:03:39.252687+0000","last_became_peered":"2024-01-31T05:03:39.252687+0000","last_unstale":"2024-01-31T05:03:39.260419+0000","last_undegraded":"2024-01-31T05:03:39.260419+0000","last_fullsized":"2024-01-31T05:03:39.260419+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:23:14.519189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278703+0000","last_change":"2024-01-31T05:03:38.224819+0000","last_active":"2024-01-31T05:03:39.278703+0000","last_peered":"2024-01-31T05:03:39.278703+0000","last_clean":"2024-01-31T05:03:39.278703+0000","last_became_active":"2024-01-31T05:03:38.224555+0000","last_became_peered":"2024-01-31T05:03:38.224555+0000","last_unstale":"2024-01-31T05:03:39.278703+0000","last_undegraded":"2024-01-31T05:03:39.278703+0000","last_fullsized":"2024-01-31T05:03:39.278703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:15:12.301634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266298+0000","last_change":"2024-01-31T05:03:39.249020+0000","last_active":"2024-01-31T05:03:39.266298+0000","last_peered":"2024-01-31T05:03:39.266298+0000","last_clean":"2024-01-31T05:03:39.266298+0000","last_became_active":"2024-01-31T05:03:39.246977+0000","last_became_peered":"2024-01-31T05:03:39.246977+0000","last_unstale":"2024-01-31T05:03:39.266298+0000","last_undegraded":"2024-01-31T05:03:39.266298+0000","last_fullsized":"2024-01-31T05:03:39.266298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272054+0000","last_change":"2024-01-31T05:03:38.239609+0000","last_active":"2024-01-31T05:03:39.272054+0000","last_peered":"2024-01-31T05:03:39.272054+0000","last_clean":"2024-01-31T05:03:39.272054+0000","last_became_active":"2024-01-31T05:03:38.238339+0000","last_became_peered":"2024-01-31T05:03:38.238339+0000","last_unstale":"2024-01-31T05:03:39.272054+0000","last_undegraded":"2024-01-31T05:03:39.272054+0000","last_fullsized":"2024-01-31T05:03:39.272054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267510+0000","last_change":"2024-01-31T05:03:39.250705+0000","last_active":"2024-01-31T05:03:39.267510+0000","last_peered":"2024-01-31T05:03:39.267510+0000","last_clean":"2024-01-31T05:03:39.267510+0000","last_became_active":"2024-01-31T05:03:39.248093+0000","last_became_peered":"2024-01-31T05:03:39.248093+0000","last_unstale":"2024-01-31T05:03:39.267510+0000","last_undegraded":"2024-01-31T05:03:39.267510+0000","last_fullsized":"2024-01-31T05:03:39.267510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240743+0000","last_change":"2024-01-31T05:03:38.237901+0000","last_active":"2024-01-31T05:03:38.240743+0000","last_peered":"2024-01-31T05:03:38.240743+0000","last_clean":"2024-01-31T05:03:38.240743+0000","last_became_active":"2024-01-31T05:03:38.237413+0000","last_became_peered":"2024-01-31T05:03:38.237413+0000","last_unstale":"2024-01-31T05:03:38.240743+0000","last_undegraded":"2024-01-31T05:03:38.240743+0000","last_fullsized":"2024-01-31T05:03:38.240743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:58:23.675405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262915+0000","last_change":"2024-01-31T05:03:39.257158+0000","last_active":"2024-01-31T05:03:39.262915+0000","last_peered":"2024-01-31T05:03:39.262915+0000","last_clean":"2024-01-31T05:03:39.262915+0000","last_became_active":"2024-01-31T05:03:39.256870+0000","last_became_peered":"2024-01-31T05:03:39.256870+0000","last_unstale":"2024-01-31T05:03:39.262915+0000","last_undegraded":"2024-01-31T05:03:39.262915+0000","last_fullsized":"2024-01-31T05:03:39.262915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277453+0000","last_change":"2024-01-31T05:03:38.230887+0000","last_active":"2024-01-31T05:03:39.277453+0000","last_peered":"2024-01-31T05:03:39.277453+0000","last_clean":"2024-01-31T05:03:39.277453+0000","last_became_active":"2024-01-31T05:03:38.229073+0000","last_became_peered":"2024-01-31T05:03:38.229073+0000","last_unstale":"2024-01-31T05:03:39.277453+0000","last_undegraded":"2024-01-31T05:03:39.277453+0000","last_fullsized":"2024-01-31T05:03:39.277453+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259604+0000","last_change":"2024-01-31T05:03:39.254309+0000","last_active":"2024-01-31T05:03:39.259604+0000","last_peered":"2024-01-31T05:03:39.259604+0000","last_clean":"2024-01-31T05:03:39.259604+0000","last_became_active":"2024-01-31T05:03:39.254044+0000","last_became_peered":"2024-01-31T05:03:39.254044+0000","last_unstale":"2024-01-31T05:03:39.259604+0000","last_undegraded":"2024-01-31T05:03:39.259604+0000","last_fullsized":"2024-01-31T05:03:39.259604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:33:55.111654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240233+0000","last_change":"2024-01-31T05:03:38.234624+0000","last_active":"2024-01-31T05:03:38.240233+0000","last_peered":"2024-01-31T05:03:38.240233+0000","last_clean":"2024-01-31T05:03:38.240233+0000","last_became_active":"2024-01-31T05:03:38.234296+0000","last_became_peered":"2024-01-31T05:03:38.234296+0000","last_unstale":"2024-01-31T05:03:38.240233+0000","last_undegraded":"2024-01-31T05:03:38.240233+0000","last_fullsized":"2024-01-31T05:03:38.240233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269815+0000","last_change":"2024-01-31T05:03:39.260082+0000","last_active":"2024-01-31T05:03:39.269815+0000","last_peered":"2024-01-31T05:03:39.269815+0000","last_clean":"2024-01-31T05:03:39.269815+0000","last_became_active":"2024-01-31T05:03:39.256821+0000","last_became_peered":"2024-01-31T05:03:39.256821+0000","last_unstale":"2024-01-31T05:03:39.269815+0000","last_undegraded":"2024-01-31T05:03:39.269815+0000","last_fullsized":"2024-01-31T05:03:39.269815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.274034+0000","last_change":"2024-01-31T05:03:38.234981+0000","last_active":"2024-01-31T05:03:39.274034+0000","last_peered":"2024-01-31T05:03:39.274034+0000","last_clean":"2024-01-31T05:03:39.274034+0000","last_became_active":"2024-01-31T05:03:38.234678+0000","last_became_peered":"2024-01-31T05:03:38.234678+0000","last_unstale":"2024-01-31T05:03:39.274034+0000","last_undegraded":"2024-01-31T05:03:39.274034+0000","last_fullsized":"2024-01-31T05:03:39.274034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:08:18.852860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218772+0000","last_change":"2024-01-31T05:03:38.218772+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218772+0000","last_undegraded":"2024-01-31T05:03:38.218772+0000","last_fullsized":"2024-01-31T05:03:38.218772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238837+0000","last_change":"2024-01-31T05:03:38.230732+0000","last_active":"2024-01-31T05:03:38.238837+0000","last_peered":"2024-01-31T05:03:38.238837+0000","last_clean":"2024-01-31T05:03:38.238837+0000","last_became_active":"2024-01-31T05:03:38.229403+0000","last_became_peered":"2024-01-31T05:03:38.229403+0000","last_unstale":"2024-01-31T05:03:38.238837+0000","last_undegraded":"2024-01-31T05:03:38.238837+0000","last_fullsized":"2024-01-31T05:03:38.238837+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:00:00.029942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.216599+0000","last_change":"2024-01-31T05:03:38.216599+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.216599+0000","last_undegraded":"2024-01-31T05:03:38.216599+0000","last_fullsized":"2024-01-31T05:03:38.216599+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232247+0000","last_change":"2024-01-31T05:03:38.230238+0000","last_active":"2024-01-31T05:03:39.232247+0000","last_peered":"2024-01-31T05:03:39.232247+0000","last_clean":"2024-01-31T05:03:39.232247+0000","last_became_active":"2024-01-31T05:03:38.229961+0000","last_became_peered":"2024-01-31T05:03:38.229961+0000","last_unstale":"2024-01-31T05:03:39.232247+0000","last_undegraded":"2024-01-31T05:03:39.232247+0000","last_fullsized":"2024-01-31T05:03:39.232247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266259+0000","last_change":"2024-01-31T05:03:39.248707+0000","last_active":"2024-01-31T05:03:39.266259+0000","last_peered":"2024-01-31T05:03:39.266259+0000","last_clean":"2024-01-31T05:03:39.266259+0000","last_became_active":"2024-01-31T05:03:39.246545+0000","last_became_peered":"2024-01-31T05:03:39.246545+0000","last_unstale":"2024-01-31T05:03:39.266259+0000","last_undegraded":"2024-01-31T05:03:39.266259+0000","last_fullsized":"2024-01-31T05:03:39.266259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.271942+0000","last_change":"2024-01-31T05:03:38.232128+0000","last_active":"2024-01-31T05:03:39.271942+0000","last_peered":"2024-01-31T05:03:39.271942+0000","last_clean":"2024-01-31T05:03:39.271942+0000","last_became_active":"2024-01-31T05:03:38.231862+0000","last_became_peered":"2024-01-31T05:03:38.231862+0000","last_unstale":"2024-01-31T05:03:39.271942+0000","last_undegraded":"2024-01-31T05:03:39.271942+0000","last_fullsized":"2024-01-31T05:03:39.271942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267560+0000","last_change":"2024-01-31T05:03:39.252646+0000","last_active":"2024-01-31T05:03:39.267560+0000","last_peered":"2024-01-31T05:03:39.267560+0000","last_clean":"2024-01-31T05:03:39.267560+0000","last_became_active":"2024-01-31T05:03:39.244259+0000","last_became_peered":"2024-01-31T05:03:39.244259+0000","last_unstale":"2024-01-31T05:03:39.267560+0000","last_undegraded":"2024-01-31T05:03:39.267560+0000","last_fullsized":"2024-01-31T05:03:39.267560+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:01:30.708807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240725+0000","last_change":"2024-01-31T05:03:38.239097+0000","last_active":"2024-01-31T05:03:38.240725+0000","last_peered":"2024-01-31T05:03:38.240725+0000","last_clean":"2024-01-31T05:03:38.240725+0000","last_became_active":"2024-01-31T05:03:38.238757+0000","last_became_peered":"2024-01-31T05:03:38.238757+0000","last_unstale":"2024-01-31T05:03:38.240725+0000","last_undegraded":"2024-01-31T05:03:38.240725+0000","last_fullsized":"2024-01-31T05:03:38.240725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:18:04.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246623+0000","last_change":"2024-01-31T05:03:39.244177+0000","last_active":"2024-01-31T05:03:39.246623+0000","last_peered":"2024-01-31T05:03:39.246623+0000","last_clean":"2024-01-31T05:03:39.246623+0000","last_became_active":"2024-01-31T05:03:39.243841+0000","last_became_peered":"2024-01-31T05:03:39.243841+0000","last_unstale":"2024-01-31T05:03:39.246623+0000","last_undegraded":"2024-01-31T05:03:39.246623+0000","last_fullsized":"2024-01-31T05:03:39.246623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234411+0000","last_change":"2024-01-31T05:03:38.224802+0000","last_active":"2024-01-31T05:03:39.234411+0000","last_peered":"2024-01-31T05:03:39.234411+0000","last_clean":"2024-01-31T05:03:39.234411+0000","last_became_active":"2024-01-31T05:03:38.223989+0000","last_became_peered":"2024-01-31T05:03:38.223989+0000","last_unstale":"2024-01-31T05:03:39.234411+0000","last_undegraded":"2024-01-31T05:03:39.234411+0000","last_fullsized":"2024-01-31T05:03:39.234411+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219620+0000","last_change":"2024-01-31T05:03:38.219620+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219620+0000","last_undegraded":"2024-01-31T05:03:38.219620+0000","last_fullsized":"2024-01-31T05:03:38.219620+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.230405+0000","last_change":"2024-01-31T05:03:38.224866+0000","last_active":"2024-01-31T05:03:39.230405+0000","last_peered":"2024-01-31T05:03:39.230405+0000","last_clean":"2024-01-31T05:03:39.230405+0000","last_became_active":"2024-01-31T05:03:38.224437+0000","last_became_peered":"2024-01-31T05:03:38.224437+0000","last_unstale":"2024-01-31T05:03:39.230405+0000","last_undegraded":"2024-01-31T05:03:39.230405+0000","last_fullsized":"2024-01-31T05:03:39.230405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224396+0000","last_change":"2024-01-31T05:03:38.224396+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224396+0000","last_undegraded":"2024-01-31T05:03:38.224396+0000","last_fullsized":"2024-01-31T05:03:38.224396+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.225219+0000","last_change":"2024-01-31T05:03:38.236959+0000","last_active":"2024-01-31T05:03:39.225219+0000","last_peered":"2024-01-31T05:03:39.225219+0000","last_clean":"2024-01-31T05:03:39.225219+0000","last_became_active":"2024-01-31T05:03:38.235737+0000","last_became_peered":"2024-01-31T05:03:38.235737+0000","last_unstale":"2024-01-31T05:03:39.225219+0000","last_undegraded":"2024-01-31T05:03:39.225219+0000","last_fullsized":"2024-01-31T05:03:39.225219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:44.258646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.217657+0000","last_change":"2024-01-31T05:03:38.217657+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.217657+0000","last_undegraded":"2024-01-31T05:03:38.217657+0000","last_fullsized":"2024-01-31T05:03:38.217657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233182+0000","last_change":"2024-01-31T05:03:38.231189+0000","last_active":"2024-01-31T05:03:39.233182+0000","last_peered":"2024-01-31T05:03:39.233182+0000","last_clean":"2024-01-31T05:03:39.233182+0000","last_became_active":"2024-01-31T05:03:38.230998+0000","last_became_peered":"2024-01-31T05:03:38.230998+0000","last_unstale":"2024-01-31T05:03:39.233182+0000","last_undegraded":"2024-01-31T05:03:39.233182+0000","last_fullsized":"2024-01-31T05:03:39.233182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.213769+0000","last_change":"2024-01-31T05:03:38.213769+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.213769+0000","last_undegraded":"2024-01-31T05:03:38.213769+0000","last_fullsized":"2024-01-31T05:03:38.213769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238725+0000","last_change":"2024-01-31T05:03:38.235803+0000","last_active":"2024-01-31T05:03:38.238725+0000","last_peered":"2024-01-31T05:03:38.238725+0000","last_clean":"2024-01-31T05:03:38.238725+0000","last_became_active":"2024-01-31T05:03:38.234927+0000","last_became_peered":"2024-01-31T05:03:38.234927+0000","last_unstale":"2024-01-31T05:03:38.238725+0000","last_undegraded":"2024-01-31T05:03:38.238725+0000","last_fullsized":"2024-01-31T05:03:38.238725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:14.881712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246903+0000","last_change":"2024-01-31T05:03:39.243770+0000","last_active":"2024-01-31T05:03:39.246903+0000","last_peered":"2024-01-31T05:03:39.246903+0000","last_clean":"2024-01-31T05:03:39.246903+0000","last_became_active":"2024-01-31T05:03:39.243396+0000","last_became_peered":"2024-01-31T05:03:39.243396+0000","last_unstale":"2024-01-31T05:03:39.246903+0000","last_undegraded":"2024-01-31T05:03:39.246903+0000","last_fullsized":"2024-01-31T05:03:39.246903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240286+0000","last_change":"2024-01-31T05:03:38.235205+0000","last_active":"2024-01-31T05:03:38.240286+0000","last_peered":"2024-01-31T05:03:38.240286+0000","last_clean":"2024-01-31T05:03:38.240286+0000","last_became_active":"2024-01-31T05:03:38.234627+0000","last_became_peered":"2024-01-31T05:03:38.234627+0000","last_unstale":"2024-01-31T05:03:38.240286+0000","last_undegraded":"2024-01-31T05:03:38.240286+0000","last_fullsized":"2024-01-31T05:03:38.240286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.212184+0000","last_change":"2024-01-31T05:03:38.212184+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.212184+0000","last_undegraded":"2024-01-31T05:03:38.212184+0000","last_fullsized":"2024-01-31T05:03:38.212184+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.236723+0000","last_change":"2024-01-31T05:03:38.233879+0000","last_active":"2024-01-31T05:03:38.236723+0000","last_peered":"2024-01-31T05:03:38.236723+0000","last_clean":"2024-01-31T05:03:38.236723+0000","last_became_active":"2024-01-31T05:03:38.233495+0000","last_became_peered":"2024-01-31T05:03:38.233495+0000","last_unstale":"2024-01-31T05:03:38.236723+0000","last_undegraded":"2024-01-31T05:03:38.236723+0000","last_fullsized":"2024-01-31T05:03:38.236723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232380+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:39.232380+0000","last_peered":"2024-01-31T05:03:39.232380+0000","last_clean":"2024-01-31T05:03:39.232380+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:39.232380+0000","last_undegraded":"2024-01-31T05:03:39.232380+0000","last_fullsized":"2024-01-31T05:03:39.232380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":275170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0149999999999999,"15min":1.0009999999999999},"min":{"1min":0.68200000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5369999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.099,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6719999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.244,"15min":1.256},"min":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6259999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.872}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.175,"15min":1.1799999999999999},"min":{"1min":0.747,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4930000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.29,"15min":1.292},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.512,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.016}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2230000000000001,"15min":1.2150000000000001},"min":{"1min":0.91900000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.609,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6120000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.179}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":0.98499999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.609,"5min":2.25,"15min":2.25},"last":1.349},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3580000000000001,"15min":1.3740000000000001},"min":{"1min":1.004,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.573,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.472,"15min":1.4750000000000001},"min":{"1min":1.0109999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.762,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.85999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":1.0089999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.1240000000000001,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5529999999999999,"15min":1.5609999999999999},"min":{"1min":1.2709999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.726,"5min":2.629,"15min":2.629},"last":1.6359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":282881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0329999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3240000000000001,"5min":1.476,"15min":1.476},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.05,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.876}]},{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.78600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.587,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1200000000000001,"15min":1.131},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5369999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.48}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.6950000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.569},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1799999999999999,"15min":1.1839999999999999},"min":{"1min":0.80400000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.421,"5min":1.883,"15min":1.883},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.381,"15min":1.395},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.72,"5min":1.73,"15min":1.73},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.95599999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0830000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6259999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.878,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.611,"5min":2.032,"15min":2.032},"last":1.012},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.585},"min":{"1min":1.0780000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.984,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.165}]},{"osd":6,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5209999999999999,"15min":1.5329999999999999},"min":{"1min":0.96899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.972,"5min":2.335,"15min":2.335},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5860000000000001,"15min":1.6060000000000001},"min":{"1min":1.2609999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.768,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.115}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.599,"15min":1.6160000000000001},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8620000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.071},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.637,"15min":1.659},"min":{"1min":1.0920000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.891,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.7350000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":282355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.151},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1919999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":1.032,"15min":1.0840000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1439999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.121,"15min":1.1779999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4910000000000001,"5min":5.617,"15min":5.617},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.2330000000000001,"15min":1.2989999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.94599999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.292,"15min":1.3680000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.484,"15min":1.5720000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4930000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.6220000000000001,"15min":1.7230000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.9990000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.921},{"interface":"front","average":{"1min":1.141,"5min":1.621,"15min":1.7010000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.696,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.2130000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.639,"15min":1.7230000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9339999999999999,"5min":6.048,"15min":6.048},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.7430000000000001,"15min":1.831},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.952,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.804}]},{"osd":6,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.6739999999999999,"15min":1.744},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.024,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.657,"15min":1.748},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5700000000000001,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.6539999999999999,"15min":1.74},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.802,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.8380000000000001,"15min":1.946},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.73,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.5609999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":284037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.667,"5min":1.73,"15min":1.73},"last":1.4390000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.88900000000000001,"15min":0.90300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.472,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.046},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.60499999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.099}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93000000000000005,"15min":0.92300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.23},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0780000000000001,"15min":1.087},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":1.798,"15min":1.798},"last":1.387}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5780000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3220000000000001,"15min":1.331},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0059999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.321}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.232,"15min":1.2410000000000001},"min":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.585,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.472},{"interface":"front","average":{"1min":1.26,"5min":1.3100000000000001,"15min":1.3180000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8220000000000001,"5min":2.52,"15min":2.52},"last":1.494}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.419},"min":{"1min":1.0189999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1320000000000001,"5min":2.992,"15min":2.992},"last":1.663},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3939999999999999,"15min":1.4079999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.6879999999999999,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.397,"15min":1.403},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.8700000000000001,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.869,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.619}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1236992,"data_stored":873215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":1.026,"15min":1.0820000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.272,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.149,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4890000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.98399999999999999,"15min":1.0289999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0229999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.131,"15min":1.1839999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4099999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":1.0109999999999999,"15min":1.0449999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4419999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1639999999999999,"15min":1.222},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.294,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.028}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.4550000000000001,"15min":1.5089999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5940000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.345,"15min":1.379},"min":{"1min":0.80900000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4630000000000001,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.4059999999999999,"15min":1.4470000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.4650000000000001,"5min":2.722,"15min":2.722},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.409,"15min":1.4550000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.587,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.5189999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4790000000000001,"15min":1.538},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6439999999999999,"5min":2.577,"15min":2.577},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.415,"15min":1.4510000000000001},"min":{"1min":0.85699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5609999999999999,"5min":2.794,"15min":2.794},"last":1.651}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.4930000000000001,"15min":1.5449999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.647,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.5640000000000001,"15min":1.615},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5489999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.581}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.135,"15min":1.151},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.091},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.552,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1339999999999999,"15min":1.131},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5589999999999999,"5min":1.573,"15min":1.573},"last":1.242},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.73799999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5349999999999999,"5min":1.976,"15min":1.976},"last":1.1459999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7470000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.266,"15min":1.28},"min":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.544,"5min":1.873,"15min":1.873},"last":1.321}]},{"osd":3,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3580000000000001,"15min":1.375},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.7050000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.335,"15min":1.339},"min":{"1min":0.95899999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.4399999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.254,"15min":1.2529999999999999},"min":{"1min":0.79300000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6890000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":0.94399999999999995,"5min":0.628,"15min":0.628},"max":{"1min":2.0499999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4430000000000001,"15min":1.454},"min":{"1min":1.0049999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.679,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.379},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.593,"15min":1.6040000000000001},"min":{"1min":1.1990000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0990000000000002,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.462}]},{"osd":7,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5620000000000001,"15min":1.5820000000000001},"min":{"1min":0.89400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9530000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6719999999999999,"15min":1.6839999999999999},"min":{"1min":1.246,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9990000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.409}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":865538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.038,"15min":1.022},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.792}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1279999999999999,"15min":1.141},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.2030000000000001,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5720000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.3320000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.206,"15min":1.222},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.663,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.29},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.296,"15min":1.3089999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.647,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.49}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.304,"15min":1.306},"min":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.704,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.389,"15min":1.399},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.712,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.5309999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":0.70399999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.0139999999999998,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.91400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.792,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4259999999999999,"15min":1.4099999999999999},"min":{"1min":0.93500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":1.494},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.851,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5860000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.948,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.0029999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5219999999999998,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.6240000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0389999999999999,"15min":1.0489999999999999},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.0680000000000001,"5min":3.282,"15min":3.282},"last":0.627}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.075,"5min":1.774,"15min":1.774},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.091,"15min":1.089},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.2250000000000001},"min":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4370000000000001,"15min":1.4399999999999999},"min":{"1min":0.93999999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.8500000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.097}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.492,"15min":1.482},"min":{"1min":0.85399999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":5.75,"5min":5.75,"15min":5.75},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.468,"15min":1.4750000000000001},"min":{"1min":0.86499999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":4.8799999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.169}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.355,"15min":1.3300000000000001},"min":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":7.415,"5min":7.415,"15min":7.415},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.704,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":0.88900000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":7.5650000000000004,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.298}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.877,"5min":0.751,"15min":0.751},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:03:44.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.231612+0000', 'last_change': '2024-01-31T05:03:38.231612+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.231612+0000', 'last_undegraded': '2024-01-31T05:03:38.231612+0000', 'last_fullsized': '2024-01-31T05:03:38.231612+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.222392+0000', 'last_change': '2024-01-31T05:03:38.222392+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.222392+0000', 'last_undegraded': '2024-01-31T05:03:38.222392+0000', 'last_fullsized': '2024-01-31T05:03:38.222392+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.226676+0000', 'last_change': '2024-01-31T05:03:38.226676+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.226676+0000', 'last_undegraded': '2024-01-31T05:03:38.226676+0000', 'last_fullsized': '2024-01-31T05:03:38.226676+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.215147+0000', 'last_change': '2024-01-31T05:03:38.215147+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.215147+0000', 'last_undegraded': '2024-01-31T05:03:38.215147+0000', 'last_fullsized': '2024-01-31T05:03:38.215147+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-01-31T05:03:44.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.226447+0000', 'last_change': '2024-01-31T05:03:38.226447+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.226447+0000', 'last_undegraded': '2024-01-31T05:03:38.226447+0000', 'last_fullsized': '2024-01-31T05:03:38.226447+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.218606+0000', 'last_change': '2024-01-31T05:03:38.218606+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.218606+0000', 'last_undegraded': '2024-01-31T05:03:38.218606+0000', 'last_fullsized': '2024-01-31T05:03:38.218606+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.222879+0000', 'last_change': '2024-01-31T05:03:38.222879+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.222879+0000', 'last_undegraded': '2024-01-31T05:03:38.222879+0000', 'last_fullsized': '2024-01-31T05:03:38.222879+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.220373+0000', 'last_change': '2024-01-31T05:03:38.220373+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.220373+0000', 'last_undegraded': '2024-01-31T05:03:38.220373+0000', 'last_fullsized': '2024-01-31T05:03:38.220373+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.231217+0000', 'last_change': '2024-01-31T05:03:38.231217+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.231217+0000', 'last_undegraded': '2024-01-31T05:03:38.231217+0000', 'last_fullsized': '2024-01-31T05:03:38.231217+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-01-31T05:03:44.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.218199+0000', 'last_change': '2024-01-31T05:03:38.218199+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.218199+0000', 'last_undegraded': '2024-01-31T05:03:38.218199+0000', 'last_fullsized': '2024-01-31T05:03:38.218199+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-31T05:03:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-01-31T05:03:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.218748+0000', 'last_change': '2024-01-31T05:03:38.218748+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.218748+0000', 'last_undegraded': '2024-01-31T05:03:38.218748+0000', 'last_fullsized': '2024-01-31T05:03:38.218748+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-31T05:03:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-01-31T05:03:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.227220+0000', 'last_change': '2024-01-31T05:03:38.227220+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.227220+0000', 'last_undegraded': '2024-01-31T05:03:38.227220+0000', 'last_fullsized': '2024-01-31T05:03:38.227220+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-01-31T05:03:44.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.219977+0000', 'last_change': '2024-01-31T05:03:38.219977+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.219977+0000', 'last_undegraded': '2024-01-31T05:03:38.219977+0000', 'last_fullsized': '2024-01-31T05:03:38.219977+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.225923+0000', 'last_change': '2024-01-31T05:03:38.225923+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.225923+0000', 'last_undegraded': '2024-01-31T05:03:38.225923+0000', 'last_fullsized': '2024-01-31T05:03:38.225923+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.230764+0000', 'last_change': '2024-01-31T05:03:38.230764+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.230764+0000', 'last_undegraded': '2024-01-31T05:03:38.230764+0000', 'last_fullsized': '2024-01-31T05:03:38.230764+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.226684+0000', 'last_change': '2024-01-31T05:03:38.226684+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.226684+0000', 'last_undegraded': '2024-01-31T05:03:38.226684+0000', 'last_fullsized': '2024-01-31T05:03:38.226684+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.224220+0000', 'last_change': '2024-01-31T05:03:38.224220+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.224220+0000', 'last_undegraded': '2024-01-31T05:03:38.224220+0000', 'last_fullsized': '2024-01-31T05:03:38.224220+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.210993+0000', 'last_change': '2024-01-31T05:03:38.210993+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.210993+0000', 'last_undegraded': '2024-01-31T05:03:38.210993+0000', 'last_fullsized': '2024-01-31T05:03:38.210993+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.220574+0000', 'last_change': '2024-01-31T05:03:38.220574+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.220574+0000', 'last_undegraded': '2024-01-31T05:03:38.220574+0000', 'last_fullsized': '2024-01-31T05:03:38.220574+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.218290+0000', 'last_change': '2024-01-31T05:03:38.218290+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.218290+0000', 'last_undegraded': '2024-01-31T05:03:38.218290+0000', 'last_fullsized': '2024-01-31T05:03:38.218290+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.219356+0000', 'last_change': '2024-01-31T05:03:38.219356+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.219356+0000', 'last_undegraded': '2024-01-31T05:03:38.219356+0000', 'last_fullsized': '2024-01-31T05:03:38.219356+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.218772+0000', 'last_change': '2024-01-31T05:03:38.218772+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.218772+0000', 'last_undegraded': '2024-01-31T05:03:38.218772+0000', 'last_fullsized': '2024-01-31T05:03:38.218772+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-01-31T05:03:44.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.216599+0000', 'last_change': '2024-01-31T05:03:38.216599+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.216599+0000', 'last_undegraded': '2024-01-31T05:03:38.216599+0000', 'last_fullsized': '2024-01-31T05:03:38.216599+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.219620+0000', 'last_change': '2024-01-31T05:03:38.219620+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.219620+0000', 'last_undegraded': '2024-01-31T05:03:38.219620+0000', 'last_fullsized': '2024-01-31T05:03:38.219620+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.224396+0000', 'last_change': '2024-01-31T05:03:38.224396+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.224396+0000', 'last_undegraded': '2024-01-31T05:03:38.224396+0000', 'last_fullsized': '2024-01-31T05:03:38.224396+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.217657+0000', 'last_change': '2024-01-31T05:03:38.217657+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.217657+0000', 'last_undegraded': '2024-01-31T05:03:38.217657+0000', 'last_fullsized': '2024-01-31T05:03:38.217657+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.213769+0000', 'last_change': '2024-01-31T05:03:38.213769+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.213769+0000', 'last_undegraded': '2024-01-31T05:03:38.213769+0000', 'last_fullsized': '2024-01-31T05:03:38.213769+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-01-31T05:03:44.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-01-31T05:03:38.212184+0000', 'last_change': '2024-01-31T05:03:38.212184+0000', 'last_active': '2024-01-31T05:03:38.190665+0000', 'last_peered': '2024-01-31T05:03:38.190665+0000', 'last_clean': '2024-01-31T05:03:38.190665+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-31T05:03:38.212184+0000', 'last_undegraded': '2024-01-31T05:03:38.212184+0000', 'last_fullsized': '2024-01-31T05:03:38.212184+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'last_clean_scrub_stamp': '2024-01-31T05:03:38.190665+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-31T05:03:44.645 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-31T05:03:44.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.887+0000 7fa903800640 1 Processor -- start 2024-01-31T05:03:44.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.887+0000 7fa903800640 1 -- start start 2024-01-31T05:03:44.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa903800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 0x7fa8fc0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa903800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8fc0ccd00 0x7fa8fc0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa903800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8fc06d1a0 con 0x7fa8fc15b400 2024-01-31T05:03:44.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa903800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8fc0cc960 con 0x7fa8fc0ccd00 2024-01-31T05:03:44.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa903800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0cd6f0 con 0x7fa8fc0cb810 2024-01-31T05:03:44.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa8f3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa8fc15b400 0x7fa8fc06d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:55408/0 (socket says 172.21.15.97:55408) 2024-01-31T05:03:44.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.889+0000 7fa8f3e00640 1 -- 172.21.15.97:0/1436523519 learned_addr learned my addr 172.21.15.97:0/1436523519 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa900e00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 0x7fa8fc0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa901800640 1 --2- 172.21.15.97:0/1436523519 >> v2:172.21.15.188:3300/0 conn(0x7fa8fc0ccd00 0x7fa8fc0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419904377 0 0) 0x7fa8fc06d1a0 con 0x7fa8fc15b400 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8dc003520 con 0x7fa8fc15b400 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 485782629 0 0) 0x7fa8dc003520 con 0x7fa8fc15b400 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 msgr2=0x7fa8fc0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 0x7fa8fc0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 >> v2:172.21.15.188:3300/0 conn(0x7fa8fc0ccd00 msgr2=0x7fa8fc0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:44.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 --2- 172.21.15.97:0/1436523519 >> v2:172.21.15.188:3300/0 conn(0x7fa8fc0ccd00 0x7fa8fc0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc06c600 con 0x7fa8fc15b400 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa8e4001670 con 0x7fa8fc15b400 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 >> v1:172.21.15.97:6789/0 conn(0x7fa8fc15b400 legacy=0x7fa8fc06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc003970 0x7fa8dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 0x7fa8dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8dc004d00 0x7fa8dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0cc960 con 0x7fa8dc004d00 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0cd6f0 con 0x7fa8dc003970 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.890+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc00b980 con 0x7fa8fc0cb810 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa901800640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8dc004d00 0x7fa8dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa900e00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc003970 0x7fa8dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa8f3e00640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 0x7fa8dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa901800640 1 -- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 msgr2=0x7fa8dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:44.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa901800640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc0cb810 0x7fa8dc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa901800640 1 -- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc003970 msgr2=0x7fa8dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa901800640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc003970 0x7fa8dc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.891+0000 7fa901800640 1 -- 172.21.15.97:0/1436523519 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc003520 con 0x7fa8dc004d00 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.892+0000 7fa901800640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8dc004d00 0x7fa8dc005180 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4007390 tx=0x7fa8dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64f759afc29abb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.892+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f402ccf0 con 0x7fa8dc004d00 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.892+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f402ce50 con 0x7fa8dc004d00 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.892+0000 7fa8fbe00640 1 -- 172.21.15.97:0/1436523519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4034660 con 0x7fa8dc004d00 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa903800640 1 -- 172.21.15.97:0/1436523519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8dc004d00 msgr2=0x7fa8dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa903800640 1 --2- 172.21.15.97:0/1436523519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8dc004d00 0x7fa8dc005180 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4007390 tx=0x7fa8dc0057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa901800640 1 -- 172.21.15.97:0/1436523519 reap_dead start 2024-01-31T05:03:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa903800640 1 -- 172.21.15.97:0/1436523519 shutdown_connections 2024-01-31T05:03:44.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa903800640 1 -- 172.21.15.97:0/1436523519 >> 172.21.15.97:0/1436523519 conn(0x7fa8fc0b0a10 msgr2=0x7fa8fc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:44.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa903800640 1 -- 172.21.15.97:0/1436523519 shutdown_connections 2024-01-31T05:03:44.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.893+0000 7fa903800640 1 -- 172.21.15.97:0/1436523519 wait complete. 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.895+0000 7fa903800640 1 Processor -- start 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.895+0000 7fa903800640 1 -- start start 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa903800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc004d00 0x7fa8fc0999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa903800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc15da90 0x7fa8fc0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa903800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 0x7fa8fc14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa903800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc14de70 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa903800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc09a210 con 0x7fa8dc004d00 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa903800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc09a4f0 con 0x7fa8fc15da90 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa901800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 0x7fa8fc14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa901800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 0x7fa8fc14d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42366/0 (socket says 172.21.15.97:42366) 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa901800640 1 -- 172.21.15.97:0/4221667497 learned_addr learned my addr 172.21.15.97:0/4221667497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa8f3e00640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc15da90 0x7fa8fc0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa901800640 1 -- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc15da90 msgr2=0x7fa8fc0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.896+0000 7fa900e00640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc004d00 0x7fa8fc0999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa901800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc15da90 0x7fa8fc0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa901800640 1 -- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc004d00 msgr2=0x7fa8fc0999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa901800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc004d00 0x7fa8fc0999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa901800640 1 -- 172.21.15.97:0/4221667497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc09a7d0 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa8f3e00640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc15da90 0x7fa8fc0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa901800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 0x7fa8fc14d930 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa8f402dfd0 tx=0x7fa8f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4003b90 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f4003d10 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc09aab0 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4004260 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.897+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc099f00 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.899+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc0677c0 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.899+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa8f403b040 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.900+0000 7fa8f9600640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa8d406af00 0x7fa8d406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.900+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa8f40aaa40 con 0x7fa8fc14d4b0 2024-01-31T05:03:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.901+0000 7fa900e00640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa8d406af00 0x7fa8d406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.901+0000 7fa900e00640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa8d406af00 0x7fa8d406d3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009910 tx=0x7fa8ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:44.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:44.905+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa8f40ace60 con 0x7fa8fc14d4b0 2024-01-31T05:03:45.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.161+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa8fc0ae3e0 con 0x7fa8fc14d4b0 2024-01-31T05:03:45.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.162+0000 7fa8f9600640 1 -- 172.21.15.97:0/4221667497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1098 (secure 0 0 0) 0x7fa8f4045460 con 0x7fa8fc14d4b0 2024-01-31T05:03:45.162 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:45.162 INFO:teuthology.orchestra.run.smithi097.stdout:{"fsid":"dc34a409-d027-49db-a591-dbd89a074581","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":198,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1706677238,"num_in_osds":8,"osd_in_since":1706677230,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":69},{"state_name":"creating+peering","count":28}],"num_pgs":97,"num_pools":3,"num_objects":15,"data_bytes":591132,"bytes_used":227635200,"bytes_avail":772866478080,"bytes_total":773094113280,"inactive_pgs_ratio":0.28865978121757507,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":27,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6240}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-01-31T05:03:25.605712+0000","services":{}},"progress_events":{}} 2024-01-31T05:03:45.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa8d406af00 msgr2=0x7fa8d406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa8d406af00 0x7fa8d406d3c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009910 tx=0x7fa8ec008040 comp rx=0 tx=0).stop 2024-01-31T05:03:45.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 msgr2=0x7fa8fc14d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 0x7fa8fc14d930 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa8f402dfd0 tx=0x7fa8f4004ab0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 shutdown_connections 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa8d406af00 0x7fa8d406d3c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8fc14d4b0 0x7fa8fc14d930 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.164+0000 7fa903800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8fc15da90 0x7fa8fc0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.165+0000 7fa903800640 1 --2- 172.21.15.97:0/4221667497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8dc004d00 0x7fa8fc0999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.165+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 >> 172.21.15.97:0/4221667497 conn(0x7fa8fc0b0a10 msgr2=0x7fa8fc0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.165+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 shutdown_connections 2024-01-31T05:03:45.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.165+0000 7fa903800640 1 -- 172.21.15.97:0/4221667497 wait complete. 2024-01-31T05:03:45.182 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-31T05:03:45.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:03:45.182 INFO:tasks.ceph.mds.c.smithi097.stderr:2024-01-31T05:03:45.183+0000 7ff578200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:03:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.387+0000 7f84d4200640 1 Processor -- start 2024-01-31T05:03:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.387+0000 7f84d4200640 1 -- start start 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d4200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f84cc1616d0 0x7f84cc0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d4200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84cc0ce430 0x7f84cc06ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d4200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84cc0abcf0 con 0x7f84cc15d520 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d4200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84cc15d900 con 0x7f84cc0ce430 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d4200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84cc06ca40 con 0x7f84cc1616d0 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d2200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84cc0ce430 0x7f84cc06ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d2200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84cc0ce430 0x7f84cc06ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41880/0 (socket says 172.21.15.97:41880) 2024-01-31T05:03:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.389+0000 7f84d2200640 1 -- 172.21.15.97:0/4245031211 learned_addr learned my addr 172.21.15.97:0/4245031211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:45.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.390+0000 7f84d2200640 1 -- 172.21.15.97:0/4245031211 >> v2:172.21.15.188:3300/0 conn(0x7f84cc1616d0 msgr2=0x7f84cc0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:45.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.390+0000 7f84d2200640 1 --2- 172.21.15.97:0/4245031211 >> v2:172.21.15.188:3300/0 conn(0x7f84cc1616d0 0x7f84cc0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.390+0000 7f84d2200640 1 -- 172.21.15.97:0/4245031211 >> v1:172.21.15.188:6789/0 conn(0x7f84cc15d520 legacy=0x7f84cc160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:45.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.390+0000 7f84d2200640 1 -- 172.21.15.97:0/4245031211 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc161e20 con 0x7f84cc0ce430 2024-01-31T05:03:45.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.391+0000 7f84d2200640 1 --2- 172.21.15.97:0/4245031211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc06ced0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0cc510 tx=0x7f84c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c849acd998c8874 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:45.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.391+0000 7f84cbe00640 1 -- 172.21.15.97:0/4245031211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c402ca80 con 0x7f84cc0ce430 2024-01-31T05:03:45.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.391+0000 7f84cbe00640 1 -- 172.21.15.97:0/4245031211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c402cbe0 con 0x7f84cc0ce430 2024-01-31T05:03:45.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.391+0000 7f84cbe00640 1 -- 172.21.15.97:0/4245031211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c40343f0 con 0x7f84cc0ce430 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 -- 172.21.15.97:0/4245031211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 msgr2=0x7f84cc06ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 --2- 172.21.15.97:0/4245031211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc06ced0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0cc510 tx=0x7f84c402bee0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 -- 172.21.15.97:0/4245031211 shutdown_connections 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 --2- 172.21.15.97:0/4245031211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc06ced0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 --2- 172.21.15.97:0/4245031211 >> v2:172.21.15.188:3300/0 conn(0x7f84cc1616d0 0x7f84cc0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 -- 172.21.15.97:0/4245031211 >> 172.21.15.97:0/4245031211 conn(0x7f84cc0b0a10 msgr2=0x7f84cc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.392+0000 7f84d4200640 1 -- 172.21.15.97:0/4245031211 shutdown_connections 2024-01-31T05:03:45.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.393+0000 7f84d4200640 1 -- 172.21.15.97:0/4245031211 wait complete. 2024-01-31T05:03:45.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.394+0000 7f84d4200640 1 Processor -- start 2024-01-31T05:03:45.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.394+0000 7f84d4200640 1 -- start start 2024-01-31T05:03:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.395+0000 7f84d4200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.395+0000 7f84d4200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84cc15d520 0x7f84cc147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.395+0000 7f84d4200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84cc1616d0 0x7f84cc1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.395+0000 7f84d4200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1f2ae0 con 0x7f84cc0ce430 2024-01-31T05:03:45.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.395+0000 7f84d4200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc15b720 con 0x7f84cc15d520 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.395+0000 7f84d4200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc15ba00 con 0x7f84cc1616d0 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41892/0 (socket says 172.21.15.97:41892) 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 -- 172.21.15.97:0/3492645556 learned_addr learned my addr 172.21.15.97:0/3492645556 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d0e00640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84cc15d520 0x7f84cc147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d2200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84cc1616d0 0x7f84cc1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 -- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84cc1616d0 msgr2=0x7f84cc1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84cc1616d0 0x7f84cc1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 -- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84cc15d520 msgr2=0x7f84cc147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84cc15d520 0x7f84cc147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 -- 172.21.15.97:0/3492645556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009b30 con 0x7f84cc0ce430 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.396+0000 7f84d1800640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc15c7c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c410 tx=0x7f84bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:45.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.397+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0035f0 con 0x7f84cc0ce430 2024-01-31T05:03:45.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.397+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc003750 con 0x7f84cc0ce430 2024-01-31T05:03:45.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.397+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc15bce0 con 0x7f84cc0ce430 2024-01-31T05:03:45.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.397+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc003b40 con 0x7f84cc0ce430 2024-01-31T05:03:45.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.397+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc15cd00 con 0x7f84cc0ce430 2024-01-31T05:03:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.398+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc15c4e0 con 0x7f84cc0ce430 2024-01-31T05:03:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.399+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f84bc003ca0 con 0x7f84cc0ce430 2024-01-31T05:03:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.400+0000 7f84c9600640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f84a406aed0 0x7f84a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.400+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f84bc088ec0 con 0x7f84cc0ce430 2024-01-31T05:03:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.400+0000 7f84d0e00640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f84a406aed0 0x7f84a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:45.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.401+0000 7f84d0e00640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f84a406aed0 0x7f84a406d390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84cc14d5c0 tx=0x7f84b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:45.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.405+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f84cc15c4e0 con 0x7f84cc0ce430 2024-01-31T05:03:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.579+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84cc0cdcf0 con 0x7f84a406aed0 2024-01-31T05:03:45.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.588+0000 7f84c9600640 1 -- 172.21.15.97:0/3492645556 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284616 (secure 0 0 0) 0x7f84b400c4f0 con 0x7f84a406aed0 2024-01-31T05:03:45.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:45.593 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:03:45.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f84a406aed0 msgr2=0x7f84a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f84a406aed0 0x7f84a406d390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f84cc14d5c0 tx=0x7f84b4008040 comp rx=0 tx=0).stop 2024-01-31T05:03:45.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 msgr2=0x7f84cc15c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:45.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc15c7c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c410 tx=0x7f84bc00c8d0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 shutdown_connections 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84cc1616d0 0x7f84cc1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84cc15d520 0x7f84cc147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f84a406aed0 0x7f84a406d390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 --2- 172.21.15.97:0/3492645556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84cc0ce430 0x7f84cc15c7c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.594+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 >> 172.21.15.97:0/3492645556 conn(0x7f84cc0b0a10 msgr2=0x7f84cc06d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.595+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 shutdown_connections 2024-01-31T05:03:45.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:45.595+0000 7f84d4200640 1 -- 172.21.15.97:0/3492645556 wait complete. 2024-01-31T05:03:45.611 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-01-31T05:03:43.610324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591132,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":188,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222300,"kb_used_data":6116,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752420,"statfs":{"total":773094113280,"available":772866478080,"internally_reserved":0,"allocated":6262784,"data_stored":3400346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001696"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231612+0000","last_change":"2024-01-31T05:03:38.231612+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231612+0000","last_undegraded":"2024-01-31T05:03:38.231612+0000","last_fullsized":"2024-01-31T05:03:38.231612+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.254203+0000","last_change":"2024-01-31T05:03:39.252312+0000","last_active":"2024-01-31T05:03:39.254203+0000","last_peered":"2024-01-31T05:03:39.254203+0000","last_clean":"2024-01-31T05:03:39.254203+0000","last_became_active":"2024-01-31T05:03:39.252042+0000","last_became_peered":"2024-01-31T05:03:39.252042+0000","last_unstale":"2024-01-31T05:03:39.254203+0000","last_undegraded":"2024-01-31T05:03:39.254203+0000","last_fullsized":"2024-01-31T05:03:39.254203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259236+0000","last_change":"2024-01-31T05:03:39.243310+0000","last_active":"2024-01-31T05:03:39.259236+0000","last_peered":"2024-01-31T05:03:39.259236+0000","last_clean":"2024-01-31T05:03:39.259236+0000","last_became_active":"2024-01-31T05:03:39.242309+0000","last_became_peered":"2024-01-31T05:03:39.242309+0000","last_unstale":"2024-01-31T05:03:39.259236+0000","last_undegraded":"2024-01-31T05:03:39.259236+0000","last_fullsized":"2024-01-31T05:03:39.259236+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.261556+0000","last_change":"2024-01-31T05:03:39.254633+0000","last_active":"2024-01-31T05:03:39.261556+0000","last_peered":"2024-01-31T05:03:39.261556+0000","last_clean":"2024-01-31T05:03:39.261556+0000","last_became_active":"2024-01-31T05:03:39.254325+0000","last_became_peered":"2024-01-31T05:03:39.254325+0000","last_unstale":"2024-01-31T05:03:39.261556+0000","last_undegraded":"2024-01-31T05:03:39.261556+0000","last_fullsized":"2024-01-31T05:03:39.261556+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:03:40.366719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222392+0000","last_change":"2024-01-31T05:03:38.222392+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222392+0000","last_undegraded":"2024-01-31T05:03:38.222392+0000","last_fullsized":"2024-01-31T05:03:38.222392+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226676+0000","last_change":"2024-01-31T05:03:38.226676+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226676+0000","last_undegraded":"2024-01-31T05:03:38.226676+0000","last_fullsized":"2024-01-31T05:03:38.226676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260398+0000","last_change":"2024-01-31T05:03:39.247885+0000","last_active":"2024-01-31T05:03:39.260398+0000","last_peered":"2024-01-31T05:03:39.260398+0000","last_clean":"2024-01-31T05:03:39.260398+0000","last_became_active":"2024-01-31T05:03:39.247556+0000","last_became_peered":"2024-01-31T05:03:39.247556+0000","last_unstale":"2024-01-31T05:03:39.260398+0000","last_undegraded":"2024-01-31T05:03:39.260398+0000","last_fullsized":"2024-01-31T05:03:39.260398+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:23:59.282060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.215147+0000","last_change":"2024-01-31T05:03:38.215147+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.215147+0000","last_undegraded":"2024-01-31T05:03:38.215147+0000","last_fullsized":"2024-01-31T05:03:38.215147+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.270024+0000","last_change":"2024-01-31T05:03:39.256441+0000","last_active":"2024-01-31T05:03:39.270024+0000","last_peered":"2024-01-31T05:03:39.270024+0000","last_clean":"2024-01-31T05:03:39.270024+0000","last_became_active":"2024-01-31T05:03:39.256105+0000","last_became_peered":"2024-01-31T05:03:39.256105+0000","last_unstale":"2024-01-31T05:03:39.270024+0000","last_undegraded":"2024-01-31T05:03:39.270024+0000","last_fullsized":"2024-01-31T05:03:39.270024+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226447+0000","last_change":"2024-01-31T05:03:38.226447+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226447+0000","last_undegraded":"2024-01-31T05:03:38.226447+0000","last_fullsized":"2024-01-31T05:03:38.226447+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259313+0000","last_change":"2024-01-31T05:03:39.251020+0000","last_active":"2024-01-31T05:03:39.259313+0000","last_peered":"2024-01-31T05:03:39.259313+0000","last_clean":"2024-01-31T05:03:39.259313+0000","last_became_active":"2024-01-31T05:03:39.250684+0000","last_became_peered":"2024-01-31T05:03:39.250684+0000","last_unstale":"2024-01-31T05:03:39.259313+0000","last_undegraded":"2024-01-31T05:03:39.259313+0000","last_fullsized":"2024-01-31T05:03:39.259313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:39:06.883661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269332+0000","last_change":"2024-01-31T05:03:39.258617+0000","last_active":"2024-01-31T05:03:39.269332+0000","last_peered":"2024-01-31T05:03:39.269332+0000","last_clean":"2024-01-31T05:03:39.269332+0000","last_became_active":"2024-01-31T05:03:39.258317+0000","last_became_peered":"2024-01-31T05:03:39.258317+0000","last_unstale":"2024-01-31T05:03:39.269332+0000","last_undegraded":"2024-01-31T05:03:39.269332+0000","last_fullsized":"2024-01-31T05:03:39.269332+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218606+0000","last_change":"2024-01-31T05:03:38.218606+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218606+0000","last_undegraded":"2024-01-31T05:03:38.218606+0000","last_fullsized":"2024-01-31T05:03:38.218606+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267681+0000","last_change":"2024-01-31T05:03:39.251453+0000","last_active":"2024-01-31T05:03:39.267681+0000","last_peered":"2024-01-31T05:03:39.267681+0000","last_clean":"2024-01-31T05:03:39.267681+0000","last_became_active":"2024-01-31T05:03:39.248059+0000","last_became_peered":"2024-01-31T05:03:39.248059+0000","last_unstale":"2024-01-31T05:03:39.267681+0000","last_undegraded":"2024-01-31T05:03:39.267681+0000","last_fullsized":"2024-01-31T05:03:39.267681+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.908110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.222879+0000","last_change":"2024-01-31T05:03:38.222879+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.222879+0000","last_undegraded":"2024-01-31T05:03:38.222879+0000","last_fullsized":"2024-01-31T05:03:38.222879+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220373+0000","last_change":"2024-01-31T05:03:38.220373+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220373+0000","last_undegraded":"2024-01-31T05:03:38.220373+0000","last_fullsized":"2024-01-31T05:03:38.220373+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.231217+0000","last_change":"2024-01-31T05:03:38.231217+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.231217+0000","last_undegraded":"2024-01-31T05:03:38.231217+0000","last_fullsized":"2024-01-31T05:03:38.231217+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266404+0000","last_change":"2024-01-31T05:03:39.249229+0000","last_active":"2024-01-31T05:03:39.266404+0000","last_peered":"2024-01-31T05:03:39.266404+0000","last_clean":"2024-01-31T05:03:39.266404+0000","last_became_active":"2024-01-31T05:03:39.248929+0000","last_became_peered":"2024-01-31T05:03:39.248929+0000","last_unstale":"2024-01-31T05:03:39.266404+0000","last_undegraded":"2024-01-31T05:03:39.266404+0000","last_fullsized":"2024-01-31T05:03:39.266404+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259739+0000","last_change":"2024-01-31T05:03:39.255701+0000","last_active":"2024-01-31T05:03:39.259739+0000","last_peered":"2024-01-31T05:03:39.259739+0000","last_clean":"2024-01-31T05:03:39.259739+0000","last_became_active":"2024-01-31T05:03:39.253975+0000","last_became_peered":"2024-01-31T05:03:39.253975+0000","last_unstale":"2024-01-31T05:03:39.259739+0000","last_undegraded":"2024-01-31T05:03:39.259739+0000","last_fullsized":"2024-01-31T05:03:39.259739+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.263250+0000","last_change":"2024-01-31T05:03:39.258999+0000","last_active":"2024-01-31T05:03:39.263250+0000","last_peered":"2024-01-31T05:03:39.263250+0000","last_clean":"2024-01-31T05:03:39.263250+0000","last_became_active":"2024-01-31T05:03:39.257976+0000","last_became_peered":"2024-01-31T05:03:39.257976+0000","last_unstale":"2024-01-31T05:03:39.263250+0000","last_undegraded":"2024-01-31T05:03:39.263250+0000","last_fullsized":"2024-01-31T05:03:39.263250+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:24:32.083418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218199+0000","last_change":"2024-01-31T05:03:38.218199+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218199+0000","last_undegraded":"2024-01-31T05:03:38.218199+0000","last_fullsized":"2024-01-31T05:03:38.218199+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.250048+0000","last_change":"2024-01-31T05:03:39.242603+0000","last_active":"2024-01-31T05:03:39.250048+0000","last_peered":"2024-01-31T05:03:39.250048+0000","last_clean":"2024-01-31T05:03:39.250048+0000","last_became_active":"2024-01-31T05:03:39.241443+0000","last_became_peered":"2024-01-31T05:03:39.241443+0000","last_unstale":"2024-01-31T05:03:39.250048+0000","last_undegraded":"2024-01-31T05:03:39.250048+0000","last_fullsized":"2024-01-31T05:03:39.250048+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:28:05.848210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260437+0000","last_change":"2024-01-31T05:03:39.253595+0000","last_active":"2024-01-31T05:03:39.260437+0000","last_peered":"2024-01-31T05:03:39.260437+0000","last_clean":"2024-01-31T05:03:39.260437+0000","last_became_active":"2024-01-31T05:03:39.253076+0000","last_became_peered":"2024-01-31T05:03:39.253076+0000","last_unstale":"2024-01-31T05:03:39.260437+0000","last_undegraded":"2024-01-31T05:03:39.260437+0000","last_fullsized":"2024-01-31T05:03:39.260437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:57:48.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272382+0000","last_change":"2024-01-31T05:03:39.260142+0000","last_active":"2024-01-31T05:03:39.272382+0000","last_peered":"2024-01-31T05:03:39.272382+0000","last_clean":"2024-01-31T05:03:39.272382+0000","last_became_active":"2024-01-31T05:03:39.255918+0000","last_became_peered":"2024-01-31T05:03:39.255918+0000","last_unstale":"2024-01-31T05:03:39.272382+0000","last_undegraded":"2024-01-31T05:03:39.272382+0000","last_fullsized":"2024-01-31T05:03:39.272382+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:03:33.411372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.257215+0000","last_change":"2024-01-31T05:03:39.253163+0000","last_active":"2024-01-31T05:03:39.257215+0000","last_peered":"2024-01-31T05:03:39.257215+0000","last_clean":"2024-01-31T05:03:39.257215+0000","last_became_active":"2024-01-31T05:03:39.252895+0000","last_became_peered":"2024-01-31T05:03:39.252895+0000","last_unstale":"2024-01-31T05:03:39.257215+0000","last_undegraded":"2024-01-31T05:03:39.257215+0000","last_fullsized":"2024-01-31T05:03:39.257215+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218748+0000","last_change":"2024-01-31T05:03:38.218748+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218748+0000","last_undegraded":"2024-01-31T05:03:38.218748+0000","last_fullsized":"2024-01-31T05:03:38.218748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.252464+0000","last_change":"2024-01-31T05:03:39.245654+0000","last_active":"2024-01-31T05:03:39.252464+0000","last_peered":"2024-01-31T05:03:39.252464+0000","last_clean":"2024-01-31T05:03:39.252464+0000","last_became_active":"2024-01-31T05:03:39.242973+0000","last_became_peered":"2024-01-31T05:03:39.242973+0000","last_unstale":"2024-01-31T05:03:39.252464+0000","last_undegraded":"2024-01-31T05:03:39.252464+0000","last_fullsized":"2024-01-31T05:03:39.252464+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.227220+0000","last_change":"2024-01-31T05:03:38.227220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.227220+0000","last_undegraded":"2024-01-31T05:03:38.227220+0000","last_fullsized":"2024-01-31T05:03:38.227220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219977+0000","last_change":"2024-01-31T05:03:38.219977+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219977+0000","last_undegraded":"2024-01-31T05:03:38.219977+0000","last_fullsized":"2024-01-31T05:03:38.219977+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262925+0000","last_change":"2024-01-31T05:03:39.258792+0000","last_active":"2024-01-31T05:03:39.262925+0000","last_peered":"2024-01-31T05:03:39.262925+0000","last_clean":"2024-01-31T05:03:39.262925+0000","last_became_active":"2024-01-31T05:03:39.258531+0000","last_became_peered":"2024-01-31T05:03:39.258531+0000","last_unstale":"2024-01-31T05:03:39.262925+0000","last_undegraded":"2024-01-31T05:03:39.262925+0000","last_fullsized":"2024-01-31T05:03:39.262925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:02:30.641509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243207+0000","last_change":"2024-01-31T05:03:39.236368+0000","last_active":"2024-01-31T05:03:39.243207+0000","last_peered":"2024-01-31T05:03:39.243207+0000","last_clean":"2024-01-31T05:03:39.243207+0000","last_became_active":"2024-01-31T05:03:39.236055+0000","last_became_peered":"2024-01-31T05:03:39.236055+0000","last_unstale":"2024-01-31T05:03:39.243207+0000","last_undegraded":"2024-01-31T05:03:39.243207+0000","last_fullsized":"2024-01-31T05:03:39.243207+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:37.901754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.241385+0000","last_change":"2024-01-31T05:03:39.239349+0000","last_active":"2024-01-31T05:03:39.241385+0000","last_peered":"2024-01-31T05:03:39.241385+0000","last_clean":"2024-01-31T05:03:39.241385+0000","last_became_active":"2024-01-31T05:03:39.238868+0000","last_became_peered":"2024-01-31T05:03:39.238868+0000","last_unstale":"2024-01-31T05:03:39.241385+0000","last_undegraded":"2024-01-31T05:03:39.241385+0000","last_fullsized":"2024-01-31T05:03:39.241385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:55:11.110405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.225923+0000","last_change":"2024-01-31T05:03:38.225923+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.225923+0000","last_undegraded":"2024-01-31T05:03:38.225923+0000","last_fullsized":"2024-01-31T05:03:38.225923+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.276730+0000","last_change":"2024-01-31T05:03:38.235515+0000","last_active":"2024-01-31T05:03:39.276730+0000","last_peered":"2024-01-31T05:03:39.276730+0000","last_clean":"2024-01-31T05:03:39.276730+0000","last_became_active":"2024-01-31T05:03:38.235228+0000","last_became_peered":"2024-01-31T05:03:38.235228+0000","last_unstale":"2024-01-31T05:03:39.276730+0000","last_undegraded":"2024-01-31T05:03:39.276730+0000","last_fullsized":"2024-01-31T05:03:39.276730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:42.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.230764+0000","last_change":"2024-01-31T05:03:38.230764+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.230764+0000","last_undegraded":"2024-01-31T05:03:38.230764+0000","last_fullsized":"2024-01-31T05:03:38.230764+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269887+0000","last_change":"2024-01-31T05:03:38.224888+0000","last_active":"2024-01-31T05:03:39.269887+0000","last_peered":"2024-01-31T05:03:39.269887+0000","last_clean":"2024-01-31T05:03:39.269887+0000","last_became_active":"2024-01-31T05:03:38.223355+0000","last_became_peered":"2024-01-31T05:03:38.223355+0000","last_unstale":"2024-01-31T05:03:39.269887+0000","last_undegraded":"2024-01-31T05:03:39.269887+0000","last_fullsized":"2024-01-31T05:03:39.269887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.226684+0000","last_change":"2024-01-31T05:03:38.226684+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.226684+0000","last_undegraded":"2024-01-31T05:03:38.226684+0000","last_fullsized":"2024-01-31T05:03:38.226684+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.239333+0000","last_change":"2024-01-31T05:03:38.234777+0000","last_active":"2024-01-31T05:03:38.239333+0000","last_peered":"2024-01-31T05:03:38.239333+0000","last_clean":"2024-01-31T05:03:38.239333+0000","last_became_active":"2024-01-31T05:03:38.234493+0000","last_became_peered":"2024-01-31T05:03:38.234493+0000","last_unstale":"2024-01-31T05:03:38.239333+0000","last_undegraded":"2024-01-31T05:03:38.239333+0000","last_fullsized":"2024-01-31T05:03:38.239333+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:48:31.432282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224220+0000","last_change":"2024-01-31T05:03:38.224220+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224220+0000","last_undegraded":"2024-01-31T05:03:38.224220+0000","last_fullsized":"2024-01-31T05:03:38.224220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277804+0000","last_change":"2024-01-31T05:03:38.221765+0000","last_active":"2024-01-31T05:03:39.277804+0000","last_peered":"2024-01-31T05:03:39.277804+0000","last_clean":"2024-01-31T05:03:39.277804+0000","last_became_active":"2024-01-31T05:03:38.221534+0000","last_became_peered":"2024-01-31T05:03:38.221534+0000","last_unstale":"2024-01-31T05:03:39.277804+0000","last_undegraded":"2024-01-31T05:03:39.277804+0000","last_fullsized":"2024-01-31T05:03:39.277804+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:22.374866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.210993+0000","last_change":"2024-01-31T05:03:38.210993+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.210993+0000","last_undegraded":"2024-01-31T05:03:38.210993+0000","last_fullsized":"2024-01-31T05:03:38.210993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277781+0000","last_change":"2024-01-31T05:03:38.226696+0000","last_active":"2024-01-31T05:03:39.277781+0000","last_peered":"2024-01-31T05:03:39.277781+0000","last_clean":"2024-01-31T05:03:39.277781+0000","last_became_active":"2024-01-31T05:03:38.226440+0000","last_became_peered":"2024-01-31T05:03:38.226440+0000","last_unstale":"2024-01-31T05:03:39.277781+0000","last_undegraded":"2024-01-31T05:03:39.277781+0000","last_fullsized":"2024-01-31T05:03:39.277781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.243539+0000","last_change":"2024-01-31T05:03:39.240337+0000","last_active":"2024-01-31T05:03:39.243539+0000","last_peered":"2024-01-31T05:03:39.243539+0000","last_clean":"2024-01-31T05:03:39.243539+0000","last_became_active":"2024-01-31T05:03:39.240029+0000","last_became_peered":"2024-01-31T05:03:39.240029+0000","last_unstale":"2024-01-31T05:03:39.243539+0000","last_undegraded":"2024-01-31T05:03:39.243539+0000","last_fullsized":"2024-01-31T05:03:39.243539+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:24:32.372349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240709+0000","last_change":"2024-01-31T05:03:38.239050+0000","last_active":"2024-01-31T05:03:38.240709+0000","last_peered":"2024-01-31T05:03:38.240709+0000","last_clean":"2024-01-31T05:03:38.240709+0000","last_became_active":"2024-01-31T05:03:38.238746+0000","last_became_peered":"2024-01-31T05:03:38.238746+0000","last_unstale":"2024-01-31T05:03:38.240709+0000","last_undegraded":"2024-01-31T05:03:38.240709+0000","last_fullsized":"2024-01-31T05:03:38.240709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:37:59.345660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260998+0000","last_change":"2024-01-31T05:03:39.244622+0000","last_active":"2024-01-31T05:03:39.260998+0000","last_peered":"2024-01-31T05:03:39.260998+0000","last_clean":"2024-01-31T05:03:39.260998+0000","last_became_active":"2024-01-31T05:03:39.243824+0000","last_became_peered":"2024-01-31T05:03:39.243824+0000","last_unstale":"2024-01-31T05:03:39.260998+0000","last_undegraded":"2024-01-31T05:03:39.260998+0000","last_fullsized":"2024-01-31T05:03:39.260998+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234692+0000","last_change":"2024-01-31T05:03:38.230982+0000","last_active":"2024-01-31T05:03:39.234692+0000","last_peered":"2024-01-31T05:03:39.234692+0000","last_clean":"2024-01-31T05:03:39.234692+0000","last_became_active":"2024-01-31T05:03:38.230687+0000","last_became_peered":"2024-01-31T05:03:38.230687+0000","last_unstale":"2024-01-31T05:03:39.234692+0000","last_undegraded":"2024-01-31T05:03:39.234692+0000","last_fullsized":"2024-01-31T05:03:39.234692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.220574+0000","last_change":"2024-01-31T05:03:38.220574+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.220574+0000","last_undegraded":"2024-01-31T05:03:38.220574+0000","last_fullsized":"2024-01-31T05:03:38.220574+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240721+0000","last_change":"2024-01-31T05:03:38.236445+0000","last_active":"2024-01-31T05:03:38.240721+0000","last_peered":"2024-01-31T05:03:38.240721+0000","last_clean":"2024-01-31T05:03:38.240721+0000","last_became_active":"2024-01-31T05:03:38.236193+0000","last_became_peered":"2024-01-31T05:03:38.236193+0000","last_unstale":"2024-01-31T05:03:38.240721+0000","last_undegraded":"2024-01-31T05:03:38.240721+0000","last_fullsized":"2024-01-31T05:03:38.240721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218290+0000","last_change":"2024-01-31T05:03:38.218290+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218290+0000","last_undegraded":"2024-01-31T05:03:38.218290+0000","last_fullsized":"2024-01-31T05:03:38.218290+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.236414+0000","last_change":"2024-01-31T05:03:38.223052+0000","last_active":"2024-01-31T05:03:39.236414+0000","last_peered":"2024-01-31T05:03:39.236414+0000","last_clean":"2024-01-31T05:03:39.236414+0000","last_became_active":"2024-01-31T05:03:38.222742+0000","last_became_peered":"2024-01-31T05:03:38.222742+0000","last_unstale":"2024-01-31T05:03:39.236414+0000","last_undegraded":"2024-01-31T05:03:39.236414+0000","last_fullsized":"2024-01-31T05:03:39.236414+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267542+0000","last_change":"2024-01-31T05:03:39.252433+0000","last_active":"2024-01-31T05:03:39.267542+0000","last_peered":"2024-01-31T05:03:39.267542+0000","last_clean":"2024-01-31T05:03:39.267542+0000","last_became_active":"2024-01-31T05:03:39.249986+0000","last_became_peered":"2024-01-31T05:03:39.249986+0000","last_unstale":"2024-01-31T05:03:39.267542+0000","last_undegraded":"2024-01-31T05:03:39.267542+0000","last_fullsized":"2024-01-31T05:03:39.267542+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:06.790339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240724+0000","last_change":"2024-01-31T05:03:38.237692+0000","last_active":"2024-01-31T05:03:38.240724+0000","last_peered":"2024-01-31T05:03:38.240724+0000","last_clean":"2024-01-31T05:03:38.240724+0000","last_became_active":"2024-01-31T05:03:38.236259+0000","last_became_peered":"2024-01-31T05:03:38.236259+0000","last_unstale":"2024-01-31T05:03:38.240724+0000","last_undegraded":"2024-01-31T05:03:38.240724+0000","last_fullsized":"2024-01-31T05:03:38.240724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:06:57.676616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219356+0000","last_change":"2024-01-31T05:03:38.219356+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219356+0000","last_undegraded":"2024-01-31T05:03:38.219356+0000","last_fullsized":"2024-01-31T05:03:38.219356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278537+0000","last_change":"2024-01-31T05:03:38.225581+0000","last_active":"2024-01-31T05:03:39.278537+0000","last_peered":"2024-01-31T05:03:39.278537+0000","last_clean":"2024-01-31T05:03:39.278537+0000","last_became_active":"2024-01-31T05:03:38.225059+0000","last_became_peered":"2024-01-31T05:03:38.225059+0000","last_unstale":"2024-01-31T05:03:39.278537+0000","last_undegraded":"2024-01-31T05:03:39.278537+0000","last_fullsized":"2024-01-31T05:03:39.278537+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:03:20.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259585+0000","last_change":"2024-01-31T05:03:39.250182+0000","last_active":"2024-01-31T05:03:39.259585+0000","last_peered":"2024-01-31T05:03:39.259585+0000","last_clean":"2024-01-31T05:03:39.259585+0000","last_became_active":"2024-01-31T05:03:39.249917+0000","last_became_peered":"2024-01-31T05:03:39.249917+0000","last_unstale":"2024-01-31T05:03:39.259585+0000","last_undegraded":"2024-01-31T05:03:39.259585+0000","last_fullsized":"2024-01-31T05:03:39.259585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:42:20.611592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.235944+0000","last_change":"2024-01-31T05:03:38.230550+0000","last_active":"2024-01-31T05:03:38.235944+0000","last_peered":"2024-01-31T05:03:38.235944+0000","last_clean":"2024-01-31T05:03:38.235944+0000","last_became_active":"2024-01-31T05:03:38.230284+0000","last_became_peered":"2024-01-31T05:03:38.230284+0000","last_unstale":"2024-01-31T05:03:38.235944+0000","last_undegraded":"2024-01-31T05:03:38.235944+0000","last_fullsized":"2024-01-31T05:03:38.235944+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.249515+0000","last_change":"2024-01-31T05:03:39.246221+0000","last_active":"2024-01-31T05:03:39.249515+0000","last_peered":"2024-01-31T05:03:39.249515+0000","last_clean":"2024-01-31T05:03:39.249515+0000","last_became_active":"2024-01-31T05:03:39.244574+0000","last_became_peered":"2024-01-31T05:03:39.244574+0000","last_unstale":"2024-01-31T05:03:39.249515+0000","last_undegraded":"2024-01-31T05:03:39.249515+0000","last_fullsized":"2024-01-31T05:03:39.249515+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240559+0000","last_change":"2024-01-31T05:03:38.237263+0000","last_active":"2024-01-31T05:03:38.240559+0000","last_peered":"2024-01-31T05:03:38.240559+0000","last_clean":"2024-01-31T05:03:38.240559+0000","last_became_active":"2024-01-31T05:03:38.236868+0000","last_became_peered":"2024-01-31T05:03:38.236868+0000","last_unstale":"2024-01-31T05:03:38.240559+0000","last_undegraded":"2024-01-31T05:03:38.240559+0000","last_fullsized":"2024-01-31T05:03:38.240559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:22:50.265508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259275+0000","last_change":"2024-01-31T05:03:39.242937+0000","last_active":"2024-01-31T05:03:39.259275+0000","last_peered":"2024-01-31T05:03:39.259275+0000","last_clean":"2024-01-31T05:03:39.259275+0000","last_became_active":"2024-01-31T05:03:39.241602+0000","last_became_peered":"2024-01-31T05:03:39.241602+0000","last_unstale":"2024-01-31T05:03:39.259275+0000","last_undegraded":"2024-01-31T05:03:39.259275+0000","last_fullsized":"2024-01-31T05:03:39.259275+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233462+0000","last_change":"2024-01-31T05:03:38.225322+0000","last_active":"2024-01-31T05:03:39.233462+0000","last_peered":"2024-01-31T05:03:39.233462+0000","last_clean":"2024-01-31T05:03:39.233462+0000","last_became_active":"2024-01-31T05:03:38.224422+0000","last_became_peered":"2024-01-31T05:03:38.224422+0000","last_unstale":"2024-01-31T05:03:39.233462+0000","last_undegraded":"2024-01-31T05:03:39.233462+0000","last_fullsized":"2024-01-31T05:03:39.233462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:44.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269779+0000","last_change":"2024-01-31T05:03:39.258087+0000","last_active":"2024-01-31T05:03:39.269779+0000","last_peered":"2024-01-31T05:03:39.269779+0000","last_clean":"2024-01-31T05:03:39.269779+0000","last_became_active":"2024-01-31T05:03:39.257724+0000","last_became_peered":"2024-01-31T05:03:39.257724+0000","last_unstale":"2024-01-31T05:03:39.269779+0000","last_undegraded":"2024-01-31T05:03:39.269779+0000","last_fullsized":"2024-01-31T05:03:39.269779+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:35:49.109502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.231884+0000","last_change":"2024-01-31T05:03:38.219525+0000","last_active":"2024-01-31T05:03:39.231884+0000","last_peered":"2024-01-31T05:03:39.231884+0000","last_clean":"2024-01-31T05:03:39.231884+0000","last_became_active":"2024-01-31T05:03:38.219302+0000","last_became_peered":"2024-01-31T05:03:38.219302+0000","last_unstale":"2024-01-31T05:03:39.231884+0000","last_undegraded":"2024-01-31T05:03:39.231884+0000","last_fullsized":"2024-01-31T05:03:39.231884+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:08.160317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.260419+0000","last_change":"2024-01-31T05:03:39.253326+0000","last_active":"2024-01-31T05:03:39.260419+0000","last_peered":"2024-01-31T05:03:39.260419+0000","last_clean":"2024-01-31T05:03:39.260419+0000","last_became_active":"2024-01-31T05:03:39.252687+0000","last_became_peered":"2024-01-31T05:03:39.252687+0000","last_unstale":"2024-01-31T05:03:39.260419+0000","last_undegraded":"2024-01-31T05:03:39.260419+0000","last_fullsized":"2024-01-31T05:03:39.260419+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:23:14.519189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.278703+0000","last_change":"2024-01-31T05:03:38.224819+0000","last_active":"2024-01-31T05:03:39.278703+0000","last_peered":"2024-01-31T05:03:39.278703+0000","last_clean":"2024-01-31T05:03:39.278703+0000","last_became_active":"2024-01-31T05:03:38.224555+0000","last_became_peered":"2024-01-31T05:03:38.224555+0000","last_unstale":"2024-01-31T05:03:39.278703+0000","last_undegraded":"2024-01-31T05:03:39.278703+0000","last_fullsized":"2024-01-31T05:03:39.278703+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:15:12.301634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266298+0000","last_change":"2024-01-31T05:03:39.249020+0000","last_active":"2024-01-31T05:03:39.266298+0000","last_peered":"2024-01-31T05:03:39.266298+0000","last_clean":"2024-01-31T05:03:39.266298+0000","last_became_active":"2024-01-31T05:03:39.246977+0000","last_became_peered":"2024-01-31T05:03:39.246977+0000","last_unstale":"2024-01-31T05:03:39.266298+0000","last_undegraded":"2024-01-31T05:03:39.266298+0000","last_fullsized":"2024-01-31T05:03:39.266298+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.272054+0000","last_change":"2024-01-31T05:03:38.239609+0000","last_active":"2024-01-31T05:03:39.272054+0000","last_peered":"2024-01-31T05:03:39.272054+0000","last_clean":"2024-01-31T05:03:39.272054+0000","last_became_active":"2024-01-31T05:03:38.238339+0000","last_became_peered":"2024-01-31T05:03:38.238339+0000","last_unstale":"2024-01-31T05:03:39.272054+0000","last_undegraded":"2024-01-31T05:03:39.272054+0000","last_fullsized":"2024-01-31T05:03:39.272054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267510+0000","last_change":"2024-01-31T05:03:39.250705+0000","last_active":"2024-01-31T05:03:39.267510+0000","last_peered":"2024-01-31T05:03:39.267510+0000","last_clean":"2024-01-31T05:03:39.267510+0000","last_became_active":"2024-01-31T05:03:39.248093+0000","last_became_peered":"2024-01-31T05:03:39.248093+0000","last_unstale":"2024-01-31T05:03:39.267510+0000","last_undegraded":"2024-01-31T05:03:39.267510+0000","last_fullsized":"2024-01-31T05:03:39.267510+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240743+0000","last_change":"2024-01-31T05:03:38.237901+0000","last_active":"2024-01-31T05:03:38.240743+0000","last_peered":"2024-01-31T05:03:38.240743+0000","last_clean":"2024-01-31T05:03:38.240743+0000","last_became_active":"2024-01-31T05:03:38.237413+0000","last_became_peered":"2024-01-31T05:03:38.237413+0000","last_unstale":"2024-01-31T05:03:38.240743+0000","last_undegraded":"2024-01-31T05:03:38.240743+0000","last_fullsized":"2024-01-31T05:03:38.240743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:58:23.675405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.262915+0000","last_change":"2024-01-31T05:03:39.257158+0000","last_active":"2024-01-31T05:03:39.262915+0000","last_peered":"2024-01-31T05:03:39.262915+0000","last_clean":"2024-01-31T05:03:39.262915+0000","last_became_active":"2024-01-31T05:03:39.256870+0000","last_became_peered":"2024-01-31T05:03:39.256870+0000","last_unstale":"2024-01-31T05:03:39.262915+0000","last_undegraded":"2024-01-31T05:03:39.262915+0000","last_fullsized":"2024-01-31T05:03:39.262915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.277453+0000","last_change":"2024-01-31T05:03:38.230887+0000","last_active":"2024-01-31T05:03:39.277453+0000","last_peered":"2024-01-31T05:03:39.277453+0000","last_clean":"2024-01-31T05:03:39.277453+0000","last_became_active":"2024-01-31T05:03:38.229073+0000","last_became_peered":"2024-01-31T05:03:38.229073+0000","last_unstale":"2024-01-31T05:03:39.277453+0000","last_undegraded":"2024-01-31T05:03:39.277453+0000","last_fullsized":"2024-01-31T05:03:39.277453+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.259604+0000","last_change":"2024-01-31T05:03:39.254309+0000","last_active":"2024-01-31T05:03:39.259604+0000","last_peered":"2024-01-31T05:03:39.259604+0000","last_clean":"2024-01-31T05:03:39.259604+0000","last_became_active":"2024-01-31T05:03:39.254044+0000","last_became_peered":"2024-01-31T05:03:39.254044+0000","last_unstale":"2024-01-31T05:03:39.259604+0000","last_undegraded":"2024-01-31T05:03:39.259604+0000","last_fullsized":"2024-01-31T05:03:39.259604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:33:55.111654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240233+0000","last_change":"2024-01-31T05:03:38.234624+0000","last_active":"2024-01-31T05:03:38.240233+0000","last_peered":"2024-01-31T05:03:38.240233+0000","last_clean":"2024-01-31T05:03:38.240233+0000","last_became_active":"2024-01-31T05:03:38.234296+0000","last_became_peered":"2024-01-31T05:03:38.234296+0000","last_unstale":"2024-01-31T05:03:38.240233+0000","last_undegraded":"2024-01-31T05:03:38.240233+0000","last_fullsized":"2024-01-31T05:03:38.240233+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:25:34.665886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.269815+0000","last_change":"2024-01-31T05:03:39.260082+0000","last_active":"2024-01-31T05:03:39.269815+0000","last_peered":"2024-01-31T05:03:39.269815+0000","last_clean":"2024-01-31T05:03:39.269815+0000","last_became_active":"2024-01-31T05:03:39.256821+0000","last_became_peered":"2024-01-31T05:03:39.256821+0000","last_unstale":"2024-01-31T05:03:39.269815+0000","last_undegraded":"2024-01-31T05:03:39.269815+0000","last_fullsized":"2024-01-31T05:03:39.269815+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:39:51.986668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.274034+0000","last_change":"2024-01-31T05:03:38.234981+0000","last_active":"2024-01-31T05:03:39.274034+0000","last_peered":"2024-01-31T05:03:39.274034+0000","last_clean":"2024-01-31T05:03:39.274034+0000","last_became_active":"2024-01-31T05:03:38.234678+0000","last_became_peered":"2024-01-31T05:03:38.234678+0000","last_unstale":"2024-01-31T05:03:39.274034+0000","last_undegraded":"2024-01-31T05:03:39.274034+0000","last_fullsized":"2024-01-31T05:03:39.274034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:08:18.852860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.218772+0000","last_change":"2024-01-31T05:03:38.218772+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.218772+0000","last_undegraded":"2024-01-31T05:03:38.218772+0000","last_fullsized":"2024-01-31T05:03:38.218772+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238837+0000","last_change":"2024-01-31T05:03:38.230732+0000","last_active":"2024-01-31T05:03:38.238837+0000","last_peered":"2024-01-31T05:03:38.238837+0000","last_clean":"2024-01-31T05:03:38.238837+0000","last_became_active":"2024-01-31T05:03:38.229403+0000","last_became_peered":"2024-01-31T05:03:38.229403+0000","last_unstale":"2024-01-31T05:03:38.238837+0000","last_undegraded":"2024-01-31T05:03:38.238837+0000","last_fullsized":"2024-01-31T05:03:38.238837+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:00:00.029942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.216599+0000","last_change":"2024-01-31T05:03:38.216599+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.216599+0000","last_undegraded":"2024-01-31T05:03:38.216599+0000","last_fullsized":"2024-01-31T05:03:38.216599+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232247+0000","last_change":"2024-01-31T05:03:38.230238+0000","last_active":"2024-01-31T05:03:39.232247+0000","last_peered":"2024-01-31T05:03:39.232247+0000","last_clean":"2024-01-31T05:03:39.232247+0000","last_became_active":"2024-01-31T05:03:38.229961+0000","last_became_peered":"2024-01-31T05:03:38.229961+0000","last_unstale":"2024-01-31T05:03:39.232247+0000","last_undegraded":"2024-01-31T05:03:39.232247+0000","last_fullsized":"2024-01-31T05:03:39.232247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.266259+0000","last_change":"2024-01-31T05:03:39.248707+0000","last_active":"2024-01-31T05:03:39.266259+0000","last_peered":"2024-01-31T05:03:39.266259+0000","last_clean":"2024-01-31T05:03:39.266259+0000","last_became_active":"2024-01-31T05:03:39.246545+0000","last_became_peered":"2024-01-31T05:03:39.246545+0000","last_unstale":"2024-01-31T05:03:39.266259+0000","last_undegraded":"2024-01-31T05:03:39.266259+0000","last_fullsized":"2024-01-31T05:03:39.266259+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:54:29.905100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.271942+0000","last_change":"2024-01-31T05:03:38.232128+0000","last_active":"2024-01-31T05:03:39.271942+0000","last_peered":"2024-01-31T05:03:39.271942+0000","last_clean":"2024-01-31T05:03:39.271942+0000","last_became_active":"2024-01-31T05:03:38.231862+0000","last_became_peered":"2024-01-31T05:03:38.231862+0000","last_unstale":"2024-01-31T05:03:39.271942+0000","last_undegraded":"2024-01-31T05:03:39.271942+0000","last_fullsized":"2024-01-31T05:03:39.271942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.267560+0000","last_change":"2024-01-31T05:03:39.252646+0000","last_active":"2024-01-31T05:03:39.267560+0000","last_peered":"2024-01-31T05:03:39.267560+0000","last_clean":"2024-01-31T05:03:39.267560+0000","last_became_active":"2024-01-31T05:03:39.244259+0000","last_became_peered":"2024-01-31T05:03:39.244259+0000","last_unstale":"2024-01-31T05:03:39.267560+0000","last_undegraded":"2024-01-31T05:03:39.267560+0000","last_fullsized":"2024-01-31T05:03:39.267560+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:01:30.708807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240725+0000","last_change":"2024-01-31T05:03:38.239097+0000","last_active":"2024-01-31T05:03:38.240725+0000","last_peered":"2024-01-31T05:03:38.240725+0000","last_clean":"2024-01-31T05:03:38.240725+0000","last_became_active":"2024-01-31T05:03:38.238757+0000","last_became_peered":"2024-01-31T05:03:38.238757+0000","last_unstale":"2024-01-31T05:03:38.240725+0000","last_undegraded":"2024-01-31T05:03:38.240725+0000","last_fullsized":"2024-01-31T05:03:38.240725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:18:04.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246623+0000","last_change":"2024-01-31T05:03:39.244177+0000","last_active":"2024-01-31T05:03:39.246623+0000","last_peered":"2024-01-31T05:03:39.246623+0000","last_clean":"2024-01-31T05:03:39.246623+0000","last_became_active":"2024-01-31T05:03:39.243841+0000","last_became_peered":"2024-01-31T05:03:39.243841+0000","last_unstale":"2024-01-31T05:03:39.246623+0000","last_undegraded":"2024-01-31T05:03:39.246623+0000","last_fullsized":"2024-01-31T05:03:39.246623+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:33:44.722771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.234411+0000","last_change":"2024-01-31T05:03:38.224802+0000","last_active":"2024-01-31T05:03:39.234411+0000","last_peered":"2024-01-31T05:03:39.234411+0000","last_clean":"2024-01-31T05:03:39.234411+0000","last_became_active":"2024-01-31T05:03:38.223989+0000","last_became_peered":"2024-01-31T05:03:38.223989+0000","last_unstale":"2024-01-31T05:03:39.234411+0000","last_undegraded":"2024-01-31T05:03:39.234411+0000","last_fullsized":"2024-01-31T05:03:39.234411+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.219620+0000","last_change":"2024-01-31T05:03:38.219620+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.219620+0000","last_undegraded":"2024-01-31T05:03:38.219620+0000","last_fullsized":"2024-01-31T05:03:38.219620+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.230405+0000","last_change":"2024-01-31T05:03:38.224866+0000","last_active":"2024-01-31T05:03:39.230405+0000","last_peered":"2024-01-31T05:03:39.230405+0000","last_clean":"2024-01-31T05:03:39.230405+0000","last_became_active":"2024-01-31T05:03:38.224437+0000","last_became_peered":"2024-01-31T05:03:38.224437+0000","last_unstale":"2024-01-31T05:03:39.230405+0000","last_undegraded":"2024-01-31T05:03:39.230405+0000","last_fullsized":"2024-01-31T05:03:39.230405+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.224396+0000","last_change":"2024-01-31T05:03:38.224396+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.224396+0000","last_undegraded":"2024-01-31T05:03:38.224396+0000","last_fullsized":"2024-01-31T05:03:38.224396+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.225219+0000","last_change":"2024-01-31T05:03:38.236959+0000","last_active":"2024-01-31T05:03:39.225219+0000","last_peered":"2024-01-31T05:03:39.225219+0000","last_clean":"2024-01-31T05:03:39.225219+0000","last_became_active":"2024-01-31T05:03:38.235737+0000","last_became_peered":"2024-01-31T05:03:38.235737+0000","last_unstale":"2024-01-31T05:03:39.225219+0000","last_undegraded":"2024-01-31T05:03:39.225219+0000","last_fullsized":"2024-01-31T05:03:39.225219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:44.258646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.217657+0000","last_change":"2024-01-31T05:03:38.217657+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.217657+0000","last_undegraded":"2024-01-31T05:03:38.217657+0000","last_fullsized":"2024-01-31T05:03:38.217657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.233182+0000","last_change":"2024-01-31T05:03:38.231189+0000","last_active":"2024-01-31T05:03:39.233182+0000","last_peered":"2024-01-31T05:03:39.233182+0000","last_clean":"2024-01-31T05:03:39.233182+0000","last_became_active":"2024-01-31T05:03:38.230998+0000","last_became_peered":"2024-01-31T05:03:38.230998+0000","last_unstale":"2024-01-31T05:03:39.233182+0000","last_undegraded":"2024-01-31T05:03:39.233182+0000","last_fullsized":"2024-01-31T05:03:39.233182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.213769+0000","last_change":"2024-01-31T05:03:38.213769+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.213769+0000","last_undegraded":"2024-01-31T05:03:38.213769+0000","last_fullsized":"2024-01-31T05:03:38.213769+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.238725+0000","last_change":"2024-01-31T05:03:38.235803+0000","last_active":"2024-01-31T05:03:38.238725+0000","last_peered":"2024-01-31T05:03:38.238725+0000","last_clean":"2024-01-31T05:03:38.238725+0000","last_became_active":"2024-01-31T05:03:38.234927+0000","last_became_peered":"2024-01-31T05:03:38.234927+0000","last_unstale":"2024-01-31T05:03:38.238725+0000","last_undegraded":"2024-01-31T05:03:38.238725+0000","last_fullsized":"2024-01-31T05:03:38.238725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:42:14.881712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.246903+0000","last_change":"2024-01-31T05:03:39.243770+0000","last_active":"2024-01-31T05:03:39.246903+0000","last_peered":"2024-01-31T05:03:39.246903+0000","last_clean":"2024-01-31T05:03:39.246903+0000","last_became_active":"2024-01-31T05:03:39.243396+0000","last_became_peered":"2024-01-31T05:03:39.243396+0000","last_unstale":"2024-01-31T05:03:39.246903+0000","last_undegraded":"2024-01-31T05:03:39.246903+0000","last_fullsized":"2024-01-31T05:03:39.246903+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:13:38.734564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.240286+0000","last_change":"2024-01-31T05:03:38.235205+0000","last_active":"2024-01-31T05:03:38.240286+0000","last_peered":"2024-01-31T05:03:38.240286+0000","last_clean":"2024-01-31T05:03:38.240286+0000","last_became_active":"2024-01-31T05:03:38.234627+0000","last_became_peered":"2024-01-31T05:03:38.234627+0000","last_unstale":"2024-01-31T05:03:38.240286+0000","last_undegraded":"2024-01-31T05:03:38.240286+0000","last_fullsized":"2024-01-31T05:03:38.240286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-01-31T05:03:38.212184+0000","last_change":"2024-01-31T05:03:38.212184+0000","last_active":"2024-01-31T05:03:38.190665+0000","last_peered":"2024-01-31T05:03:38.190665+0000","last_clean":"2024-01-31T05:03:38.190665+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-31T05:03:38.212184+0000","last_undegraded":"2024-01-31T05:03:38.212184+0000","last_fullsized":"2024-01-31T05:03:38.212184+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-01-31T05:03:38.236723+0000","last_change":"2024-01-31T05:03:38.233879+0000","last_active":"2024-01-31T05:03:38.236723+0000","last_peered":"2024-01-31T05:03:38.236723+0000","last_clean":"2024-01-31T05:03:38.236723+0000","last_became_active":"2024-01-31T05:03:38.233495+0000","last_became_peered":"2024-01-31T05:03:38.233495+0000","last_unstale":"2024-01-31T05:03:38.236723+0000","last_undegraded":"2024-01-31T05:03:38.236723+0000","last_fullsized":"2024-01-31T05:03:38.236723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:39:50.967816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-01-31T05:03:39.232380+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:39.232380+0000","last_peered":"2024-01-31T05:03:39.232380+0000","last_clean":"2024-01-31T05:03:39.232380+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:39.232380+0000","last_undegraded":"2024-01-31T05:03:39.232380+0000","last_fullsized":"2024-01-31T05:03:39.232380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":764,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":275170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.64800000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0149999999999999,"15min":1.0009999999999999},"min":{"1min":0.68200000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5369999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.1259999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.099,"15min":1.089},"min":{"1min":0.72799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6719999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.244,"15min":1.256},"min":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6259999999999999,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.872}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.175,"15min":1.1799999999999999},"min":{"1min":0.747,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.4930000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.29,"15min":1.292},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.512,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.016}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2230000000000001,"15min":1.2150000000000001},"min":{"1min":0.91900000000000004,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.609,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2190000000000001,"15min":1.224},"min":{"1min":0.79200000000000004,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6120000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.179}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.3420000000000001,"15min":1.3480000000000001},"min":{"1min":0.98499999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.609,"5min":2.25,"15min":2.25},"last":1.349},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.3580000000000001,"15min":1.3740000000000001},"min":{"1min":1.004,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.573,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3180000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.472,"15min":1.4750000000000001},"min":{"1min":1.0109999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.762,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.85999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5960000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.48,"15min":1.4890000000000001},"min":{"1min":1.0089999999999999,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":2.1240000000000001,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5529999999999999,"15min":1.5609999999999999},"min":{"1min":1.2709999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.726,"5min":2.629,"15min":2.629},"last":1.6359999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":647168,"data_stored":282881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0329999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3240000000000001,"5min":1.476,"15min":1.476},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.05,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.876}]},{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.78600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.587,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1200000000000001,"15min":1.131},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5369999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.48}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.6950000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.569},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1799999999999999,"15min":1.1839999999999999},"min":{"1min":0.80400000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.421,"5min":1.883,"15min":1.883},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.381,"15min":1.395},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.72,"5min":1.73,"15min":1.73},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.95599999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0830000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.6259999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.878,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.611,"5min":2.032,"15min":2.032},"last":1.012},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.585},"min":{"1min":1.0780000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.984,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.165}]},{"osd":6,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5209999999999999,"15min":1.5329999999999999},"min":{"1min":0.96899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.972,"5min":2.335,"15min":2.335},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.468,"5min":1.5860000000000001,"15min":1.6060000000000001},"min":{"1min":1.2609999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.768,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.115}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.599,"15min":1.6160000000000001},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8620000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.071},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.637,"15min":1.659},"min":{"1min":1.0920000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.891,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.7350000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":282355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.151},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1919999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":1.032,"15min":1.0840000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1439999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.121,"15min":1.1779999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4910000000000001,"5min":5.617,"15min":5.617},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.2330000000000001,"15min":1.2989999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.94599999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.292,"15min":1.3680000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.484,"15min":1.5720000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4930000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.093}]},{"osd":4,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.6220000000000001,"15min":1.7230000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.9990000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.921},{"interface":"front","average":{"1min":1.141,"5min":1.621,"15min":1.7010000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.696,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.2130000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.639,"15min":1.7230000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9339999999999999,"5min":6.048,"15min":6.048},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.7430000000000001,"15min":1.831},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.952,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.804}]},{"osd":6,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.6739999999999999,"15min":1.744},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.024,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.8759999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.657,"15min":1.748},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5700000000000001,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.6459999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.6539999999999999,"15min":1.74},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.802,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":2.0499999999999998},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.8380000000000001,"15min":1.946},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.73,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.5609999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":284037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.88500000000000001,"15min":0.875},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.667,"5min":1.73,"15min":1.73},"last":1.4390000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.88900000000000001,"15min":0.90300000000000002},"min":{"1min":0.47499999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.472,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.046},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.60499999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.099}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.93000000000000005,"15min":0.92300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4330000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.23},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0780000000000001,"15min":1.087},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":1.798,"15min":1.798},"last":1.387}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1850000000000001,"15min":1.181},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5780000000000001,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3220000000000001,"15min":1.331},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0059999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.321}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.232,"15min":1.2410000000000001},"min":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.585,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.472},{"interface":"front","average":{"1min":1.26,"5min":1.3100000000000001,"15min":1.3180000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8220000000000001,"5min":2.52,"15min":2.52},"last":1.494}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4159999999999999,"15min":1.419},"min":{"1min":1.0189999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.1320000000000001,"5min":2.992,"15min":2.992},"last":1.663},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.3939999999999999,"15min":1.4079999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.6879999999999999,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.397,"15min":1.403},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.8700000000000001,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.411,"15min":1.4199999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.869,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.619}]}]},{"osd":1,"up_from":15,"seq":64424509478,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1236992,"data_stored":873215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":1.026,"15min":1.0820000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.272,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.90100000000000002,"15min":0.92100000000000004},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.149,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4890000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.98399999999999999,"15min":1.0289999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0229999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.131,"15min":1.1839999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.4099999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":1.0109999999999999,"15min":1.0449999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4419999999999999,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.1639999999999999,"15min":1.222},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.294,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.028}]},{"osd":4,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.4550000000000001,"15min":1.5089999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5940000000000001,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.091},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.345,"15min":1.379},"min":{"1min":0.80900000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4630000000000001,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.4059999999999999,"15min":1.4470000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.4650000000000001,"5min":2.722,"15min":2.722},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.409,"15min":1.4550000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.587,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.5189999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4790000000000001,"15min":1.538},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6439999999999999,"5min":2.577,"15min":2.577},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.415,"15min":1.4510000000000001},"min":{"1min":0.85699999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5609999999999999,"5min":2.794,"15min":2.794},"last":1.651}]},{"osd":7,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.4930000000000001,"15min":1.5449999999999999},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.647,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.5640000000000001,"15min":1.615},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.5489999999999999,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.581}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.135,"15min":1.151},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.091},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.552,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1339999999999999,"15min":1.131},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5589999999999999,"5min":1.573,"15min":1.573},"last":1.242},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.73799999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5349999999999999,"5min":1.976,"15min":1.976},"last":1.1459999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7470000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.266,"15min":1.28},"min":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.544,"5min":1.873,"15min":1.873},"last":1.321}]},{"osd":3,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3580000000000001,"15min":1.375},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.7050000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.335,"15min":1.339},"min":{"1min":0.95899999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.4399999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.254,"15min":1.2529999999999999},"min":{"1min":0.79300000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6890000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":0.94399999999999995,"5min":0.628,"15min":0.628},"max":{"1min":2.0499999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.206}]},{"osd":5,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4430000000000001,"15min":1.454},"min":{"1min":1.0049999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.679,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.379},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.593,"15min":1.6040000000000001},"min":{"1min":1.1990000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0990000000000002,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.462}]},{"osd":7,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5620000000000001,"15min":1.5820000000000001},"min":{"1min":0.89400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9530000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6719999999999999,"15min":1.6839999999999999},"min":{"1min":1.246,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9990000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.409}]}]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":865538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.038,"15min":1.022},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.792}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1279999999999999,"15min":1.141},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.2030000000000001,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5720000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.3320000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.206,"15min":1.222},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.663,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.29},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.296,"15min":1.3089999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.647,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.49}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.304,"15min":1.306},"min":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.704,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.389,"15min":1.399},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.712,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.5309999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":0.70399999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.0139999999999998,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.91400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.792,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4259999999999999,"15min":1.4099999999999999},"min":{"1min":0.93500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":1.494},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.851,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5860000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.948,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.0029999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5219999999999998,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.6240000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542183,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":268575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0389999999999999,"15min":1.0489999999999999},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.0680000000000001,"5min":3.282,"15min":3.282},"last":0.627}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.075,"5min":1.774,"15min":1.774},"last":0.79600000000000004}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.103,"5min":1.091,"15min":1.089},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.2250000000000001},"min":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4370000000000001,"15min":1.4399999999999999},"min":{"1min":0.93999999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.8500000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.097}]},{"osd":5,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":1.492,"15min":1.482},"min":{"1min":0.85399999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":5.75,"5min":5.75,"15min":5.75},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.468,"15min":1.4750000000000001},"min":{"1min":0.86499999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":4.8799999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.169}]},{"osd":6,"last update":"Wed Jan 31 05:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.355,"15min":1.3300000000000001},"min":{"1min":0.80700000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":7.415,"5min":7.415,"15min":7.415},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.704,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":0.88900000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":7.5650000000000004,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.298}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.877,"5min":0.751,"15min":0.751},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.1439999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:03:48.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:03:48.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.841+0000 7f1866a00640 1 Processor -- start 2024-01-31T05:03:48.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.841+0000 7f1866a00640 1 -- start start 2024-01-31T05:03:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.842+0000 7f1866a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1860159a50 0x7f18600caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1866a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18600b1d80 0x7f186015d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1866a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186015dd90 con 0x7f186015d980 2024-01-31T05:03:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1866a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600cb440 con 0x7f1860159a50 2024-01-31T05:03:48.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1866a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600cb650 con 0x7f18600b1d80 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f185fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1860159a50 0x7f18600caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1864a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f186015d980 0x7f18600ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54514/0 (socket says 172.21.15.97:54514) 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1864a00640 1 -- 172.21.15.97:0/1314777196 learned_addr learned my addr 172.21.15.97:0/1314777196 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f1857e00640 1 --2- 172.21.15.97:0/1314777196 >> v2:172.21.15.188:3300/0 conn(0x7f18600b1d80 0x7f186015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f185fe00640 1 -- 172.21.15.97:0/1314777196 >> v2:172.21.15.188:3300/0 conn(0x7f18600b1d80 msgr2=0x7f186015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f185fe00640 1 --2- 172.21.15.97:0/1314777196 >> v2:172.21.15.188:3300/0 conn(0x7f18600b1d80 0x7f186015d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:48.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f185fe00640 1 -- 172.21.15.97:0/1314777196 >> v1:172.21.15.97:6789/0 conn(0x7f186015d980 legacy=0x7f18600ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:48.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f185fe00640 1 -- 172.21.15.97:0/1314777196 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600cddb0 con 0x7f1860159a50 2024-01-31T05:03:48.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.843+0000 7f185f400640 1 -- 172.21.15.97:0/1314777196 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99908784 0 0) 0x7f186015dd90 con 0x7f186015d980 2024-01-31T05:03:48.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.845+0000 7f185fe00640 1 --2- 172.21.15.97:0/1314777196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1860159a50 0x7f18600caf00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f186006e470 tx=0x7f185002cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c11696185f7c650 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:48.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.845+0000 7f185f400640 1 -- 172.21.15.97:0/1314777196 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185002db50 con 0x7f1860159a50 2024-01-31T05:03:48.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.845+0000 7f185f400640 1 -- 172.21.15.97:0/1314777196 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185002dcb0 con 0x7f1860159a50 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.845+0000 7f185f400640 1 -- 172.21.15.97:0/1314777196 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850035660 con 0x7f1860159a50 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.845+0000 7f1866a00640 1 -- 172.21.15.97:0/1314777196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1860159a50 msgr2=0x7f18600caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.845+0000 7f1866a00640 1 --2- 172.21.15.97:0/1314777196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1860159a50 0x7f18600caf00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f186006e470 tx=0x7f185002cda0 comp rx=0 tx=0).stop 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.846+0000 7f1866a00640 1 -- 172.21.15.97:0/1314777196 shutdown_connections 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.846+0000 7f1866a00640 1 --2- 172.21.15.97:0/1314777196 >> v2:172.21.15.188:3300/0 conn(0x7f18600b1d80 0x7f186015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.846+0000 7f1866a00640 1 --2- 172.21.15.97:0/1314777196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1860159a50 0x7f18600caf00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.846+0000 7f1866a00640 1 -- 172.21.15.97:0/1314777196 >> 172.21.15.97:0/1314777196 conn(0x7f18600b0a10 msgr2=0x7f18600b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.846+0000 7f1866a00640 1 -- 172.21.15.97:0/1314777196 shutdown_connections 2024-01-31T05:03:48.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.846+0000 7f1866a00640 1 -- 172.21.15.97:0/1314777196 wait complete. 2024-01-31T05:03:48.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.847+0000 7f1866a00640 1 Processor -- start 2024-01-31T05:03:48.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.847+0000 7f1866a00640 1 -- start start 2024-01-31T05:03:48.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f1866a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 0x7f1860159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:48.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f1866a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1860159a50 0x7f1860151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:48.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f1866a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186015d980 0x7f186015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:48.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f1866a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186015a370 con 0x7f18600b1d80 2024-01-31T05:03:48.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f1866a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186015cd80 con 0x7f1860159a50 2024-01-31T05:03:48.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f1866a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600c94f0 con 0x7f186015d980 2024-01-31T05:03:48.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.848+0000 7f185fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 0x7f1860159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:48.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 0x7f1860159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41914/0 (socket says 172.21.15.97:41914) 2024-01-31T05:03:48.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 -- 172.21.15.97:0/616438853 learned_addr learned my addr 172.21.15.97:0/616438853 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:48.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 -- 172.21.15.97:0/616438853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186015d980 msgr2=0x7f186015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:48.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186015d980 0x7f186015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:48.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 -- 172.21.15.97:0/616438853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1860159a50 msgr2=0x7f1860151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:48.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1860159a50 0x7f1860151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:48.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 -- 172.21.15.97:0/616438853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185000a5a0 con 0x7f18600b1d80 2024-01-31T05:03:48.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185fe00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 0x7f1860159e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1850038040 tx=0x7f18500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:48.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185003e610 con 0x7f18600b1d80 2024-01-31T05:03:48.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.849+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185003e770 con 0x7f18600b1d80 2024-01-31T05:03:48.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.850+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186009b0e0 con 0x7f18600b1d80 2024-01-31T05:03:48.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.850+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185003eae0 con 0x7f18600b1d80 2024-01-31T05:03:48.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.850+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18600c9a70 con 0x7f18600b1d80 2024-01-31T05:03:48.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.851+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18600677c0 con 0x7f18600b1d80 2024-01-31T05:03:48.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.852+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f185003c040 con 0x7f18600b1d80 2024-01-31T05:03:48.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.852+0000 7f185cc00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f183806aed0 0x7f183806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:48.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.853+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f18500ab9f0 con 0x7f18600b1d80 2024-01-31T05:03:48.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.853+0000 7f1857e00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f183806aed0 0x7f183806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:48.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.854+0000 7f1857e00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f183806aed0 0x7f183806d390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18601f6eb0 tx=0x7f1848008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:48.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:48.857+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1850076770 con 0x7f18600b1d80 2024-01-31T05:03:49.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.048+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18600ce370 con 0x7f183806aed0 2024-01-31T05:03:49.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.070+0000 7f185cc00640 1 -- 172.21.15.97:0/616438853 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287075 (secure 0 0 0) 0x7f1848009e40 con 0x7f183806aed0 2024-01-31T05:03:49.074 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:49.075 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:03:49.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.075+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f183806aed0 msgr2=0x7f183806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.075+0000 7f1866a00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f183806aed0 0x7f183806d390 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18601f6eb0 tx=0x7f1848008040 comp rx=0 tx=0).stop 2024-01-31T05:03:49.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.075+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 msgr2=0x7f1860159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.075+0000 7f1866a00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 0x7f1860159e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1850038040 tx=0x7f18500046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 shutdown_connections 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f186015d980 0x7f186015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1860159a50 0x7f1860151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f183806aed0 0x7f183806d390 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 --2- 172.21.15.97:0/616438853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18600b1d80 0x7f1860159e30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 >> 172.21.15.97:0/616438853 conn(0x7f18600b0a10 msgr2=0x7f18601e4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 shutdown_connections 2024-01-31T05:03:49.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.076+0000 7f1866a00640 1 -- 172.21.15.97:0/616438853 wait complete. 2024-01-31T05:03:49.095 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-01-31T05:03:47.611374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222488,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752232,"statfs":{"total":773094113280,"available":772866285568,"internally_reserved":0,"allocated":6463488,"data_stored":3496636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.393508"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489800+0000","last_change":"2024-01-31T05:03:39.260557+0000","last_active":"2024-01-31T05:03:40.489800+0000","last_peered":"2024-01-31T05:03:40.489800+0000","last_clean":"2024-01-31T05:03:40.489800+0000","last_became_active":"2024-01-31T05:03:39.258375+0000","last_became_peered":"2024-01-31T05:03:39.258375+0000","last_unstale":"2024-01-31T05:03:40.489800+0000","last_undegraded":"2024-01-31T05:03:40.489800+0000","last_fullsized":"2024-01-31T05:03:40.489800+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:19:31.653654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.232304+0000","last_change":"2024-01-31T05:03:39.252312+0000","last_active":"2024-01-31T05:03:40.232304+0000","last_peered":"2024-01-31T05:03:40.232304+0000","last_clean":"2024-01-31T05:03:40.232304+0000","last_became_active":"2024-01-31T05:03:39.252042+0000","last_became_peered":"2024-01-31T05:03:39.252042+0000","last_unstale":"2024-01-31T05:03:40.232304+0000","last_undegraded":"2024-01-31T05:03:40.232304+0000","last_fullsized":"2024-01-31T05:03:40.232304+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:22:56.186949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.487767+0000","last_change":"2024-01-31T05:03:39.243310+0000","last_active":"2024-01-31T05:03:40.487767+0000","last_peered":"2024-01-31T05:03:40.487767+0000","last_clean":"2024-01-31T05:03:40.487767+0000","last_became_active":"2024-01-31T05:03:39.242309+0000","last_became_peered":"2024-01-31T05:03:39.242309+0000","last_unstale":"2024-01-31T05:03:40.487767+0000","last_undegraded":"2024-01-31T05:03:40.487767+0000","last_fullsized":"2024-01-31T05:03:40.487767+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:24:47.464933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.494252+0000","last_change":"2024-01-31T05:03:39.254633+0000","last_active":"2024-01-31T05:03:40.494252+0000","last_peered":"2024-01-31T05:03:40.494252+0000","last_clean":"2024-01-31T05:03:40.494252+0000","last_became_active":"2024-01-31T05:03:39.254325+0000","last_became_peered":"2024-01-31T05:03:39.254325+0000","last_unstale":"2024-01-31T05:03:40.494252+0000","last_undegraded":"2024-01-31T05:03:40.494252+0000","last_fullsized":"2024-01-31T05:03:40.494252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:49:55.963675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489001+0000","last_change":"2024-01-31T05:03:39.261633+0000","last_active":"2024-01-31T05:03:40.489001+0000","last_peered":"2024-01-31T05:03:40.489001+0000","last_clean":"2024-01-31T05:03:40.489001+0000","last_became_active":"2024-01-31T05:03:39.261202+0000","last_became_peered":"2024-01-31T05:03:39.261202+0000","last_unstale":"2024-01-31T05:03:40.489001+0000","last_undegraded":"2024-01-31T05:03:40.489001+0000","last_fullsized":"2024-01-31T05:03:40.489001+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:31:02.622030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488794+0000","last_change":"2024-01-31T05:03:39.263885+0000","last_active":"2024-01-31T05:03:40.488794+0000","last_peered":"2024-01-31T05:03:40.488794+0000","last_clean":"2024-01-31T05:03:40.488794+0000","last_became_active":"2024-01-31T05:03:39.262867+0000","last_became_peered":"2024-01-31T05:03:39.262867+0000","last_unstale":"2024-01-31T05:03:40.488794+0000","last_undegraded":"2024-01-31T05:03:40.488794+0000","last_fullsized":"2024-01-31T05:03:40.488794+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:21:03.401619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489864+0000","last_change":"2024-01-31T05:03:39.247885+0000","last_active":"2024-01-31T05:03:40.489864+0000","last_peered":"2024-01-31T05:03:40.489864+0000","last_clean":"2024-01-31T05:03:40.489864+0000","last_became_active":"2024-01-31T05:03:39.247556+0000","last_became_peered":"2024-01-31T05:03:39.247556+0000","last_unstale":"2024-01-31T05:03:40.489864+0000","last_undegraded":"2024-01-31T05:03:40.489864+0000","last_fullsized":"2024-01-31T05:03:40.489864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:51:00.104685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227479+0000","last_change":"2024-01-31T05:03:39.247522+0000","last_active":"2024-01-31T05:03:40.227479+0000","last_peered":"2024-01-31T05:03:40.227479+0000","last_clean":"2024-01-31T05:03:40.227479+0000","last_became_active":"2024-01-31T05:03:39.245280+0000","last_became_peered":"2024-01-31T05:03:39.245280+0000","last_unstale":"2024-01-31T05:03:40.227479+0000","last_undegraded":"2024-01-31T05:03:40.227479+0000","last_fullsized":"2024-01-31T05:03:40.227479+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:07:43.558579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.494360+0000","last_change":"2024-01-31T05:03:39.256441+0000","last_active":"2024-01-31T05:03:40.494360+0000","last_peered":"2024-01-31T05:03:40.494360+0000","last_clean":"2024-01-31T05:03:40.494360+0000","last_became_active":"2024-01-31T05:03:39.256105+0000","last_became_peered":"2024-01-31T05:03:39.256105+0000","last_unstale":"2024-01-31T05:03:40.494360+0000","last_undegraded":"2024-01-31T05:03:40.494360+0000","last_fullsized":"2024-01-31T05:03:40.494360+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:58:51.139110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489672+0000","last_change":"2024-01-31T05:03:39.262017+0000","last_active":"2024-01-31T05:03:40.489672+0000","last_peered":"2024-01-31T05:03:40.489672+0000","last_clean":"2024-01-31T05:03:40.489672+0000","last_became_active":"2024-01-31T05:03:39.261701+0000","last_became_peered":"2024-01-31T05:03:39.261701+0000","last_unstale":"2024-01-31T05:03:40.489672+0000","last_undegraded":"2024-01-31T05:03:40.489672+0000","last_fullsized":"2024-01-31T05:03:40.489672+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:59:34.944415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.229438+0000","last_change":"2024-01-31T05:03:39.251020+0000","last_active":"2024-01-31T05:03:40.229438+0000","last_peered":"2024-01-31T05:03:40.229438+0000","last_clean":"2024-01-31T05:03:40.229438+0000","last_became_active":"2024-01-31T05:03:39.250684+0000","last_became_peered":"2024-01-31T05:03:39.250684+0000","last_unstale":"2024-01-31T05:03:40.229438+0000","last_undegraded":"2024-01-31T05:03:40.229438+0000","last_fullsized":"2024-01-31T05:03:40.229438+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:28:58.106544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493872+0000","last_change":"2024-01-31T05:03:39.258617+0000","last_active":"2024-01-31T05:03:40.493872+0000","last_peered":"2024-01-31T05:03:40.493872+0000","last_clean":"2024-01-31T05:03:40.493872+0000","last_became_active":"2024-01-31T05:03:39.258317+0000","last_became_peered":"2024-01-31T05:03:39.258317+0000","last_unstale":"2024-01-31T05:03:40.493872+0000","last_undegraded":"2024-01-31T05:03:40.493872+0000","last_fullsized":"2024-01-31T05:03:40.493872+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:45:03.644259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490571+0000","last_change":"2024-01-31T05:03:39.256764+0000","last_active":"2024-01-31T05:03:40.490571+0000","last_peered":"2024-01-31T05:03:40.490571+0000","last_clean":"2024-01-31T05:03:40.490571+0000","last_became_active":"2024-01-31T05:03:39.256474+0000","last_became_peered":"2024-01-31T05:03:39.256474+0000","last_unstale":"2024-01-31T05:03:40.490571+0000","last_undegraded":"2024-01-31T05:03:40.490571+0000","last_fullsized":"2024-01-31T05:03:40.490571+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:21:03.401619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.487766+0000","last_change":"2024-01-31T05:03:39.251453+0000","last_active":"2024-01-31T05:03:40.487766+0000","last_peered":"2024-01-31T05:03:40.487766+0000","last_clean":"2024-01-31T05:03:40.487766+0000","last_became_active":"2024-01-31T05:03:39.248059+0000","last_became_peered":"2024-01-31T05:03:39.248059+0000","last_unstale":"2024-01-31T05:03:40.487766+0000","last_undegraded":"2024-01-31T05:03:40.487766+0000","last_fullsized":"2024-01-31T05:03:40.487766+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:58:51.139110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489938+0000","last_change":"2024-01-31T05:03:39.256908+0000","last_active":"2024-01-31T05:03:40.489938+0000","last_peered":"2024-01-31T05:03:40.489938+0000","last_clean":"2024-01-31T05:03:40.489938+0000","last_became_active":"2024-01-31T05:03:39.256641+0000","last_became_peered":"2024-01-31T05:03:39.256641+0000","last_unstale":"2024-01-31T05:03:40.489938+0000","last_undegraded":"2024-01-31T05:03:40.489938+0000","last_fullsized":"2024-01-31T05:03:40.489938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:18:00.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488669+0000","last_change":"2024-01-31T05:03:39.256477+0000","last_active":"2024-01-31T05:03:40.488669+0000","last_peered":"2024-01-31T05:03:40.488669+0000","last_clean":"2024-01-31T05:03:40.488669+0000","last_became_active":"2024-01-31T05:03:39.256188+0000","last_became_peered":"2024-01-31T05:03:39.256188+0000","last_unstale":"2024-01-31T05:03:40.488669+0000","last_undegraded":"2024-01-31T05:03:40.488669+0000","last_fullsized":"2024-01-31T05:03:40.488669+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:18:00.501508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490096+0000","last_change":"2024-01-31T05:03:39.268505+0000","last_active":"2024-01-31T05:03:40.490096+0000","last_peered":"2024-01-31T05:03:40.490096+0000","last_clean":"2024-01-31T05:03:40.490096+0000","last_became_active":"2024-01-31T05:03:39.268239+0000","last_became_peered":"2024-01-31T05:03:39.268239+0000","last_unstale":"2024-01-31T05:03:40.490096+0000","last_undegraded":"2024-01-31T05:03:40.490096+0000","last_fullsized":"2024-01-31T05:03:40.490096+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:33:21.399355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488404+0000","last_change":"2024-01-31T05:03:39.249229+0000","last_active":"2024-01-31T05:03:40.488404+0000","last_peered":"2024-01-31T05:03:40.488404+0000","last_clean":"2024-01-31T05:03:40.488404+0000","last_became_active":"2024-01-31T05:03:39.248929+0000","last_became_peered":"2024-01-31T05:03:39.248929+0000","last_unstale":"2024-01-31T05:03:40.488404+0000","last_undegraded":"2024-01-31T05:03:40.488404+0000","last_fullsized":"2024-01-31T05:03:40.488404+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:02:27.584127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.232548+0000","last_change":"2024-01-31T05:03:39.255701+0000","last_active":"2024-01-31T05:03:40.232548+0000","last_peered":"2024-01-31T05:03:40.232548+0000","last_clean":"2024-01-31T05:03:40.232548+0000","last_became_active":"2024-01-31T05:03:39.253975+0000","last_became_peered":"2024-01-31T05:03:39.253975+0000","last_unstale":"2024-01-31T05:03:40.232548+0000","last_undegraded":"2024-01-31T05:03:40.232548+0000","last_fullsized":"2024-01-31T05:03:40.232548+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:02:27.584127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.229364+0000","last_change":"2024-01-31T05:03:39.258999+0000","last_active":"2024-01-31T05:03:40.229364+0000","last_peered":"2024-01-31T05:03:40.229364+0000","last_clean":"2024-01-31T05:03:40.229364+0000","last_became_active":"2024-01-31T05:03:39.257976+0000","last_became_peered":"2024-01-31T05:03:39.257976+0000","last_unstale":"2024-01-31T05:03:40.229364+0000","last_undegraded":"2024-01-31T05:03:40.229364+0000","last_fullsized":"2024-01-31T05:03:40.229364+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:56:43.002256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491252+0000","last_change":"2024-01-31T05:03:39.267338+0000","last_active":"2024-01-31T05:03:40.491252+0000","last_peered":"2024-01-31T05:03:40.491252+0000","last_clean":"2024-01-31T05:03:40.491252+0000","last_became_active":"2024-01-31T05:03:39.267065+0000","last_became_peered":"2024-01-31T05:03:39.267065+0000","last_unstale":"2024-01-31T05:03:40.491252+0000","last_undegraded":"2024-01-31T05:03:40.491252+0000","last_fullsized":"2024-01-31T05:03:40.491252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:59:34.944415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231289+0000","last_change":"2024-01-31T05:03:39.242603+0000","last_active":"2024-01-31T05:03:40.231289+0000","last_peered":"2024-01-31T05:03:40.231289+0000","last_clean":"2024-01-31T05:03:40.231289+0000","last_became_active":"2024-01-31T05:03:39.241443+0000","last_became_peered":"2024-01-31T05:03:39.241443+0000","last_unstale":"2024-01-31T05:03:40.231289+0000","last_undegraded":"2024-01-31T05:03:40.231289+0000","last_fullsized":"2024-01-31T05:03:40.231289+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:56:38.222983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490215+0000","last_change":"2024-01-31T05:03:39.253595+0000","last_active":"2024-01-31T05:03:40.490215+0000","last_peered":"2024-01-31T05:03:40.490215+0000","last_clean":"2024-01-31T05:03:40.490215+0000","last_became_active":"2024-01-31T05:03:39.253076+0000","last_became_peered":"2024-01-31T05:03:39.253076+0000","last_unstale":"2024-01-31T05:03:40.490215+0000","last_undegraded":"2024-01-31T05:03:40.490215+0000","last_fullsized":"2024-01-31T05:03:40.490215+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:55:13.286439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491682+0000","last_change":"2024-01-31T05:03:39.260142+0000","last_active":"2024-01-31T05:03:40.491682+0000","last_peered":"2024-01-31T05:03:40.491682+0000","last_clean":"2024-01-31T05:03:40.491682+0000","last_became_active":"2024-01-31T05:03:39.255918+0000","last_became_peered":"2024-01-31T05:03:39.255918+0000","last_unstale":"2024-01-31T05:03:40.491682+0000","last_undegraded":"2024-01-31T05:03:40.491682+0000","last_fullsized":"2024-01-31T05:03:40.491682+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:55:50.908110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.232643+0000","last_change":"2024-01-31T05:03:39.253163+0000","last_active":"2024-01-31T05:03:40.232643+0000","last_peered":"2024-01-31T05:03:40.232643+0000","last_clean":"2024-01-31T05:03:40.232643+0000","last_became_active":"2024-01-31T05:03:39.252895+0000","last_became_peered":"2024-01-31T05:03:39.252895+0000","last_unstale":"2024-01-31T05:03:40.232643+0000","last_undegraded":"2024-01-31T05:03:40.232643+0000","last_fullsized":"2024-01-31T05:03:40.232643+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:45:19.315615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490980+0000","last_change":"2024-01-31T05:03:39.269057+0000","last_active":"2024-01-31T05:03:40.490980+0000","last_peered":"2024-01-31T05:03:40.490980+0000","last_clean":"2024-01-31T05:03:40.490980+0000","last_became_active":"2024-01-31T05:03:39.268737+0000","last_became_peered":"2024-01-31T05:03:39.268737+0000","last_unstale":"2024-01-31T05:03:40.490980+0000","last_undegraded":"2024-01-31T05:03:40.490980+0000","last_fullsized":"2024-01-31T05:03:40.490980+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:09:21.970001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.232751+0000","last_change":"2024-01-31T05:03:39.245654+0000","last_active":"2024-01-31T05:03:40.232751+0000","last_peered":"2024-01-31T05:03:40.232751+0000","last_clean":"2024-01-31T05:03:40.232751+0000","last_became_active":"2024-01-31T05:03:39.242973+0000","last_became_peered":"2024-01-31T05:03:39.242973+0000","last_unstale":"2024-01-31T05:03:40.232751+0000","last_undegraded":"2024-01-31T05:03:40.232751+0000","last_fullsized":"2024-01-31T05:03:40.232751+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:40:28.117412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228339+0000","last_change":"2024-01-31T05:03:39.258069+0000","last_active":"2024-01-31T05:03:40.228339+0000","last_peered":"2024-01-31T05:03:40.228339+0000","last_clean":"2024-01-31T05:03:40.228339+0000","last_became_active":"2024-01-31T05:03:39.257656+0000","last_became_peered":"2024-01-31T05:03:39.257656+0000","last_unstale":"2024-01-31T05:03:40.228339+0000","last_undegraded":"2024-01-31T05:03:40.228339+0000","last_fullsized":"2024-01-31T05:03:40.228339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:51:41.823493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227863+0000","last_change":"2024-01-31T05:03:39.255737+0000","last_active":"2024-01-31T05:03:40.227863+0000","last_peered":"2024-01-31T05:03:40.227863+0000","last_clean":"2024-01-31T05:03:40.227863+0000","last_became_active":"2024-01-31T05:03:39.255036+0000","last_became_peered":"2024-01-31T05:03:39.255036+0000","last_unstale":"2024-01-31T05:03:40.227863+0000","last_undegraded":"2024-01-31T05:03:40.227863+0000","last_fullsized":"2024-01-31T05:03:40.227863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:44:52.749741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491746+0000","last_change":"2024-01-31T05:03:39.258792+0000","last_active":"2024-01-31T05:03:40.491746+0000","last_peered":"2024-01-31T05:03:40.491746+0000","last_clean":"2024-01-31T05:03:40.491746+0000","last_became_active":"2024-01-31T05:03:39.258531+0000","last_became_peered":"2024-01-31T05:03:39.258531+0000","last_unstale":"2024-01-31T05:03:40.491746+0000","last_undegraded":"2024-01-31T05:03:40.491746+0000","last_fullsized":"2024-01-31T05:03:40.491746+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:15:27.762230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231139+0000","last_change":"2024-01-31T05:03:39.236368+0000","last_active":"2024-01-31T05:03:40.231139+0000","last_peered":"2024-01-31T05:03:40.231139+0000","last_clean":"2024-01-31T05:03:40.231139+0000","last_became_active":"2024-01-31T05:03:39.236055+0000","last_became_peered":"2024-01-31T05:03:39.236055+0000","last_unstale":"2024-01-31T05:03:40.231139+0000","last_undegraded":"2024-01-31T05:03:40.231139+0000","last_fullsized":"2024-01-31T05:03:40.231139+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:01:30.708807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.226893+0000","last_change":"2024-01-31T05:03:39.239349+0000","last_active":"2024-01-31T05:03:40.226893+0000","last_peered":"2024-01-31T05:03:40.226893+0000","last_clean":"2024-01-31T05:03:40.226893+0000","last_became_active":"2024-01-31T05:03:39.238868+0000","last_became_peered":"2024-01-31T05:03:39.238868+0000","last_unstale":"2024-01-31T05:03:40.226893+0000","last_undegraded":"2024-01-31T05:03:40.226893+0000","last_fullsized":"2024-01-31T05:03:40.226893+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:09:13.415768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489812+0000","last_change":"2024-01-31T05:03:39.267582+0000","last_active":"2024-01-31T05:03:40.489812+0000","last_peered":"2024-01-31T05:03:40.489812+0000","last_clean":"2024-01-31T05:03:40.489812+0000","last_became_active":"2024-01-31T05:03:39.267293+0000","last_became_peered":"2024-01-31T05:03:39.267293+0000","last_unstale":"2024-01-31T05:03:40.489812+0000","last_undegraded":"2024-01-31T05:03:40.489812+0000","last_fullsized":"2024-01-31T05:03:40.489812+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:34:30.294051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.229697+0000","last_change":"2024-01-31T05:03:38.235515+0000","last_active":"2024-01-31T05:03:40.229697+0000","last_peered":"2024-01-31T05:03:40.229697+0000","last_clean":"2024-01-31T05:03:40.229697+0000","last_became_active":"2024-01-31T05:03:38.235228+0000","last_became_peered":"2024-01-31T05:03:38.235228+0000","last_unstale":"2024-01-31T05:03:40.229697+0000","last_undegraded":"2024-01-31T05:03:40.229697+0000","last_fullsized":"2024-01-31T05:03:40.229697+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:20:42.179467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490032+0000","last_change":"2024-01-31T05:03:39.262932+0000","last_active":"2024-01-31T05:03:40.490032+0000","last_peered":"2024-01-31T05:03:40.490032+0000","last_clean":"2024-01-31T05:03:40.490032+0000","last_became_active":"2024-01-31T05:03:39.262659+0000","last_became_peered":"2024-01-31T05:03:39.262659+0000","last_unstale":"2024-01-31T05:03:40.490032+0000","last_undegraded":"2024-01-31T05:03:40.490032+0000","last_fullsized":"2024-01-31T05:03:40.490032+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:16:46.541447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.232571+0000","last_change":"2024-01-31T05:03:38.224888+0000","last_active":"2024-01-31T05:03:40.232571+0000","last_peered":"2024-01-31T05:03:40.232571+0000","last_clean":"2024-01-31T05:03:40.232571+0000","last_became_active":"2024-01-31T05:03:38.223355+0000","last_became_peered":"2024-01-31T05:03:38.223355+0000","last_unstale":"2024-01-31T05:03:40.232571+0000","last_undegraded":"2024-01-31T05:03:40.232571+0000","last_fullsized":"2024-01-31T05:03:40.232571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228747+0000","last_change":"2024-01-31T05:03:39.254497+0000","last_active":"2024-01-31T05:03:40.228747+0000","last_peered":"2024-01-31T05:03:40.228747+0000","last_clean":"2024-01-31T05:03:40.228747+0000","last_became_active":"2024-01-31T05:03:39.254124+0000","last_became_peered":"2024-01-31T05:03:39.254124+0000","last_unstale":"2024-01-31T05:03:40.228747+0000","last_undegraded":"2024-01-31T05:03:40.228747+0000","last_fullsized":"2024-01-31T05:03:40.228747+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T17:00:28.672300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489317+0000","last_change":"2024-01-31T05:03:38.234777+0000","last_active":"2024-01-31T05:03:40.489317+0000","last_peered":"2024-01-31T05:03:40.489317+0000","last_clean":"2024-01-31T05:03:40.489317+0000","last_became_active":"2024-01-31T05:03:38.234493+0000","last_became_peered":"2024-01-31T05:03:38.234493+0000","last_unstale":"2024-01-31T05:03:40.489317+0000","last_undegraded":"2024-01-31T05:03:40.489317+0000","last_fullsized":"2024-01-31T05:03:40.489317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:58:54.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489382+0000","last_change":"2024-01-31T05:03:39.256322+0000","last_active":"2024-01-31T05:03:40.489382+0000","last_peered":"2024-01-31T05:03:40.489382+0000","last_clean":"2024-01-31T05:03:40.489382+0000","last_became_active":"2024-01-31T05:03:39.255987+0000","last_became_peered":"2024-01-31T05:03:39.255987+0000","last_unstale":"2024-01-31T05:03:40.489382+0000","last_undegraded":"2024-01-31T05:03:40.489382+0000","last_fullsized":"2024-01-31T05:03:40.489382+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:09:21.970001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.494301+0000","last_change":"2024-01-31T05:03:38.221765+0000","last_active":"2024-01-31T05:03:40.494301+0000","last_peered":"2024-01-31T05:03:40.494301+0000","last_clean":"2024-01-31T05:03:40.494301+0000","last_became_active":"2024-01-31T05:03:38.221534+0000","last_became_peered":"2024-01-31T05:03:38.221534+0000","last_unstale":"2024-01-31T05:03:40.494301+0000","last_undegraded":"2024-01-31T05:03:40.494301+0000","last_fullsized":"2024-01-31T05:03:40.494301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:38:22.374866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227677+0000","last_change":"2024-01-31T05:03:39.243617+0000","last_active":"2024-01-31T05:03:40.227677+0000","last_peered":"2024-01-31T05:03:40.227677+0000","last_clean":"2024-01-31T05:03:40.227677+0000","last_became_active":"2024-01-31T05:03:39.242339+0000","last_became_peered":"2024-01-31T05:03:39.242339+0000","last_unstale":"2024-01-31T05:03:40.227677+0000","last_undegraded":"2024-01-31T05:03:40.227677+0000","last_fullsized":"2024-01-31T05:03:40.227677+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:33:51.683855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488079+0000","last_change":"2024-01-31T05:03:38.226696+0000","last_active":"2024-01-31T05:03:40.488079+0000","last_peered":"2024-01-31T05:03:40.488079+0000","last_clean":"2024-01-31T05:03:40.488079+0000","last_became_active":"2024-01-31T05:03:38.226440+0000","last_became_peered":"2024-01-31T05:03:38.226440+0000","last_unstale":"2024-01-31T05:03:40.488079+0000","last_undegraded":"2024-01-31T05:03:40.488079+0000","last_fullsized":"2024-01-31T05:03:40.488079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.230864+0000","last_change":"2024-01-31T05:03:39.240337+0000","last_active":"2024-01-31T05:03:40.230864+0000","last_peered":"2024-01-31T05:03:40.230864+0000","last_clean":"2024-01-31T05:03:40.230864+0000","last_became_active":"2024-01-31T05:03:39.240029+0000","last_became_peered":"2024-01-31T05:03:39.240029+0000","last_unstale":"2024-01-31T05:03:40.230864+0000","last_undegraded":"2024-01-31T05:03:40.230864+0000","last_fullsized":"2024-01-31T05:03:40.230864+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:45:27.211771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489123+0000","last_change":"2024-01-31T05:03:38.239050+0000","last_active":"2024-01-31T05:03:40.489123+0000","last_peered":"2024-01-31T05:03:40.489123+0000","last_clean":"2024-01-31T05:03:40.489123+0000","last_became_active":"2024-01-31T05:03:38.238746+0000","last_became_peered":"2024-01-31T05:03:38.238746+0000","last_unstale":"2024-01-31T05:03:40.489123+0000","last_undegraded":"2024-01-31T05:03:40.489123+0000","last_fullsized":"2024-01-31T05:03:40.489123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:25:37.971919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.487588+0000","last_change":"2024-01-31T05:03:39.244622+0000","last_active":"2024-01-31T05:03:40.487588+0000","last_peered":"2024-01-31T05:03:40.487588+0000","last_clean":"2024-01-31T05:03:40.487588+0000","last_became_active":"2024-01-31T05:03:39.243824+0000","last_became_peered":"2024-01-31T05:03:39.243824+0000","last_unstale":"2024-01-31T05:03:40.487588+0000","last_undegraded":"2024-01-31T05:03:40.487588+0000","last_fullsized":"2024-01-31T05:03:40.487588+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:45:03.644259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493035+0000","last_change":"2024-01-31T05:03:38.230982+0000","last_active":"2024-01-31T05:03:40.493035+0000","last_peered":"2024-01-31T05:03:40.493035+0000","last_clean":"2024-01-31T05:03:40.493035+0000","last_became_active":"2024-01-31T05:03:38.230687+0000","last_became_peered":"2024-01-31T05:03:38.230687+0000","last_unstale":"2024-01-31T05:03:40.493035+0000","last_undegraded":"2024-01-31T05:03:40.493035+0000","last_fullsized":"2024-01-31T05:03:40.493035+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488663+0000","last_change":"2024-01-31T05:03:39.261537+0000","last_active":"2024-01-31T05:03:40.488663+0000","last_peered":"2024-01-31T05:03:40.488663+0000","last_clean":"2024-01-31T05:03:40.488663+0000","last_became_active":"2024-01-31T05:03:39.261192+0000","last_became_peered":"2024-01-31T05:03:39.261192+0000","last_unstale":"2024-01-31T05:03:40.488663+0000","last_undegraded":"2024-01-31T05:03:40.488663+0000","last_fullsized":"2024-01-31T05:03:40.488663+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:48:34.398014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489616+0000","last_change":"2024-01-31T05:03:38.236445+0000","last_active":"2024-01-31T05:03:40.489616+0000","last_peered":"2024-01-31T05:03:40.489616+0000","last_clean":"2024-01-31T05:03:40.489616+0000","last_became_active":"2024-01-31T05:03:38.236193+0000","last_became_peered":"2024-01-31T05:03:38.236193+0000","last_unstale":"2024-01-31T05:03:40.489616+0000","last_undegraded":"2024-01-31T05:03:40.489616+0000","last_fullsized":"2024-01-31T05:03:40.489616+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:58:23.675405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490979+0000","last_change":"2024-01-31T05:03:39.260860+0000","last_active":"2024-01-31T05:03:40.490979+0000","last_peered":"2024-01-31T05:03:40.490979+0000","last_clean":"2024-01-31T05:03:40.490979+0000","last_became_active":"2024-01-31T05:03:39.260462+0000","last_became_peered":"2024-01-31T05:03:39.260462+0000","last_unstale":"2024-01-31T05:03:40.490979+0000","last_undegraded":"2024-01-31T05:03:40.490979+0000","last_fullsized":"2024-01-31T05:03:40.490979+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:31:02.622030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493824+0000","last_change":"2024-01-31T05:03:38.223052+0000","last_active":"2024-01-31T05:03:40.493824+0000","last_peered":"2024-01-31T05:03:40.493824+0000","last_clean":"2024-01-31T05:03:40.493824+0000","last_became_active":"2024-01-31T05:03:38.222742+0000","last_became_peered":"2024-01-31T05:03:38.222742+0000","last_unstale":"2024-01-31T05:03:40.493824+0000","last_undegraded":"2024-01-31T05:03:40.493824+0000","last_fullsized":"2024-01-31T05:03:40.493824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489053+0000","last_change":"2024-01-31T05:03:39.252433+0000","last_active":"2024-01-31T05:03:40.489053+0000","last_peered":"2024-01-31T05:03:40.489053+0000","last_clean":"2024-01-31T05:03:40.489053+0000","last_became_active":"2024-01-31T05:03:39.249986+0000","last_became_peered":"2024-01-31T05:03:39.249986+0000","last_unstale":"2024-01-31T05:03:40.489053+0000","last_undegraded":"2024-01-31T05:03:40.489053+0000","last_fullsized":"2024-01-31T05:03:40.489053+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:52:50.320846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228750+0000","last_change":"2024-01-31T05:03:38.237692+0000","last_active":"2024-01-31T05:03:40.228750+0000","last_peered":"2024-01-31T05:03:40.228750+0000","last_clean":"2024-01-31T05:03:40.228750+0000","last_became_active":"2024-01-31T05:03:38.236259+0000","last_became_peered":"2024-01-31T05:03:38.236259+0000","last_unstale":"2024-01-31T05:03:40.228750+0000","last_undegraded":"2024-01-31T05:03:40.228750+0000","last_fullsized":"2024-01-31T05:03:40.228750+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:40:28.240188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.229137+0000","last_change":"2024-01-31T05:03:39.254365+0000","last_active":"2024-01-31T05:03:40.229137+0000","last_peered":"2024-01-31T05:03:40.229137+0000","last_clean":"2024-01-31T05:03:40.229137+0000","last_became_active":"2024-01-31T05:03:39.254024+0000","last_became_peered":"2024-01-31T05:03:39.254024+0000","last_unstale":"2024-01-31T05:03:40.229137+0000","last_undegraded":"2024-01-31T05:03:40.229137+0000","last_fullsized":"2024-01-31T05:03:40.229137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:55:02.313673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493572+0000","last_change":"2024-01-31T05:03:38.225581+0000","last_active":"2024-01-31T05:03:40.493572+0000","last_peered":"2024-01-31T05:03:40.493572+0000","last_clean":"2024-01-31T05:03:40.493572+0000","last_became_active":"2024-01-31T05:03:38.225059+0000","last_became_peered":"2024-01-31T05:03:38.225059+0000","last_unstale":"2024-01-31T05:03:40.493572+0000","last_undegraded":"2024-01-31T05:03:40.493572+0000","last_fullsized":"2024-01-31T05:03:40.493572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:03:20.290478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493201+0000","last_change":"2024-01-31T05:03:39.250182+0000","last_active":"2024-01-31T05:03:40.493201+0000","last_peered":"2024-01-31T05:03:40.493201+0000","last_clean":"2024-01-31T05:03:40.493201+0000","last_became_active":"2024-01-31T05:03:39.249917+0000","last_became_peered":"2024-01-31T05:03:39.249917+0000","last_unstale":"2024-01-31T05:03:40.493201+0000","last_undegraded":"2024-01-31T05:03:40.493201+0000","last_fullsized":"2024-01-31T05:03:40.493201+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:30:41.979456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":20,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490380+0000","last_change":"2024-01-31T05:03:38.230550+0000","last_active":"2024-01-31T05:03:40.490380+0000","last_peered":"2024-01-31T05:03:40.490380+0000","last_clean":"2024-01-31T05:03:40.490380+0000","last_became_active":"2024-01-31T05:03:38.230284+0000","last_became_peered":"2024-01-31T05:03:38.230284+0000","last_unstale":"2024-01-31T05:03:40.490380+0000","last_undegraded":"2024-01-31T05:03:40.490380+0000","last_fullsized":"2024-01-31T05:03:40.490380+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:22.646337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493260+0000","last_change":"2024-01-31T05:03:39.246221+0000","last_active":"2024-01-31T05:03:40.493260+0000","last_peered":"2024-01-31T05:03:40.493260+0000","last_clean":"2024-01-31T05:03:40.493260+0000","last_became_active":"2024-01-31T05:03:39.244574+0000","last_became_peered":"2024-01-31T05:03:39.244574+0000","last_unstale":"2024-01-31T05:03:40.493260+0000","last_undegraded":"2024-01-31T05:03:40.493260+0000","last_fullsized":"2024-01-31T05:03:40.493260+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:27:06.790339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228538+0000","last_change":"2024-01-31T05:03:38.237263+0000","last_active":"2024-01-31T05:03:40.228538+0000","last_peered":"2024-01-31T05:03:40.228538+0000","last_clean":"2024-01-31T05:03:40.228538+0000","last_became_active":"2024-01-31T05:03:38.236868+0000","last_became_peered":"2024-01-31T05:03:38.236868+0000","last_unstale":"2024-01-31T05:03:40.228538+0000","last_undegraded":"2024-01-31T05:03:40.228538+0000","last_fullsized":"2024-01-31T05:03:40.228538+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:42:22.646337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.487831+0000","last_change":"2024-01-31T05:03:39.242937+0000","last_active":"2024-01-31T05:03:40.487831+0000","last_peered":"2024-01-31T05:03:40.487831+0000","last_clean":"2024-01-31T05:03:40.487831+0000","last_became_active":"2024-01-31T05:03:39.241602+0000","last_became_peered":"2024-01-31T05:03:39.241602+0000","last_unstale":"2024-01-31T05:03:40.487831+0000","last_undegraded":"2024-01-31T05:03:40.487831+0000","last_fullsized":"2024-01-31T05:03:40.487831+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:22:56.186949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491888+0000","last_change":"2024-01-31T05:03:38.225322+0000","last_active":"2024-01-31T05:03:40.491888+0000","last_peered":"2024-01-31T05:03:40.491888+0000","last_clean":"2024-01-31T05:03:40.491888+0000","last_became_active":"2024-01-31T05:03:38.224422+0000","last_became_peered":"2024-01-31T05:03:38.224422+0000","last_unstale":"2024-01-31T05:03:40.491888+0000","last_undegraded":"2024-01-31T05:03:40.491888+0000","last_fullsized":"2024-01-31T05:03:40.491888+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:42:44.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491640+0000","last_change":"2024-01-31T05:03:39.258087+0000","last_active":"2024-01-31T05:03:40.491640+0000","last_peered":"2024-01-31T05:03:40.491640+0000","last_clean":"2024-01-31T05:03:40.491640+0000","last_became_active":"2024-01-31T05:03:39.257724+0000","last_became_peered":"2024-01-31T05:03:39.257724+0000","last_unstale":"2024-01-31T05:03:40.491640+0000","last_undegraded":"2024-01-31T05:03:40.491640+0000","last_fullsized":"2024-01-31T05:03:40.491640+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:16:42.655596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490657+0000","last_change":"2024-01-31T05:03:38.219525+0000","last_active":"2024-01-31T05:03:40.490657+0000","last_peered":"2024-01-31T05:03:40.490657+0000","last_clean":"2024-01-31T05:03:40.490657+0000","last_became_active":"2024-01-31T05:03:38.219302+0000","last_became_peered":"2024-01-31T05:03:38.219302+0000","last_unstale":"2024-01-31T05:03:40.490657+0000","last_undegraded":"2024-01-31T05:03:40.490657+0000","last_fullsized":"2024-01-31T05:03:40.490657+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:14:08.160317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490520+0000","last_change":"2024-01-31T05:03:39.253326+0000","last_active":"2024-01-31T05:03:40.490520+0000","last_peered":"2024-01-31T05:03:40.490520+0000","last_clean":"2024-01-31T05:03:40.490520+0000","last_became_active":"2024-01-31T05:03:39.252687+0000","last_became_peered":"2024-01-31T05:03:39.252687+0000","last_unstale":"2024-01-31T05:03:40.490520+0000","last_undegraded":"2024-01-31T05:03:40.490520+0000","last_fullsized":"2024-01-31T05:03:40.490520+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:23:24.823193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":22,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.492674+0000","last_change":"2024-01-31T05:03:38.224819+0000","last_active":"2024-01-31T05:03:40.492674+0000","last_peered":"2024-01-31T05:03:40.492674+0000","last_clean":"2024-01-31T05:03:40.492674+0000","last_became_active":"2024-01-31T05:03:38.224555+0000","last_became_peered":"2024-01-31T05:03:38.224555+0000","last_unstale":"2024-01-31T05:03:40.492674+0000","last_undegraded":"2024-01-31T05:03:40.492674+0000","last_fullsized":"2024-01-31T05:03:40.492674+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:15:12.301634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488540+0000","last_change":"2024-01-31T05:03:39.249020+0000","last_active":"2024-01-31T05:03:40.488540+0000","last_peered":"2024-01-31T05:03:40.488540+0000","last_clean":"2024-01-31T05:03:40.488540+0000","last_became_active":"2024-01-31T05:03:39.246977+0000","last_became_peered":"2024-01-31T05:03:39.246977+0000","last_unstale":"2024-01-31T05:03:40.488540+0000","last_undegraded":"2024-01-31T05:03:40.488540+0000","last_fullsized":"2024-01-31T05:03:40.488540+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T16:45:19.315615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.232877+0000","last_change":"2024-01-31T05:03:38.239609+0000","last_active":"2024-01-31T05:03:40.232877+0000","last_peered":"2024-01-31T05:03:40.232877+0000","last_clean":"2024-01-31T05:03:40.232877+0000","last_became_active":"2024-01-31T05:03:38.238339+0000","last_became_peered":"2024-01-31T05:03:38.238339+0000","last_unstale":"2024-01-31T05:03:40.232877+0000","last_undegraded":"2024-01-31T05:03:40.232877+0000","last_fullsized":"2024-01-31T05:03:40.232877+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489252+0000","last_change":"2024-01-31T05:03:39.250705+0000","last_active":"2024-01-31T05:03:40.489252+0000","last_peered":"2024-01-31T05:03:40.489252+0000","last_clean":"2024-01-31T05:03:40.489252+0000","last_became_active":"2024-01-31T05:03:39.248093+0000","last_became_peered":"2024-01-31T05:03:39.248093+0000","last_unstale":"2024-01-31T05:03:40.489252+0000","last_undegraded":"2024-01-31T05:03:40.489252+0000","last_fullsized":"2024-01-31T05:03:40.489252+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:44:51.983908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228988+0000","last_change":"2024-01-31T05:03:38.237901+0000","last_active":"2024-01-31T05:03:40.228988+0000","last_peered":"2024-01-31T05:03:40.228988+0000","last_clean":"2024-01-31T05:03:40.228988+0000","last_became_active":"2024-01-31T05:03:38.237413+0000","last_became_peered":"2024-01-31T05:03:38.237413+0000","last_unstale":"2024-01-31T05:03:40.228988+0000","last_undegraded":"2024-01-31T05:03:40.228988+0000","last_fullsized":"2024-01-31T05:03:40.228988+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:48:48.105897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231441+0000","last_change":"2024-01-31T05:03:39.257158+0000","last_active":"2024-01-31T05:03:40.231441+0000","last_peered":"2024-01-31T05:03:40.231441+0000","last_clean":"2024-01-31T05:03:40.231441+0000","last_became_active":"2024-01-31T05:03:39.256870+0000","last_became_peered":"2024-01-31T05:03:39.256870+0000","last_unstale":"2024-01-31T05:03:40.231441+0000","last_undegraded":"2024-01-31T05:03:40.231441+0000","last_fullsized":"2024-01-31T05:03:40.231441+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:45:03.644259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488144+0000","last_change":"2024-01-31T05:03:38.230887+0000","last_active":"2024-01-31T05:03:40.488144+0000","last_peered":"2024-01-31T05:03:40.488144+0000","last_clean":"2024-01-31T05:03:40.488144+0000","last_became_active":"2024-01-31T05:03:38.229073+0000","last_became_peered":"2024-01-31T05:03:38.229073+0000","last_unstale":"2024-01-31T05:03:40.488144+0000","last_undegraded":"2024-01-31T05:03:40.488144+0000","last_fullsized":"2024-01-31T05:03:40.488144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:03:36.897525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493435+0000","last_change":"2024-01-31T05:03:39.254309+0000","last_active":"2024-01-31T05:03:40.493435+0000","last_peered":"2024-01-31T05:03:40.493435+0000","last_clean":"2024-01-31T05:03:40.493435+0000","last_became_active":"2024-01-31T05:03:39.254044+0000","last_became_peered":"2024-01-31T05:03:39.254044+0000","last_unstale":"2024-01-31T05:03:40.493435+0000","last_undegraded":"2024-01-31T05:03:40.493435+0000","last_fullsized":"2024-01-31T05:03:40.493435+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:44:18.715967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.229366+0000","last_change":"2024-01-31T05:03:38.234624+0000","last_active":"2024-01-31T05:03:40.229366+0000","last_peered":"2024-01-31T05:03:40.229366+0000","last_clean":"2024-01-31T05:03:40.229366+0000","last_became_active":"2024-01-31T05:03:38.234296+0000","last_became_peered":"2024-01-31T05:03:38.234296+0000","last_unstale":"2024-01-31T05:03:40.229366+0000","last_undegraded":"2024-01-31T05:03:40.229366+0000","last_fullsized":"2024-01-31T05:03:40.229366+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:30:59.107191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493430+0000","last_change":"2024-01-31T05:03:39.260082+0000","last_active":"2024-01-31T05:03:40.493430+0000","last_peered":"2024-01-31T05:03:40.493430+0000","last_clean":"2024-01-31T05:03:40.493430+0000","last_became_active":"2024-01-31T05:03:39.256821+0000","last_became_peered":"2024-01-31T05:03:39.256821+0000","last_unstale":"2024-01-31T05:03:40.493430+0000","last_undegraded":"2024-01-31T05:03:40.493430+0000","last_fullsized":"2024-01-31T05:03:40.493430+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:01:30.708807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231770+0000","last_change":"2024-01-31T05:03:38.234981+0000","last_active":"2024-01-31T05:03:40.231770+0000","last_peered":"2024-01-31T05:03:40.231770+0000","last_clean":"2024-01-31T05:03:40.231770+0000","last_became_active":"2024-01-31T05:03:38.234678+0000","last_became_peered":"2024-01-31T05:03:38.234678+0000","last_unstale":"2024-01-31T05:03:40.231770+0000","last_undegraded":"2024-01-31T05:03:40.231770+0000","last_fullsized":"2024-01-31T05:03:40.231770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T12:08:18.852860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228277+0000","last_change":"2024-01-31T05:03:39.255296+0000","last_active":"2024-01-31T05:03:40.228277+0000","last_peered":"2024-01-31T05:03:40.228277+0000","last_clean":"2024-01-31T05:03:40.228277+0000","last_became_active":"2024-01-31T05:03:39.254711+0000","last_became_peered":"2024-01-31T05:03:39.254711+0000","last_unstale":"2024-01-31T05:03:40.228277+0000","last_undegraded":"2024-01-31T05:03:40.228277+0000","last_fullsized":"2024-01-31T05:03:40.228277+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:41:15.593227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227898+0000","last_change":"2024-01-31T05:03:38.230732+0000","last_active":"2024-01-31T05:03:40.227898+0000","last_peered":"2024-01-31T05:03:40.227898+0000","last_clean":"2024-01-31T05:03:40.227898+0000","last_became_active":"2024-01-31T05:03:38.229403+0000","last_became_peered":"2024-01-31T05:03:38.229403+0000","last_unstale":"2024-01-31T05:03:40.227898+0000","last_undegraded":"2024-01-31T05:03:40.227898+0000","last_fullsized":"2024-01-31T05:03:40.227898+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:10:04.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227220+0000","last_change":"2024-01-31T05:03:39.259260+0000","last_active":"2024-01-31T05:03:40.227220+0000","last_peered":"2024-01-31T05:03:40.227220+0000","last_clean":"2024-01-31T05:03:40.227220+0000","last_became_active":"2024-01-31T05:03:39.258933+0000","last_became_peered":"2024-01-31T05:03:39.258933+0000","last_unstale":"2024-01-31T05:03:40.227220+0000","last_undegraded":"2024-01-31T05:03:40.227220+0000","last_fullsized":"2024-01-31T05:03:40.227220+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T05:55:24.236703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491731+0000","last_change":"2024-01-31T05:03:38.230238+0000","last_active":"2024-01-31T05:03:40.491731+0000","last_peered":"2024-01-31T05:03:40.491731+0000","last_clean":"2024-01-31T05:03:40.491731+0000","last_became_active":"2024-01-31T05:03:38.229961+0000","last_became_peered":"2024-01-31T05:03:38.229961+0000","last_unstale":"2024-01-31T05:03:40.491731+0000","last_undegraded":"2024-01-31T05:03:40.491731+0000","last_fullsized":"2024-01-31T05:03:40.491731+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T14:19:49.701715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488670+0000","last_change":"2024-01-31T05:03:39.248707+0000","last_active":"2024-01-31T05:03:40.488670+0000","last_peered":"2024-01-31T05:03:40.488670+0000","last_clean":"2024-01-31T05:03:40.488670+0000","last_became_active":"2024-01-31T05:03:39.246545+0000","last_became_peered":"2024-01-31T05:03:39.246545+0000","last_unstale":"2024-01-31T05:03:40.488670+0000","last_undegraded":"2024-01-31T05:03:40.488670+0000","last_fullsized":"2024-01-31T05:03:40.488670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:40:28.117412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.233115+0000","last_change":"2024-01-31T05:03:38.232128+0000","last_active":"2024-01-31T05:03:40.233115+0000","last_peered":"2024-01-31T05:03:40.233115+0000","last_clean":"2024-01-31T05:03:40.233115+0000","last_became_active":"2024-01-31T05:03:38.231862+0000","last_became_peered":"2024-01-31T05:03:38.231862+0000","last_unstale":"2024-01-31T05:03:40.233115+0000","last_undegraded":"2024-01-31T05:03:40.233115+0000","last_fullsized":"2024-01-31T05:03:40.233115+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.489420+0000","last_change":"2024-01-31T05:03:39.252646+0000","last_active":"2024-01-31T05:03:40.489420+0000","last_peered":"2024-01-31T05:03:40.489420+0000","last_clean":"2024-01-31T05:03:40.489420+0000","last_became_active":"2024-01-31T05:03:39.244259+0000","last_became_peered":"2024-01-31T05:03:39.244259+0000","last_unstale":"2024-01-31T05:03:40.489420+0000","last_undegraded":"2024-01-31T05:03:40.489420+0000","last_fullsized":"2024-01-31T05:03:40.489420+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T11:30:45.496413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490043+0000","last_change":"2024-01-31T05:03:38.239097+0000","last_active":"2024-01-31T05:03:40.490043+0000","last_peered":"2024-01-31T05:03:40.490043+0000","last_clean":"2024-01-31T05:03:40.490043+0000","last_became_active":"2024-01-31T05:03:38.238757+0000","last_became_peered":"2024-01-31T05:03:38.238757+0000","last_unstale":"2024-01-31T05:03:40.490043+0000","last_undegraded":"2024-01-31T05:03:40.490043+0000","last_fullsized":"2024-01-31T05:03:40.490043+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:00:00.029942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231846+0000","last_change":"2024-01-31T05:03:39.244177+0000","last_active":"2024-01-31T05:03:40.231846+0000","last_peered":"2024-01-31T05:03:40.231846+0000","last_clean":"2024-01-31T05:03:40.231846+0000","last_became_active":"2024-01-31T05:03:39.243841+0000","last_became_peered":"2024-01-31T05:03:39.243841+0000","last_unstale":"2024-01-31T05:03:40.231846+0000","last_undegraded":"2024-01-31T05:03:40.231846+0000","last_fullsized":"2024-01-31T05:03:40.231846+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:24:47.464933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.493839+0000","last_change":"2024-01-31T05:03:38.224802+0000","last_active":"2024-01-31T05:03:40.493839+0000","last_peered":"2024-01-31T05:03:40.493839+0000","last_clean":"2024-01-31T05:03:40.493839+0000","last_became_active":"2024-01-31T05:03:38.223989+0000","last_became_peered":"2024-01-31T05:03:38.223989+0000","last_unstale":"2024-01-31T05:03:40.493839+0000","last_undegraded":"2024-01-31T05:03:40.493839+0000","last_fullsized":"2024-01-31T05:03:40.493839+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228606+0000","last_change":"2024-01-31T05:03:39.258544+0000","last_active":"2024-01-31T05:03:40.228606+0000","last_peered":"2024-01-31T05:03:40.228606+0000","last_clean":"2024-01-31T05:03:40.228606+0000","last_became_active":"2024-01-31T05:03:39.257823+0000","last_became_peered":"2024-01-31T05:03:39.257823+0000","last_unstale":"2024-01-31T05:03:40.228606+0000","last_undegraded":"2024-01-31T05:03:40.228606+0000","last_fullsized":"2024-01-31T05:03:40.228606+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T10:05:22.466544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.487893+0000","last_change":"2024-01-31T05:03:38.224866+0000","last_active":"2024-01-31T05:03:40.487893+0000","last_peered":"2024-01-31T05:03:40.487893+0000","last_clean":"2024-01-31T05:03:40.487893+0000","last_became_active":"2024-01-31T05:03:38.224437+0000","last_became_peered":"2024-01-31T05:03:38.224437+0000","last_unstale":"2024-01-31T05:03:40.487893+0000","last_undegraded":"2024-01-31T05:03:40.487893+0000","last_fullsized":"2024-01-31T05:03:40.487893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T13:10:59.850970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488896+0000","last_change":"2024-01-31T05:03:39.264130+0000","last_active":"2024-01-31T05:03:40.488896+0000","last_peered":"2024-01-31T05:03:40.488896+0000","last_clean":"2024-01-31T05:03:40.488896+0000","last_became_active":"2024-01-31T05:03:39.262302+0000","last_became_peered":"2024-01-31T05:03:39.262302+0000","last_unstale":"2024-01-31T05:03:40.488896+0000","last_undegraded":"2024-01-31T05:03:40.488896+0000","last_fullsized":"2024-01-31T05:03:40.488896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:59:26.252153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231109+0000","last_change":"2024-01-31T05:03:38.236959+0000","last_active":"2024-01-31T05:03:40.231109+0000","last_peered":"2024-01-31T05:03:40.231109+0000","last_clean":"2024-01-31T05:03:40.231109+0000","last_became_active":"2024-01-31T05:03:38.235737+0000","last_became_peered":"2024-01-31T05:03:38.235737+0000","last_unstale":"2024-01-31T05:03:40.231109+0000","last_undegraded":"2024-01-31T05:03:40.231109+0000","last_fullsized":"2024-01-31T05:03:40.231109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:14:44.258646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490919+0000","last_change":"2024-01-31T05:03:39.258896+0000","last_active":"2024-01-31T05:03:40.490919+0000","last_peered":"2024-01-31T05:03:40.490919+0000","last_clean":"2024-01-31T05:03:40.490919+0000","last_became_active":"2024-01-31T05:03:39.258637+0000","last_became_peered":"2024-01-31T05:03:39.258637+0000","last_unstale":"2024-01-31T05:03:40.490919+0000","last_undegraded":"2024-01-31T05:03:40.490919+0000","last_fullsized":"2024-01-31T05:03:40.490919+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:59:26.252153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.487893+0000","last_change":"2024-01-31T05:03:38.231189+0000","last_active":"2024-01-31T05:03:40.487893+0000","last_peered":"2024-01-31T05:03:40.487893+0000","last_clean":"2024-01-31T05:03:40.487893+0000","last_became_active":"2024-01-31T05:03:38.230998+0000","last_became_peered":"2024-01-31T05:03:38.230998+0000","last_unstale":"2024-01-31T05:03:40.487893+0000","last_undegraded":"2024-01-31T05:03:40.487893+0000","last_fullsized":"2024-01-31T05:03:40.487893+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:47:32.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227882+0000","last_change":"2024-01-31T05:03:39.247844+0000","last_active":"2024-01-31T05:03:40.227882+0000","last_peered":"2024-01-31T05:03:40.227882+0000","last_clean":"2024-01-31T05:03:40.227882+0000","last_became_active":"2024-01-31T05:03:39.245721+0000","last_became_peered":"2024-01-31T05:03:39.245721+0000","last_unstale":"2024-01-31T05:03:40.227882+0000","last_undegraded":"2024-01-31T05:03:40.227882+0000","last_fullsized":"2024-01-31T05:03:40.227882+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:29:44.344043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.488718+0000","last_change":"2024-01-31T05:03:38.235803+0000","last_active":"2024-01-31T05:03:40.488718+0000","last_peered":"2024-01-31T05:03:40.488718+0000","last_clean":"2024-01-31T05:03:40.488718+0000","last_became_active":"2024-01-31T05:03:38.234927+0000","last_became_peered":"2024-01-31T05:03:38.234927+0000","last_unstale":"2024-01-31T05:03:40.488718+0000","last_undegraded":"2024-01-31T05:03:40.488718+0000","last_fullsized":"2024-01-31T05:03:40.488718+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:53:05.699453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.231591+0000","last_change":"2024-01-31T05:03:39.243770+0000","last_active":"2024-01-31T05:03:40.231591+0000","last_peered":"2024-01-31T05:03:40.231591+0000","last_clean":"2024-01-31T05:03:40.231591+0000","last_became_active":"2024-01-31T05:03:39.243396+0000","last_became_peered":"2024-01-31T05:03:39.243396+0000","last_unstale":"2024-01-31T05:03:40.231591+0000","last_undegraded":"2024-01-31T05:03:40.231591+0000","last_fullsized":"2024-01-31T05:03:40.231591+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T06:58:51.139110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.227758+0000","last_change":"2024-01-31T05:03:38.235205+0000","last_active":"2024-01-31T05:03:40.227758+0000","last_peered":"2024-01-31T05:03:40.227758+0000","last_clean":"2024-01-31T05:03:40.227758+0000","last_became_active":"2024-01-31T05:03:38.234627+0000","last_became_peered":"2024-01-31T05:03:38.234627+0000","last_unstale":"2024-01-31T05:03:40.227758+0000","last_undegraded":"2024-01-31T05:03:40.227758+0000","last_fullsized":"2024-01-31T05:03:40.227758+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:45:13.986778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.228167+0000","last_change":"2024-01-31T05:03:39.248362+0000","last_active":"2024-01-31T05:03:40.228167+0000","last_peered":"2024-01-31T05:03:40.228167+0000","last_clean":"2024-01-31T05:03:40.228167+0000","last_became_active":"2024-01-31T05:03:39.248117+0000","last_became_peered":"2024-01-31T05:03:39.248117+0000","last_unstale":"2024-01-31T05:03:40.228167+0000","last_undegraded":"2024-01-31T05:03:40.228167+0000","last_fullsized":"2024-01-31T05:03:40.228167+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:38.190665+0000","last_clean_scrub_stamp":"2024-01-31T05:03:38.190665+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T15:29:08.685184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.490121+0000","last_change":"2024-01-31T05:03:38.233879+0000","last_active":"2024-01-31T05:03:40.490121+0000","last_peered":"2024-01-31T05:03:40.490121+0000","last_clean":"2024-01-31T05:03:40.490121+0000","last_became_active":"2024-01-31T05:03:38.233495+0000","last_became_peered":"2024-01-31T05:03:38.233495+0000","last_unstale":"2024-01-31T05:03:40.490121+0000","last_undegraded":"2024-01-31T05:03:40.490121+0000","last_fullsized":"2024-01-31T05:03:40.490121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:03:37.186436+0000","last_clean_scrub_stamp":"2024-01-31T05:03:37.186436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T09:10:04.803820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":232,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491830+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:40.491830+0000","last_peered":"2024-01-31T05:03:40.491830+0000","last_clean":"2024-01-31T05:03:40.491830+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:40.491830+0000","last_undegraded":"2024-01-31T05:03:40.491830+0000","last_fullsized":"2024-01-31T05:03:40.491830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":288977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":1.514,"15min":1.514},"last":1.23},{"interface":"front","average":{"1min":1.155,"5min":1.0469999999999999,"15min":1.012},"min":{"1min":0.69499999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0329999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0960000000000001,"15min":1.0880000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0259999999999998,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6890000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.403}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8100000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2869999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8720000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.157}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.78000000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7270000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.2250000000000001},"min":{"1min":0.749,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.792,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.3420000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3540000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9670000000000001,"5min":2.25,"15min":2.25},"last":1.383},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3660000000000001},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8049999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4330000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.476,"15min":1.476},"min":{"1min":1.0169999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.905,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.083},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.92900000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.784,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.2949999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4570000000000001,"15min":1.4810000000000001},"min":{"1min":1.044,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.617,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5409999999999999,"15min":1.5569999999999999},"min":{"1min":1.0209999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8660000000000001,"5min":2.629,"15min":2.629},"last":1.4990000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":289501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0409999999999999,"15min":1.0329999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3240000000000001,"5min":1.476,"15min":1.476},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.05,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.84199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.78600000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.587,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.1200000000000001,"15min":1.131},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5369999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.633}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.1950000000000001,"15min":1.1830000000000001},"min":{"1min":0.85299999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.6950000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1799999999999999,"15min":1.1839999999999999},"min":{"1min":0.80400000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.421,"5min":1.883,"15min":1.883},"last":0.77900000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.381,"15min":1.395},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.72,"5min":1.73,"15min":1.73},"last":1.05},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.4279999999999999},"min":{"1min":0.95599999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0830000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.5589999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.407,"15min":1.423},"min":{"1min":0.878,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.611,"5min":2.032,"15min":2.032},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.585},"min":{"1min":1.0780000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.984,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.508}]},{"osd":6,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.5209999999999999,"15min":1.5329999999999999},"min":{"1min":0.96899999999999997,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.972,"5min":2.335,"15min":2.335},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.5860000000000001,"15min":1.6060000000000001},"min":{"1min":1.2609999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.768,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.599,"15min":1.6160000000000001},"min":{"1min":0.98599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8620000000000001,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.137},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.637,"15min":1.659},"min":{"1min":1.0920000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.891,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.7729999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":655360,"data_stored":288975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":1.0900000000000001,"15min":1.151},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1919999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":1.032,"15min":1.0840000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.1439999999999999,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.615}]},{"osd":1,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":1.121,"15min":1.1779999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4910000000000001,"5min":5.617,"15min":5.617},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.2330000000000001,"15min":1.2989999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.032}]},{"osd":2,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.292,"15min":1.3680000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0489999999999999,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.484,"15min":1.5720000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4930000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.5680000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.6220000000000001,"15min":1.7230000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.9990000000000001,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.621,"15min":1.7010000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.696,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.4410000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.639,"15min":1.7230000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9339999999999999,"5min":6.048,"15min":6.048},"last":1.3},{"interface":"front","average":{"1min":1.216,"5min":1.7430000000000001,"15min":1.831},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.952,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.383}]},{"osd":6,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.6739999999999999,"15min":1.744},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.024,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.657,"15min":1.748},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5700000000000001,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.337}]},{"osd":7,"last update":"Wed Jan 31 05:02:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.6539999999999999,"15min":1.74},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.802,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":1.486},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.8380000000000001,"15min":1.946},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.73,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.5269999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":290657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1080000000000001,"5min":1.361,"15min":1.361},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4390000000000001,"5min":1.73,"15min":1.73},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.536,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.167},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3580000000000001,"5min":1.671,"15min":1.671},"last":1.2330000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.544,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.504},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0289999999999999,"15min":1.071},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":0.80500000000000005}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.72,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.272,"15min":1.3140000000000001},"min":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.0049999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.218,"15min":1.236},"min":{"1min":0.746,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5549999999999999,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.387},{"interface":"front","average":{"1min":1.133,"5min":1.272,"15min":1.306},"min":{"1min":0.65300000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.522,"5min":2.52,"15min":2.52},"last":1.2749999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.89800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.663,"5min":2.992,"15min":2.992},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.3520000000000001,"15min":1.3939999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.631,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.0580000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.3859999999999999},"min":{"1min":0.76700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5349999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.3700000000000001,"15min":1.4059999999999999},"min":{"1min":0.76800000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.619,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.3140000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":879835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98299999999999998,"15min":1.0680000000000001},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.409,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.86199999999999999,"15min":0.90800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4890000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.94899999999999995,"15min":1.0169999999999999},"min":{"1min":0.52700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.3440000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.163},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.321,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":0.91000000000000003}]},{"osd":3,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.99099999999999999,"15min":1.038},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.605,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.111,"15min":1.204},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4830000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3859999999999999,"15min":1.486},"min":{"1min":0.78600000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5469999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3009999999999999,"15min":1.365},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5469999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3540000000000001,"15min":1.4299999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.611,"5min":2.722,"15min":2.722},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.4390000000000001},"min":{"1min":0.84099999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5189999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.88100000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.427,"15min":1.5209999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.625,"5min":2.577,"15min":2.577},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4410000000000001},"min":{"1min":0.85199999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6930000000000001,"5min":2.794,"15min":2.794},"last":0.85199999999999998}]},{"osd":7,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4390000000000001,"15min":1.5269999999999999},"min":{"1min":0.95599999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.9870000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.365,"5min":1.5089999999999999,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8460000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.97799999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":655360,"data_stored":288907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.135,"15min":1.151},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.1220000000000001,"15min":1.1279999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.552,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.72899999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1339999999999999,"15min":1.131},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5589999999999999,"5min":1.573,"15min":1.573},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1459999999999999,"15min":1.1559999999999999},"min":{"1min":0.73799999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5349999999999999,"5min":1.976,"15min":1.976},"last":1.089}]},{"osd":2,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7470000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.266,"15min":1.28},"min":{"1min":0.79900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.544,"5min":1.873,"15min":1.873},"last":1.024}]},{"osd":3,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3580000000000001,"15min":1.375},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"max":{"1min":1.7050000000000001,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.335,"15min":1.339},"min":{"1min":0.95899999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":1.228}]},{"osd":4,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.254,"15min":1.2529999999999999},"min":{"1min":0.79300000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.6890000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.47,"15min":1.4790000000000001},"min":{"1min":0.94399999999999995,"5min":0.628,"15min":0.628},"max":{"1min":2.0499999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.421}]},{"osd":5,"last update":"Wed Jan 31 05:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.4430000000000001,"15min":1.454},"min":{"1min":1.0049999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.679,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.593,"15min":1.6040000000000001},"min":{"1min":1.1990000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0990000000000002,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.633}]},{"osd":7,"last update":"Wed Jan 31 05:02:47 2024","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.5620000000000001,"15min":1.5820000000000001},"min":{"1min":0.89400000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9530000000000001,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.482},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.6719999999999999,"15min":1.6839999999999999},"min":{"1min":1.246,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.9990000000000001,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.53}]}]},{"osd":5,"up_from":14,"seq":60129542184,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":879287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.038,"15min":1.022},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.117,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.71799999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.5900000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.1279999999999999,"15min":1.141},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.169},{"interface":"front","average":{"1min":1.099,"5min":1.2030000000000001,"15min":1.22},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.5720000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.458}]},{"osd":2,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.206,"15min":1.222},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.663,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.296,"15min":1.3089999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.647,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.304,"15min":1.306},"min":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.704,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.389,"15min":1.399},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.712,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.514}]},{"osd":4,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.369,"15min":1.3720000000000001},"min":{"1min":0.70399999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.0139999999999998,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.9530000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3740000000000001,"15min":1.3819999999999999},"min":{"1min":0.91400000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.792,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.8280000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.4259999999999999,"15min":1.4099999999999999},"min":{"1min":0.93500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":2.0510000000000002},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4890000000000001,"15min":1.494},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.851,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.7450000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.494,"15min":1.496},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.948,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.653},{"interface":"front","average":{"1min":1.617,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":1.0029999999999999,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.5219999999999998,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":2.2040000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":290497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.85699999999999998,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.77,"5min":3.77,"15min":3.77},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0389999999999999,"15min":1.0489999999999999},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.0680000000000001,"5min":3.282,"15min":3.282},"last":1.2909999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.99299999999999999,"15min":1.012},"min":{"1min":0.52400000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.23,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.84499999999999997,"15min":0.86299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.075,"5min":1.774,"15min":1.774},"last":1.073}]},{"osd":3,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.56699999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.6880000000000002,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.103,"5min":1.091,"15min":1.089},"min":{"1min":0.56299999999999994,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.97899999999999998}]},{"osd":4,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.2250000000000001},"min":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4370000000000001,"15min":1.4399999999999999},"min":{"1min":0.93999999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":3.8500000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.4379999999999999,"15min":1.464},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7709999999999999,"5min":5.75,"15min":5.75},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.4099999999999999,"15min":1.456},"min":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5089999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":1.3580000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.323,"15min":1.319},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":7.415,"15min":7.415},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.7130000000000001,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.4410000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:02:43 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.3939999999999999},"min":{"1min":0.76000000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":3.633,"5min":3.633,"15min":3.633},"last":1.486},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5589999999999999,"15min":1.569},"min":{"1min":0.877,"5min":0.751,"15min":0.751},"max":{"1min":3.9169999999999998,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.3340000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:03:49.100 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-31T05:03:49.100 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-31T05:03:49.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.359+0000 7f6d8d600640 1 Processor -- start 2024-01-31T05:03:49.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.359+0000 7f6d8d600640 1 -- start start 2024-01-31T05:03:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d8d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880adea0 0x7f6d880ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d8d600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6d881599a0 0x7f6d880cb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d8d600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d880b1d10 0x7f6d880ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d8d600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d88159e00 con 0x7f6d880b1d10 2024-01-31T05:03:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d8d600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880ae280 con 0x7f6d881599a0 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d8d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880cc5e0 con 0x7f6d880adea0 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d87400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d880b1d10 0x7f6d880ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d87400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d880b1d10 0x7f6d880ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41926/0 (socket says 172.21.15.97:41926) 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d87400640 1 -- 172.21.15.97:0/7070779 learned_addr learned my addr 172.21.15.97:0/7070779 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d86000640 1 --2- 172.21.15.97:0/7070779 >> v2:172.21.15.188:3300/0 conn(0x7f6d881599a0 0x7f6d880cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.361+0000 7f6d86a00640 1 --2- 172.21.15.97:0/7070779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880adea0 0x7f6d880ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.362+0000 7f6d87400640 1 -- 172.21.15.97:0/7070779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880adea0 msgr2=0x7f6d880ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.362+0000 7f6d87400640 1 --2- 172.21.15.97:0/7070779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880adea0 0x7f6d880ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.362+0000 7f6d87400640 1 -- 172.21.15.97:0/7070779 >> v2:172.21.15.188:3300/0 conn(0x7f6d881599a0 msgr2=0x7f6d880cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.362+0000 7f6d87400640 1 --2- 172.21.15.97:0/7070779 >> v2:172.21.15.188:3300/0 conn(0x7f6d881599a0 0x7f6d880cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.362+0000 7f6d87400640 1 -- 172.21.15.97:0/7070779 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8815e210 con 0x7f6d880b1d10 2024-01-31T05:03:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.362+0000 7f6d86000640 1 --2- 172.21.15.97:0/7070779 >> v2:172.21.15.188:3300/0 conn(0x7f6d881599a0 0x7f6d880cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.363+0000 7f6d87400640 1 --2- 172.21.15.97:0/7070779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880b1d10 0x7f6d880ccd00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6d88072d90 tx=0x7f6d8002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f211e8bf234acea6 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.363+0000 7f6d85600640 1 -- 172.21.15.97:0/7070779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8002e9e0 con 0x7f6d880b1d10 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.363+0000 7f6d85600640 1 -- 172.21.15.97:0/7070779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8002eb40 con 0x7f6d880b1d10 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.363+0000 7f6d85600640 1 -- 172.21.15.97:0/7070779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80009600 con 0x7f6d880b1d10 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 -- 172.21.15.97:0/7070779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880b1d10 msgr2=0x7f6d880ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 --2- 172.21.15.97:0/7070779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880b1d10 0x7f6d880ccd00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6d88072d90 tx=0x7f6d8002e040 comp rx=0 tx=0).stop 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 -- 172.21.15.97:0/7070779 shutdown_connections 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 --2- 172.21.15.97:0/7070779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880b1d10 0x7f6d880ccd00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 --2- 172.21.15.97:0/7070779 >> v2:172.21.15.188:3300/0 conn(0x7f6d881599a0 0x7f6d880cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 --2- 172.21.15.97:0/7070779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880adea0 0x7f6d880ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 -- 172.21.15.97:0/7070779 >> 172.21.15.97:0/7070779 conn(0x7f6d880b0a10 msgr2=0x7f6d880b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 -- 172.21.15.97:0/7070779 shutdown_connections 2024-01-31T05:03:49.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.364+0000 7f6d8d600640 1 -- 172.21.15.97:0/7070779 wait complete. 2024-01-31T05:03:49.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.366+0000 7f6d8d600640 1 Processor -- start 2024-01-31T05:03:49.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.366+0000 7f6d8d600640 1 -- start start 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d8d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 0x7f6d8809ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d8d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880b1d10 0x7f6d881e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d8d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d881599a0 0x7f6d880709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d8d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8815bb10 con 0x7f6d880adea0 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d8d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d881e82b0 con 0x7f6d881599a0 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d8d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d881e8590 con 0x7f6d880b1d10 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 0x7f6d8809ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 0x7f6d8809ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41936/0 (socket says 172.21.15.97:41936) 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 -- 172.21.15.97:0/3101002843 learned_addr learned my addr 172.21.15.97:0/3101002843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 -- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880b1d10 msgr2=0x7f6d881e79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880b1d10 0x7f6d881e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 -- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d881599a0 msgr2=0x7f6d880709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d87400640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d881599a0 0x7f6d880709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d881599a0 0x7f6d880709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.367+0000 7f6d86a00640 1 -- 172.21.15.97:0/3101002843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8000bd40 con 0x7f6d880adea0 2024-01-31T05:03:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.368+0000 7f6d86a00640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 0x7f6d8809ace0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800d800 tx=0x7f6d7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.368+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780096a0 con 0x7f6d880adea0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.368+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78009800 con 0x7f6d880adea0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.368+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78009b90 con 0x7f6d880adea0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.368+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d881e8870 con 0x7f6d880adea0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.368+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8815b1e0 con 0x7f6d880adea0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.369+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d88073080 con 0x7f6d880adea0 2024-01-31T05:03:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.370+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f6d78009cf0 con 0x7f6d880adea0 2024-01-31T05:03:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.371+0000 7f6d7ea00640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6d6006aed0 0x7f6d6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.371+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6d78089d30 con 0x7f6d880adea0 2024-01-31T05:03:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.371+0000 7f6d86000640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6d6006aed0 0x7f6d6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.372+0000 7f6d86000640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6d6006aed0 0x7f6d6006d390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d70006fd0 tx=0x7f6d70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.375+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6d7808f020 con 0x7f6d880adea0 2024-01-31T05:03:49.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.639+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6d8815bd80 con 0x7f6d880adea0 2024-01-31T05:03:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.644+0000 7f6d7ea00640 1 -- 172.21.15.97:0/3101002843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6d780584b0 con 0x7f6d880adea0 2024-01-31T05:03:49.643 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:03:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:03:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-01-31T05:03:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:03:49.644 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-31T05:03:49.644 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:03:49.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.646+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6d6006aed0 msgr2=0x7f6d6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.646+0000 7f6d8d600640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6d6006aed0 0x7f6d6006d390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d70006fd0 tx=0x7f6d70008040 comp rx=0 tx=0).stop 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.646+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 msgr2=0x7f6d8809ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.646+0000 7f6d8d600640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 0x7f6d8809ace0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6d7800d800 tx=0x7f6d7800dcc0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 shutdown_connections 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d881599a0 0x7f6d880709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d880b1d10 0x7f6d881e79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f6d6006aed0 0x7f6d6006d390 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 --2- 172.21.15.97:0/3101002843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d880adea0 0x7f6d8809ace0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 >> 172.21.15.97:0/3101002843 conn(0x7f6d880b0a10 msgr2=0x7f6d880abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 shutdown_connections 2024-01-31T05:03:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.647+0000 7f6d8d600640 1 -- 172.21.15.97:0/3101002843 wait complete. 2024-01-31T05:03:49.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:03:49.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.871+0000 7fae79600640 1 Processor -- start 2024-01-31T05:03:49.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.871+0000 7fae79600640 1 -- start start 2024-01-31T05:03:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae79600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae7415b400 0x7fae7406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae79600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fae7406db70 0x7fae740ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae79600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7406d1a0 con 0x7fae740cb810 2024-01-31T05:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae79600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae740cc960 con 0x7fae7415b400 2024-01-31T05:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae79600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7406c600 con 0x7fae7406db70 2024-01-31T05:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae6be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae7415b400 0x7fae7406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae6be00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae7415b400 0x7fae7406d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41946/0 (socket says 172.21.15.97:41946) 2024-01-31T05:03:49.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae6be00640 1 -- 172.21.15.97:0/242271598 learned_addr learned my addr 172.21.15.97:0/242271598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.873+0000 7fae73400640 1 --2- 172.21.15.97:0/242271598 >> v2:172.21.15.188:3300/0 conn(0x7fae7406db70 0x7fae740ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.874+0000 7fae6be00640 1 -- 172.21.15.97:0/242271598 >> v2:172.21.15.188:3300/0 conn(0x7fae7406db70 msgr2=0x7fae740ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.874+0000 7fae6be00640 1 --2- 172.21.15.97:0/242271598 >> v2:172.21.15.188:3300/0 conn(0x7fae7406db70 0x7fae740ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.874+0000 7fae6be00640 1 -- 172.21.15.97:0/242271598 >> v1:172.21.15.188:6789/0 conn(0x7fae740cb810 legacy=0x7fae740cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:49.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.874+0000 7fae6be00640 1 -- 172.21.15.97:0/242271598 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7406c870 con 0x7fae7415b400 2024-01-31T05:03:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.875+0000 7fae6be00640 1 --2- 172.21.15.97:0/242271598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7406d630 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fae740ae210 tx=0x7fae5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200e04e270a1b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.875+0000 7fae72000640 1 -- 172.21.15.97:0/242271598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c031870 con 0x7fae7415b400 2024-01-31T05:03:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.875+0000 7fae72000640 1 -- 172.21.15.97:0/242271598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c0319d0 con 0x7fae7415b400 2024-01-31T05:03:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.875+0000 7fae72000640 1 -- 172.21.15.97:0/242271598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c031d60 con 0x7fae7415b400 2024-01-31T05:03:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 -- 172.21.15.97:0/242271598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 msgr2=0x7fae7406d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 --2- 172.21.15.97:0/242271598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7406d630 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fae740ae210 tx=0x7fae5c02b910 comp rx=0 tx=0).stop 2024-01-31T05:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 -- 172.21.15.97:0/242271598 shutdown_connections 2024-01-31T05:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 --2- 172.21.15.97:0/242271598 >> v2:172.21.15.188:3300/0 conn(0x7fae7406db70 0x7fae740ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 --2- 172.21.15.97:0/242271598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7406d630 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 -- 172.21.15.97:0/242271598 >> 172.21.15.97:0/242271598 conn(0x7fae740b0a10 msgr2=0x7fae740b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 -- 172.21.15.97:0/242271598 shutdown_connections 2024-01-31T05:03:49.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.876+0000 7fae79600640 1 -- 172.21.15.97:0/242271598 wait complete. 2024-01-31T05:03:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.878+0000 7fae79600640 1 Processor -- start 2024-01-31T05:03:49.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.878+0000 7fae79600640 1 -- start start 2024-01-31T05:03:49.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae79600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7406db70 0x7fae7415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae79600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae740cb810 0x7fae7414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae79600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae79600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7415be60 con 0x7fae7415b400 2024-01-31T05:03:49.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae79600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7415b880 con 0x7fae7406db70 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae79600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae74154ce0 con 0x7fae740cb810 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae73400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae73400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7409aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41962/0 (socket says 172.21.15.97:41962) 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.879+0000 7fae73400640 1 -- 172.21.15.97:0/94312191 learned_addr learned my addr 172.21.15.97:0/94312191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae6be00640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae740cb810 0x7fae7414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae72a00640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7406db70 0x7fae7415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae73400640 1 -- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae740cb810 msgr2=0x7fae7414c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae73400640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae740cb810 0x7fae7414c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae73400640 1 -- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7406db70 msgr2=0x7fae7415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae73400640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7406db70 0x7fae7415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:49.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae73400640 1 -- 172.21.15.97:0/94312191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0094d0 con 0x7fae7415b400 2024-01-31T05:03:49.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae73400640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7409aa00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fae6c007e90 tx=0x7fae6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:49.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c014620 con 0x7fae7415b400 2024-01-31T05:03:49.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c014780 con 0x7fae7415b400 2024-01-31T05:03:49.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.880+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c014b30 con 0x7fae7415b400 2024-01-31T05:03:49.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.881+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7415c830 con 0x7fae7415b400 2024-01-31T05:03:49.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.881+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7415baf0 con 0x7fae7415b400 2024-01-31T05:03:49.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.882+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7415c010 con 0x7fae7415b400 2024-01-31T05:03:49.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.883+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fae6c014c90 con 0x7fae7415b400 2024-01-31T05:03:49.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.884+0000 7fae6aa00640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae4c06aed0 0x7fae4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:49.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.884+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae6c08a150 con 0x7fae7415b400 2024-01-31T05:03:49.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.884+0000 7fae72a00640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae4c06aed0 0x7fae4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:49.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.885+0000 7fae72a00640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae4c06aed0 0x7fae4c06d390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae64004ea0 tx=0x7fae64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:49.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:49.889+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fae7415c010 con 0x7fae7415b400 2024-01-31T05:03:50.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.122+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae7415c010 con 0x7fae7415b400 2024-01-31T05:03:50.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.123+0000 7fae6aa00640 1 -- 172.21.15.97:0/94312191 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fae7415c010 con 0x7fae7415b400 2024-01-31T05:03:50.123 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:50.123 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:03:38.646144+0000","modified":"2024-01-31T05:03:41.747278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3986947126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3986947126},{"type":"v1","addr":"172.21.15.97:6837","nonce":3986947126}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-31T05:03:50.123 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae4c06aed0 msgr2=0x7fae4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae4c06aed0 0x7fae4c06d390 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae64004ea0 tx=0x7fae64009040 comp rx=0 tx=0).stop 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 msgr2=0x7fae7409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7409aa00 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fae6c007e90 tx=0x7fae6c00e4b0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 shutdown_connections 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae4c06aed0 0x7fae4c06d390 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae7415b400 0x7fae7409aa00 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae740cb810 0x7fae7414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 --2- 172.21.15.97:0/94312191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7406db70 0x7fae7415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 >> 172.21.15.97:0/94312191 conn(0x7fae740b0a10 msgr2=0x7fae740abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 shutdown_connections 2024-01-31T05:03:50.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.126+0000 7fae79600640 1 -- 172.21.15.97:0/94312191 wait complete. 2024-01-31T05:03:50.547 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-31T05:03:38.646144+0000', 'modified': '2024-01-31T05:03:41.747278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6240}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6240': {'gid': 6240, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/3986947126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3986947126}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3986947126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-31T05:03:50.547 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-31T05:03:50.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-31T05:03:50.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.760+0000 7fbb2f400640 1 Processor -- start 2024-01-31T05:03:50.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.760+0000 7fbb2f400640 1 -- start start 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.761+0000 7fbb2f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2815a730 0x7fbb280abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.761+0000 7fbb2f400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb280cde10 0x7fbb280ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.761+0000 7fbb2f400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2815abc0 con 0x7fbb2815a350 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.761+0000 7fbb2f400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb280b2c90 con 0x7fbb280cde10 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.761+0000 7fbb2f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280ce7c0 con 0x7fbb2815a730 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb280cde10 0x7fbb280ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb280cde10 0x7fbb280ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41974/0 (socket says 172.21.15.97:41974) 2024-01-31T05:03:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 -- 172.21.15.97:0/3390482251 learned_addr learned my addr 172.21.15.97:0/3390482251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 -- 172.21.15.97:0/3390482251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2815a730 msgr2=0x7fbb280abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb27e00640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2815a730 0x7fbb280abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2815a730 0x7fbb280abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 -- 172.21.15.97:0/3390482251 >> v1:172.21.15.188:6789/0 conn(0x7fbb2815a350 legacy=0x7fbb2806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb2d400640 1 -- 172.21.15.97:0/3390482251 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2815aea0 con 0x7fbb280cde10 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.762+0000 7fbb27e00640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2815a730 0x7fbb280abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.763+0000 7fbb2d400640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb280cde10 0x7fbb280ce280 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009e80 tx=0x7fbb2002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722f0e2b95dc1871 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.764+0000 7fbb27400640 1 -- 172.21.15.97:0/3390482251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2002ca60 con 0x7fbb280cde10 2024-01-31T05:03:50.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.764+0000 7fbb27400640 1 -- 172.21.15.97:0/3390482251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2002cbc0 con 0x7fbb280cde10 2024-01-31T05:03:50.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.764+0000 7fbb27400640 1 -- 172.21.15.97:0/3390482251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb200343f0 con 0x7fbb280cde10 2024-01-31T05:03:50.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.764+0000 7fbb2f400640 1 -- 172.21.15.97:0/3390482251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb280cde10 msgr2=0x7fbb280ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.764+0000 7fbb2f400640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb280cde10 0x7fbb280ce280 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009e80 tx=0x7fbb2002bec0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.765+0000 7fbb2f400640 1 -- 172.21.15.97:0/3390482251 shutdown_connections 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.765+0000 7fbb2f400640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb280cde10 0x7fbb280ce280 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.765+0000 7fbb2f400640 1 --2- 172.21.15.97:0/3390482251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2815a730 0x7fbb280abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.765+0000 7fbb2f400640 1 -- 172.21.15.97:0/3390482251 >> 172.21.15.97:0/3390482251 conn(0x7fbb280b0a10 msgr2=0x7fbb280b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.765+0000 7fbb2f400640 1 -- 172.21.15.97:0/3390482251 shutdown_connections 2024-01-31T05:03:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.765+0000 7fbb2f400640 1 -- 172.21.15.97:0/3390482251 wait complete. 2024-01-31T05:03:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.766+0000 7fbb2f400640 1 Processor -- start 2024-01-31T05:03:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.767+0000 7fbb2f400640 1 -- start start 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.767+0000 7fbb2f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280cde10 0x7fbb280af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.767+0000 7fbb2f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 0x7fbb2809b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb2f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2814f2c0 0x7fbb2809b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb2f400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2814f150 con 0x7fbb2815a350 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb2f400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2815c680 con 0x7fbb280cde10 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb2f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2815ba70 con 0x7fbb2814f2c0 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 0x7fbb2809b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 0x7fbb2809b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41988/0 (socket says 172.21.15.97:41988) 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 -- 172.21.15.97:0/278667331 learned_addr learned my addr 172.21.15.97:0/278667331 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280cde10 0x7fbb280af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 -- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2814f2c0 msgr2=0x7fbb2809b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2814f2c0 0x7fbb2809b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 -- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280cde10 msgr2=0x7fbb280af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280cde10 0x7fbb280af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb27e00640 1 -- 172.21.15.97:0/278667331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb20009b30 con 0x7fbb2815a350 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.768+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280cde10 0x7fbb280af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.769+0000 7fbb27e00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 0x7fbb2809b400 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000c7f0 tx=0x7fbb1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.769+0000 7fbb24c00640 1 -- 172.21.15.97:0/278667331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10007820 con 0x7fbb2815a350 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.769+0000 7fbb24c00640 1 -- 172.21.15.97:0/278667331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10007980 con 0x7fbb2815a350 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.769+0000 7fbb24c00640 1 -- 172.21.15.97:0/278667331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10007cf0 con 0x7fbb2815a350 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.769+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2815bd50 con 0x7fbb2815a350 2024-01-31T05:03:50.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.769+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2815c310 con 0x7fbb2815a350 2024-01-31T05:03:50.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.772+0000 7fbb24c00640 1 -- 172.21.15.97:0/278667331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fbb100113f0 con 0x7fbb2815a350 2024-01-31T05:03:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.773+0000 7fbb24c00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbb0006af00 0x7fbb0006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.773+0000 7fbb24c00640 1 -- 172.21.15.97:0/278667331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb10089b30 con 0x7fbb2815a350 2024-01-31T05:03:50.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.773+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbb0006af00 0x7fbb0006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.774+0000 7fbb3236a740 1 Processor -- start 2024-01-31T05:03:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.774+0000 7fbb3236a740 1 -- start start 2024-01-31T05:03:50.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.774+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbb0006af00 0x7fbb0006d3c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbb180016b0 tx=0x7fbb18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb8368a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb8398810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 0x556bb8398d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb3236a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb82dbf50 con 0x556bb8368f60 2024-01-31T05:03:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb82d9640 con 0x556bb8395e60 2024-01-31T05:03:50.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bb82d4f30 con 0x556bb8395a80 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 0x556bb8398d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 0x556bb8398d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41996/0 (socket says 172.21.15.97:41996) 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb2d400640 1 -- 172.21.15.97:0/1858843105 learned_addr learned my addr 172.21.15.97:0/1858843105 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.775+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb8368a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb8398810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb2d400640 1 -- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 msgr2=0x556bb8368a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb2d400640 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb8368a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb2d400640 1 -- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 msgr2=0x556bb8398810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb2d400640 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb8398810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb2d400640 1 -- 172.21.15.97:0/1858843105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bb82972d0 con 0x556bb8368f60 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb8398810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:50.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb2d400640 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 0x556bb8398d50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009fa0 tx=0x7fbb2002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ceaa867b002a7d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb1d200640 1 -- 172.21.15.97:0/1858843105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2002c860 con 0x556bb8368f60 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb1d200640 1 -- 172.21.15.97:0/1858843105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2002c9c0 con 0x556bb8368f60 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.776+0000 7fbb1d200640 1 -- 172.21.15.97:0/1858843105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2003e690 con 0x556bb8368f60 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 -- 172.21.15.97:0/1858843105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 msgr2=0x556bb8398d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 0x556bb8398d50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009fa0 tx=0x7fbb2002e070 comp rx=0 tx=0).stop 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 -- 172.21.15.97:0/1858843105 shutdown_connections 2024-01-31T05:03:50.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8368f60 0x556bb8398d50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb8398810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1858843105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb8368a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 -- 172.21.15.97:0/1858843105 >> 172.21.15.97:0/1858843105 conn(0x556bb836ac90 msgr2=0x556bb83b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 -- 172.21.15.97:0/1858843105 shutdown_connections 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.777+0000 7fbb3236a740 1 -- 172.21.15.97:0/1858843105 wait complete. 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.778+0000 7fbb3236a740 1 -- start start 2024-01-31T05:03:50.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.778+0000 7fbb3236a740 1 Processor -- start 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb839b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb83999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb8383910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb828e6b0 con 0x556bb8395e60 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb828e350 con 0x556bb8383490 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bb8036f20 con 0x556bb8395a80 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb3236a740 10 client.0 _ll_register_callbacks cb 0x556bb82909c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb83999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb83999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42012/0 (socket says 172.21.15.97:42012) 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.779+0000 7fbb27e00640 1 -- 172.21.15.97:0/1615617106 learned_addr learned my addr 172.21.15.97:0/1615617106 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb839b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb2d400640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb8383910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb27e00640 1 -- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 msgr2=0x556bb839b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb839b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb27e00640 1 -- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 msgr2=0x556bb8383910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb8383910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb27e00640 1 -- 172.21.15.97:0/1615617106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2815aea0 con 0x556bb8395e60 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb2d400640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb8383910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb83999b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbb100602b0 tx=0x7fbb1005c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2102e7b20767b49d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb15600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:03:50.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb15600640 1 -- 172.21.15.97:0/1615617106 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1006e030 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb15600640 1 -- 172.21.15.97:0/1615617106 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb1006f070 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bb82fce60 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb15600640 1 -- 172.21.15.97:0/1615617106 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb10056940 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.780+0000 7fbb3236a740 10 client.6411 fetch_fsmap learned FSMap version 27 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x556bb82d35b0 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb15600640 1 -- 172.21.15.97:0/1615617106 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556bb82fce60 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb15600640 1 -- 172.21.15.97:0/1615617106 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb1005c170 con 0x556bb8395e60 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 10 client.6411 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 10 client.6411 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 20 client.6411 populate_metadata read hostname 'smithi097' 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb8397310 0x556bb83976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 4 client.6411 mds_command: new command op to 6240 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-31T05:03:50.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 --> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bb82a6c40 con 0x556bb8397310 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.781+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb8397310 0x556bb83976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.782+0000 7fbb15600640 10 client.6411 ms_handle_connect on v2:172.21.15.97:6836/3986947126 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.782+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb8397310 0x556bb83976f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c75800cfa7f625 server_cookie=9fbd07c5e7a100a1 in_seq=0 out_seq=0 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb15600640 1 -- 172.21.15.97:0/1615617106 <== mds.0 v2:172.21.15.97:6836/3986947126 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bb82a6c40 con 0x556bb8397310 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb15600640 10 client.6411 handle_command_reply: tid=0 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 client.6411 shutdown 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 msgr2=0x556bb83999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb83999b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbb100602b0 tx=0x7fbb1005c630 comp rx=0 tx=0).stop 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 shutdown_connections 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb8397310 0x556bb83976f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb8383910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb15600640 0 client.6411 ms_handle_reset on v2:172.21.15.97:6836/3986947126 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb83999b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1615617106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb839b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 >> 172.21.15.97:0/1615617106 conn(0x556bb8368f60 msgr2=0x556bb83b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 shutdown_connections 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 1 -- 172.21.15.97:0/1615617106 wait complete. 2024-01-31T05:03:50.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.783+0000 7fbb3236a740 20 client.6411 trim_cache size 0 max 16384 2024-01-31T05:03:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.800+0000 7fbb3236a740 1 Processor -- start 2024-01-31T05:03:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 -- start start 2024-01-31T05:03:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb836c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb83902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb8390810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb839a8d0 con 0x556bb8395e60 2024-01-31T05:03:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb827f990 con 0x556bb8383490 2024-01-31T05:03:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.801+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bb83679d0 con 0x556bb8395a80 2024-01-31T05:03:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb8390810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb8390810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42016/0 (socket says 172.21.15.97:42016) 2024-01-31T05:03:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 -- 172.21.15.97:0/3909971673 learned_addr learned my addr 172.21.15.97:0/3909971673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb836c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb27e00640 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb83902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 -- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 msgr2=0x556bb83902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb83902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 -- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 msgr2=0x556bb836c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb836c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2d400640 1 -- 172.21.15.97:0/3909971673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bb82701e0 con 0x556bb8395e60 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.802+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb836c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb2d400640 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb8390810 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbb2003f000 tx=0x7fbb20030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68993f68a7fa75d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb17e00640 1 -- 172.21.15.97:0/3909971673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20003b90 con 0x556bb8395e60 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb17e00640 1 -- 172.21.15.97:0/3909971673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20003cf0 con 0x556bb8395e60 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb17e00640 1 -- 172.21.15.97:0/3909971673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20004260 con 0x556bb8395e60 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 -- 172.21.15.97:0/3909971673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 msgr2=0x556bb8390810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb8390810 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbb2003f000 tx=0x7fbb20030000 comp rx=0 tx=0).stop 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 -- 172.21.15.97:0/3909971673 shutdown_connections 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395e60 0x556bb8390810 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8395a80 0x556bb83902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 --2- 172.21.15.97:0/3909971673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8383490 0x556bb836c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.803+0000 7fbb3236a740 1 -- 172.21.15.97:0/3909971673 >> 172.21.15.97:0/3909971673 conn(0x556bb8368f60 msgr2=0x556bb83b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.804+0000 7fbb3236a740 1 -- 172.21.15.97:0/3909971673 shutdown_connections 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.804+0000 7fbb3236a740 1 -- 172.21.15.97:0/3909971673 wait complete. 2024-01-31T05:03:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.804+0000 7fbb3236a740 1 -- start start 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.804+0000 7fbb3236a740 1 Processor -- start 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.805+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8383490 0x556bb8383aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.805+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 0x556bb83cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.805+0000 7fbb3236a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb83906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.805+0000 7fbb3236a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb83b38d0 con 0x556bb8395a80 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.805+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bb83cd410 con 0x556bb8395e60 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.805+0000 7fbb3236a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bb8369cf0 con 0x556bb8383490 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb3236a740 10 client.0 _ll_register_callbacks cb 0x7fbb100582b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 0x556bb83cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 0x556bb83cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42028/0 (socket says 172.21.15.97:42028) 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 -- 172.21.15.97:0/1563790749 learned_addr learned my addr 172.21.15.97:0/1563790749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb2d400640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb83906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8383490 0x556bb8383aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 -- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8383490 msgr2=0x556bb8383aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8383490 0x556bb8383aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 -- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 msgr2=0x556bb83906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb83906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb27e00640 1 -- 172.21.15.97:0/1563790749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bb82972d0 con 0x556bb8395a80 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.806+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8383490 0x556bb8383aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb27e00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 0x556bb83cc9d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000a0a0 tx=0x7fbb10087d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cffd9142e0b515c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb15600640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb15600640 1 -- 172.21.15.97:0/1563790749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1006e030 con 0x556bb8395a80 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb15600640 1 -- 172.21.15.97:0/1563790749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb1006f070 con 0x556bb8395a80 2024-01-31T05:03:50.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb2002c9c0 con 0x556bb8395a80 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb15600640 1 -- 172.21.15.97:0/1563790749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb10054820 con 0x556bb8395a80 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 10 client.6417 fetch_fsmap learned FSMap version 27 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x556bb8295b30 con 0x556bb8395a80 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb15600640 1 -- 172.21.15.97:0/1563790749 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1007a030 con 0x556bb8395a80 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb15600640 1 -- 172.21.15.97:0/1563790749 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb100542d0 con 0x556bb8395a80 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 10 client.6417 fetch_fsmap finished waiting for FSMap version 27 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 10 client.6417 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 20 client.6417 populate_metadata read hostname 'smithi097' 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb83976f0 0x556bb8397ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 4 client.6417 mds_command: new command op to 6240 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.807+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 --> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556bb82cddc0 con 0x556bb83976f0 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb83976f0 0x556bb8397ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb2ca00640 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb83976f0 0x556bb8397ad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4eac3932df46e07 server_cookie=82347996fd65bbc0 in_seq=0 out_seq=0 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb15600640 10 client.6417 ms_handle_connect on v2:172.21.15.97:6836/3986947126 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb15600640 1 -- 172.21.15.97:0/1563790749 <== mds.0 v2:172.21.15.97:6836/3986947126 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x556bb82cddc0 con 0x556bb83976f0 2024-01-31T05:03:50.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb15600640 10 client.6417 handle_command_reply: tid=0 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb3236a740 1 client.6417 shutdown 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 msgr2=0x556bb83cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.808+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 0x556bb83cc9d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000a0a0 tx=0x7fbb10087d50 comp rx=0 tx=0).stop 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 shutdown_connections 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:6836/3986947126,v1:172.21.15.97:6837/3986947126] conn(0x556bb83976f0 0x556bb8397ad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556bb8395e60 0x556bb83906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556bb8395a80 0x556bb83cc9d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb15600640 0 client.6417 ms_handle_reset on v2:172.21.15.97:6836/3986947126 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 --2- 172.21.15.97:0/1563790749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556bb8383490 0x556bb8383aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 >> 172.21.15.97:0/1563790749 conn(0x556bb8368f60 msgr2=0x556bb83950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 shutdown_connections 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 1 -- 172.21.15.97:0/1563790749 wait complete. 2024-01-31T05:03:50.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb3236a740 20 client.6417 trim_cache size 0 max 16384 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "dc34a409-d027-49db-a591-dbd89a074581", 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6240, 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 11.563590985999999, 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 27, 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 55, 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 55, 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 16.977377240999999 2024-01-31T05:03:50.813 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:03:50.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbb0006af00 msgr2=0x7fbb0006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb2f400640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbb0006af00 0x7fbb0006d3c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbb180016b0 tx=0x7fbb18009040 comp rx=0 tx=0).stop 2024-01-31T05:03:50.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 msgr2=0x7fbb2809b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:50.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.809+0000 7fbb2f400640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 0x7fbb2809b400 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000c7f0 tx=0x7fbb1000ccb0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 shutdown_connections 2024-01-31T05:03:50.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb2814f2c0 0x7fbb2809b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fbb0006af00 0x7fbb0006d3c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb2815a350 0x7fbb2809b400 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 --2- 172.21.15.97:0/278667331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb280cde10 0x7fbb280af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 >> 172.21.15.97:0/278667331 conn(0x7fbb280b0a10 msgr2=0x7fbb280b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 shutdown_connections 2024-01-31T05:03:50.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:50.810+0000 7fbb2f400640 1 -- 172.21.15.97:0/278667331 wait complete. 2024-01-31T05:03:50.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:03:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.021+0000 7f2b7da00640 1 Processor -- start 2024-01-31T05:03:51.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.021+0000 7f2b7da00640 1 -- start start 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b7da00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b780cb810 0x7f2b780cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b7da00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b780ccd00 0x7f2b780cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b7da00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7806d1a0 con 0x7f2b7815b400 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b7da00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b780cc960 con 0x7f2b780cb810 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b7da00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7806c600 con 0x7f2b780ccd00 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b76a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b780cb810 0x7f2b780cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b76a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b780cb810 0x7f2b780cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42036/0 (socket says 172.21.15.97:42036) 2024-01-31T05:03:51.023 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.023+0000 7f2b76a00640 1 -- 172.21.15.97:0/3066365470 learned_addr learned my addr 172.21.15.97:0/3066365470 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.024+0000 7f2b76a00640 1 -- 172.21.15.97:0/3066365470 >> v2:172.21.15.188:3300/0 conn(0x7f2b780ccd00 msgr2=0x7f2b780cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.024+0000 7f2b77400640 1 --2- 172.21.15.97:0/3066365470 >> v2:172.21.15.188:3300/0 conn(0x7f2b780ccd00 0x7f2b780cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.024+0000 7f2b76a00640 1 --2- 172.21.15.97:0/3066365470 >> v2:172.21.15.188:3300/0 conn(0x7f2b780ccd00 0x7f2b780cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.024+0000 7f2b76a00640 1 -- 172.21.15.97:0/3066365470 >> v1:172.21.15.188:6789/0 conn(0x7f2b7815b400 legacy=0x7f2b7806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.024+0000 7f2b76a00640 1 -- 172.21.15.97:0/3066365470 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b780cd6b0 con 0x7f2b780cb810 2024-01-31T05:03:51.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.024+0000 7f2b77400640 1 --2- 172.21.15.97:0/3066365470 >> v2:172.21.15.188:3300/0 conn(0x7f2b780ccd00 0x7f2b780cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.025+0000 7f2b76a00640 1 --2- 172.21.15.97:0/3066365470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b780cb810 0x7f2b780cbbf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2b7815fe30 tx=0x7f2b6802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a7aad4e6e9a0e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.026+0000 7f2b75600640 1 -- 172.21.15.97:0/3066365470 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6802ca80 con 0x7f2b780cb810 2024-01-31T05:03:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.026+0000 7f2b75600640 1 -- 172.21.15.97:0/3066365470 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6802cbe0 con 0x7f2b780cb810 2024-01-31T05:03:51.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.026+0000 7f2b75600640 1 -- 172.21.15.97:0/3066365470 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b680343f0 con 0x7f2b780cb810 2024-01-31T05:03:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.026+0000 7f2b7da00640 1 -- 172.21.15.97:0/3066365470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b780cb810 msgr2=0x7f2b780cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.026+0000 7f2b7da00640 1 --2- 172.21.15.97:0/3066365470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b780cb810 0x7f2b780cbbf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2b7815fe30 tx=0x7f2b6802bee0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.027+0000 7f2b7da00640 1 -- 172.21.15.97:0/3066365470 shutdown_connections 2024-01-31T05:03:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.027+0000 7f2b7da00640 1 --2- 172.21.15.97:0/3066365470 >> v2:172.21.15.188:3300/0 conn(0x7f2b780ccd00 0x7f2b780cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.027+0000 7f2b7da00640 1 --2- 172.21.15.97:0/3066365470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b780cb810 0x7f2b780cbbf0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.026 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.027+0000 7f2b7da00640 1 -- 172.21.15.97:0/3066365470 >> 172.21.15.97:0/3066365470 conn(0x7f2b780b0a10 msgr2=0x7f2b780b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.027+0000 7f2b7da00640 1 -- 172.21.15.97:0/3066365470 shutdown_connections 2024-01-31T05:03:51.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.027+0000 7f2b7da00640 1 -- 172.21.15.97:0/3066365470 wait complete. 2024-01-31T05:03:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.028+0000 7f2b7da00640 1 Processor -- start 2024-01-31T05:03:51.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.029+0000 7f2b7da00640 1 -- start start 2024-01-31T05:03:51.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.029+0000 7f2b7da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b780cb810 0x7f2b781f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:51.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.029+0000 7f2b7da00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b780ccd00 0x7f2b780cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.029+0000 7f2b7da00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 0x7f2b780cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.029+0000 7f2b7da00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7815b810 con 0x7f2b7815b400 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.029+0000 7f2b7da00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b2be0 con 0x7f2b780ccd00 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b7da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b780b1fd0 con 0x7f2b780cb810 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 0x7f2b780cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 0x7f2b780cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42044/0 (socket says 172.21.15.97:42044) 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 -- 172.21.15.97:0/796762118 learned_addr learned my addr 172.21.15.97:0/796762118 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b76000640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b780ccd00 0x7f2b780cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b76a00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b780cb810 0x7f2b781f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 -- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b780cb810 msgr2=0x7f2b781f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b780cb810 0x7f2b781f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 -- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b780ccd00 msgr2=0x7f2b780cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b780ccd00 0x7f2b780cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.030+0000 7f2b77400640 1 -- 172.21.15.97:0/796762118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b68009b30 con 0x7f2b7815b400 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.031+0000 7f2b77400640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 0x7f2b780cf340 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000c7f0 tx=0x7f2b7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.031+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70007820 con 0x7f2b7815b400 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.031+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70007980 con 0x7f2b7815b400 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.031+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70007d70 con 0x7f2b7815b400 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.031+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b780b22b0 con 0x7f2b7815b400 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.031+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b780b2870 con 0x7f2b7815b400 2024-01-31T05:03:51.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.032+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7809bb60 con 0x7f2b7815b400 2024-01-31T05:03:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.033+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2b700113f0 con 0x7f2b7815b400 2024-01-31T05:03:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.034+0000 7f2b6ea00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5006aed0 0x7f2b5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:51.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.034+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b70089bf0 con 0x7f2b7815b400 2024-01-31T05:03:51.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.034+0000 7f2b76a00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5006aed0 0x7f2b5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:51.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.035+0000 7f2b76a00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5006aed0 0x7f2b5006d390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b68035000 tx=0x7f2b68030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:51.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.038+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2b700548e0 con 0x7f2b7815b400 2024-01-31T05:03:51.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.271+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b780b2d20 con 0x7f2b7815b400 2024-01-31T05:03:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.272+0000 7f2b6ea00640 1 -- 172.21.15.97:0/796762118 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2b7001f070 con 0x7f2b7815b400 2024-01-31T05:03:51.271 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-31T05:03:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5006aed0 msgr2=0x7f2b5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5006aed0 0x7f2b5006d390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b68035000 tx=0x7f2b68030000 comp rx=0 tx=0).stop 2024-01-31T05:03:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 msgr2=0x7f2b780cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 0x7f2b780cf340 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000c7f0 tx=0x7f2b7000ccb0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 shutdown_connections 2024-01-31T05:03:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2b5006aed0 0x7f2b5006d390 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b7815b400 0x7f2b780cf340 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b780ccd00 0x7f2b780cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 --2- 172.21.15.97:0/796762118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b780cb810 0x7f2b781f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 >> 172.21.15.97:0/796762118 conn(0x7f2b780b0a10 msgr2=0x7f2b78155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.274+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 shutdown_connections 2024-01-31T05:03:51.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:51.275+0000 7f2b7da00640 1 -- 172.21.15.97:0/796762118 wait complete. 2024-01-31T05:03:51.293 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:03:51.293 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:03:51.293 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:03:51.293 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:03:51.293 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:51.294 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:03:51.294 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:03:51.294 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:03:51.294 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:03:51.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:51.294 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:51.342 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42293sec preferred_lft 42293sec 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:03:51.343 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether ca:84:a9:c5:b2:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c884:a9ff:fec5:b219/64 scope link 2024-01-31T05:03:51.344 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:03:51.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:51.345 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:03:51.425 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:03:51.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:51.430 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:03:51.483 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:03:51.483 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:03:51.483 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:03:51.483 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:03:51.483 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:03:51.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:51.484 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:03:51.484 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:03:51.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:03:51.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:03:51.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:03:51.484 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:03:51.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:51.663 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:03:51.713 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:03:51.713 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:03:51.713 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:03:51.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:51.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:03:51.804 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:03:51.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:03:51.805 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:51.855 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:03:51.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:03:51.856 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:51.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:03:51.997 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:03:52.052 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:03:52.053 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:52.054 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:52.055 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:52.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:52.056 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:03:52.110 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:03:52.110 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:03:52.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:03:52.186 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:03:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:03:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:03:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:03:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:03:52.227 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:03:52.228 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:03:52.229 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:03:52.230 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:52.230 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:03:52.230 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:52.230 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:03:52.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:52.231 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:03:52.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:03:52.290+0000 7fce7a038480 -1 init, newargv = 0x55a8334d0ac0 newargc=15 2024-01-31T05:03:52.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[58403]: starting ceph client 2024-01-31T05:03:52.297 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[58403]: starting fuse 2024-01-31T05:03:52.314 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:03:52.314 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:03:52.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:03:52.315 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:03:52.316 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:03:52.505 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.58403.asok 2024-01-31T05:03:52.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:03:52.522 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58403.asok 2024-01-31T05:03:52.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:52.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58403.asok status 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "58403", 2024-01-31T05:03:52.752 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6426, 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6426 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2248050682 2024-01-31T05:03:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2248050682 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6426 192.168.0.1:0/2248050682", 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/2248050682", 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 27, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 55, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:03:52.754 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:03:52.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:52.767 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:03:52.767 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:52.818 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:03:52.819 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:52.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:52.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:52.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:52.913 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-01-31T05:03:52.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-01-31T05:03:53.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.181+0000 7f5fd9400640 1 Processor -- start 2024-01-31T05:03:53.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.181+0000 7f5fd9400640 1 -- start start 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.182+0000 7f5fd9400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fd415a350 0x7f5fd40abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.182+0000 7f5fd9400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415a730 0x7f5fd406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.182+0000 7f5fd9400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fd406dfc0 0x7f5fd40cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.182+0000 7f5fd9400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd40b2c90 con 0x7f5fd406dfc0 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.182+0000 7f5fd9400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd415abc0 con 0x7f5fd415a350 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.182+0000 7f5fd9400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd415aea0 con 0x7f5fd415a730 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fd406dfc0 0x7f5fd40cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fd406dfc0 0x7f5fd40cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42058/0 (socket says 172.21.15.97:42058) 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 -- 172.21.15.97:0/1514095692 learned_addr learned my addr 172.21.15.97:0/1514095692 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd2800640 1 --2- 172.21.15.97:0/1514095692 >> v2:172.21.15.188:3300/0 conn(0x7f5fd415a350 0x7f5fd40abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd1e00640 1 --2- 172.21.15.97:0/1514095692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415a730 0x7f5fd406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 -- 172.21.15.97:0/1514095692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415a730 msgr2=0x7f5fd406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 --2- 172.21.15.97:0/1514095692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415a730 0x7f5fd406da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 -- 172.21.15.97:0/1514095692 >> v2:172.21.15.188:3300/0 conn(0x7f5fd415a350 msgr2=0x7f5fd40abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 --2- 172.21.15.97:0/1514095692 >> v2:172.21.15.188:3300/0 conn(0x7f5fd415a350 0x7f5fd40abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.183+0000 7f5fd3200640 1 -- 172.21.15.97:0/1514095692 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd415b180 con 0x7f5fd406dfc0 2024-01-31T05:03:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.184+0000 7f5fd3200640 1 --2- 172.21.15.97:0/1514095692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cde10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4067370 tx=0x7f5fcc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ac7b1dfedc9683 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd1400640 1 -- 172.21.15.97:0/1514095692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc02ca70 con 0x7f5fd406dfc0 2024-01-31T05:03:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd1400640 1 -- 172.21.15.97:0/1514095692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc02cbd0 con 0x7f5fd406dfc0 2024-01-31T05:03:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd1400640 1 -- 172.21.15.97:0/1514095692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0343f0 con 0x7f5fd406dfc0 2024-01-31T05:03:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 -- 172.21.15.97:0/1514095692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd406dfc0 msgr2=0x7f5fd40cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 --2- 172.21.15.97:0/1514095692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cde10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4067370 tx=0x7f5fcc02bed0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 -- 172.21.15.97:0/1514095692 shutdown_connections 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 --2- 172.21.15.97:0/1514095692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cde10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 --2- 172.21.15.97:0/1514095692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd415a730 0x7f5fd406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 --2- 172.21.15.97:0/1514095692 >> v2:172.21.15.188:3300/0 conn(0x7f5fd415a350 0x7f5fd40abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.185+0000 7f5fd9400640 1 -- 172.21.15.97:0/1514095692 >> 172.21.15.97:0/1514095692 conn(0x7f5fd40b0a10 msgr2=0x7f5fd40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.186+0000 7f5fd9400640 1 -- 172.21.15.97:0/1514095692 shutdown_connections 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.186+0000 7f5fd9400640 1 -- 172.21.15.97:0/1514095692 wait complete. 2024-01-31T05:03:53.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.187+0000 7f5fd9400640 1 Processor -- start 2024-01-31T05:03:53.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.187+0000 7f5fd9400640 1 -- start start 2024-01-31T05:03:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd9400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 0x7f5fd4155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd9400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd41f2140 0x7f5fd41f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd9400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40b0810 con 0x7f5fd415a350 2024-01-31T05:03:53.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd9400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41f2af0 con 0x7f5fd406dfc0 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd9400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41f2d20 con 0x7f5fd41f2140 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.188+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 0x7f5fd4155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 0x7f5fd4155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42074/0 (socket says 172.21.15.97:42074) 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 -- 172.21.15.97:0/4028922534 learned_addr learned my addr 172.21.15.97:0/4028922534 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd3200640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd41f2140 0x7f5fd41f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd2800640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 -- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd41f2140 msgr2=0x7f5fd41f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd41f2140 0x7f5fd41f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 -- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd406dfc0 msgr2=0x7f5fd40cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 -- 172.21.15.97:0/4028922534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009b30 con 0x7f5fd415a350 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd2800640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:53.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd3200640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd41f2140 0x7f5fd41f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fd1e00640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 0x7f5fd4155200 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c7f0 tx=0x7f5fbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc004220 con 0x7f5fd415a350 2024-01-31T05:03:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.189+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc004380 con 0x7f5fd415a350 2024-01-31T05:03:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.190+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc007840 con 0x7f5fd415a350 2024-01-31T05:03:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.190+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd41f3000 con 0x7f5fd415a350 2024-01-31T05:03:53.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.190+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd409af10 con 0x7f5fd415a350 2024-01-31T05:03:53.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.191+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd4067830 con 0x7f5fd415a350 2024-01-31T05:03:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.192+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f5fbc007bb0 con 0x7f5fd415a350 2024-01-31T05:03:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.192+0000 7f5fcaa00640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5fac06aed0 0x7f5fac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.193+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5fbc089650 con 0x7f5fd415a350 2024-01-31T05:03:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.193+0000 7f5fd2800640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5fac06aed0 0x7f5fac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.193+0000 7f5fd2800640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5fac06aed0 0x7f5fac06d390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097a0 tx=0x7f5fc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:53.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.197+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5fbc0543e0 con 0x7f5fd415a350 2024-01-31T05:03:53.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.467+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7f5fd40ad500 con 0x7f5fd415a350 2024-01-31T05:03:53.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.471+0000 7f5fcaa00640 1 -- 172.21.15.97:0/4028922534 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 183+0+140 (secure 0 0 0) 0x7f5fbc087e60 con 0x7f5fd415a350 2024-01-31T05:03:53.471 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:03:53.471 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:03:53.471 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-01-31T05:03:53.471 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:03:53.471 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow r pool=cephfs_data" 2024-01-31T05:03:53.471 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:03:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.473+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5fac06aed0 msgr2=0x7f5fac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.473+0000 7f5fd9400640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5fac06aed0 0x7f5fac06d390 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097a0 tx=0x7f5fc4008040 comp rx=0 tx=0).stop 2024-01-31T05:03:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 msgr2=0x7f5fd4155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 0x7f5fd4155200 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c7f0 tx=0x7f5fbc00ccb0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 shutdown_connections 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd41f2140 0x7f5fd41f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f5fac06aed0 0x7f5fac06d390 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fd415a350 0x7f5fd4155200 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 --2- 172.21.15.97:0/4028922534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd406dfc0 0x7f5fd40cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 >> 172.21.15.97:0/4028922534 conn(0x7f5fd40b0a10 msgr2=0x7f5fd40acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 shutdown_connections 2024-01-31T05:03:53.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.474+0000 7f5fd9400640 1 -- 172.21.15.97:0/4028922534 wait complete. 2024-01-31T05:03:53.492 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:53.492 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:53.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.794+0000 7f7f4d800640 1 Processor -- start 2024-01-31T05:03:53.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.795+0000 7f7f4d800640 1 -- start start 2024-01-31T05:03:53.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.796+0000 7f7f4d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 0x7f7f480b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.796+0000 7f7f4d800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4815d8c0 con 0x7f7f480d0d60 2024-01-31T05:03:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.796+0000 7f7f4d800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480b1920 con 0x7f7f4815d4e0 2024-01-31T05:03:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.796+0000 7f7f4d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480b0b20 con 0x7f7f480d1c70 2024-01-31T05:03:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.796+0000 7f7f4ce00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7f480d0d60 0x7f7f480b5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54516/0 (socket says 172.21.15.97:54516) 2024-01-31T05:03:53.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.796+0000 7f7f4ce00640 1 -- 172.21.15.97:0/1137326206 learned_addr learned my addr 172.21.15.97:0/1137326206 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f47400640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 0x7f7f480b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792872782 0 0) 0x7f7f40001490 con 0x7f7f480d0d60 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f28003520 con 0x7f7f480d0d60 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334449900 0 0) 0x7f7f480b1920 con 0x7f7f4815d4e0 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f40001490 con 0x7f7f4815d4e0 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2317582623 0 0) 0x7f7f28003520 con 0x7f7f480d0d60 2024-01-31T05:03:53.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 msgr2=0x7f7f480b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 0x7f7f480b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 >> v1:172.21.15.188:6789/0 conn(0x7f7f4815d4e0 legacy=0x7f7f480d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f480cef50 con 0x7f7f480d0d60 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f7f40001b80 con 0x7f7f480d0d60 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 >> v1:172.21.15.97:6789/0 conn(0x7f7f480d0d60 legacy=0x7f7f480b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f28003b30 0x7f7f28003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 0x7f7f280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f28004d00 0x7f7f28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2800ba30 con 0x7f7f28004d00 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2800bb70 con 0x7f7f28003b30 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.797+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2800bce0 con 0x7f7f480d1c70 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f4ce00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f28004d00 0x7f7f28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f47e00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f28003b30 0x7f7f28003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f4ce00640 1 -- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 msgr2=0x7f7f280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f4ce00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f480d1c70 0x7f7f280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f4ce00640 1 -- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f28003b30 msgr2=0x7f7f28003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f4ce00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f28003b30 0x7f7f28003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.798+0000 7f7f4ce00640 1 -- 172.21.15.97:0/1137326206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28003520 con 0x7f7f28004d00 2024-01-31T05:03:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.799+0000 7f7f4ce00640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f28004d00 0x7f7f28005180 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000a7b0 tx=0x7f7f280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921fcb0a821971f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.799+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40034630 con 0x7f7f28004d00 2024-01-31T05:03:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.799+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4002f070 con 0x7f7f28004d00 2024-01-31T05:03:53.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.799+0000 7f7f46a00640 1 -- 172.21.15.97:0/1137326206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f400349f0 con 0x7f7f28004d00 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4d800640 1 -- 172.21.15.97:0/1137326206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f28004d00 msgr2=0x7f7f28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1137326206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f28004d00 0x7f7f28005180 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000a7b0 tx=0x7f7f280057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4ce00640 1 -- 172.21.15.97:0/1137326206 reap_dead start 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4d800640 1 -- 172.21.15.97:0/1137326206 shutdown_connections 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4d800640 1 -- 172.21.15.97:0/1137326206 >> 172.21.15.97:0/1137326206 conn(0x7f7f480d3010 msgr2=0x7f7f480d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4d800640 1 -- 172.21.15.97:0/1137326206 shutdown_connections 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.800+0000 7f7f4d800640 1 -- 172.21.15.97:0/1137326206 wait complete. 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.802+0000 7f7f4d800640 1 Processor -- start 2024-01-31T05:03:53.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.802+0000 7f7f4d800640 1 -- start start 2024-01-31T05:03:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f28004d00 0x7f7f480cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 0x7f7f48157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480b2910 0x7f7f480ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4d800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480ccb20 con 0x7f7f48160570 2024-01-31T05:03:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4815dbe0 con 0x7f7f480b2910 2024-01-31T05:03:53.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4815dec0 con 0x7f7f28004d00 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 0x7f7f48157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f47400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 0x7f7f48157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42084/0 (socket says 172.21.15.97:42084) 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f47400640 1 -- 172.21.15.97:0/1604192248 learned_addr learned my addr 172.21.15.97:0/1604192248 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f47e00640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f28004d00 0x7f7f480cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f4ce00640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480b2910 0x7f7f480ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f47400640 1 -- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f28004d00 msgr2=0x7f7f480cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.803+0000 7f7f47400640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f28004d00 0x7f7f480cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f47400640 1 -- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480b2910 msgr2=0x7f7f480ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f47400640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480b2910 0x7f7f480ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f47400640 1 -- 172.21.15.97:0/1604192248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4815e1a0 con 0x7f7f48160570 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f4ce00640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480b2910 0x7f7f480ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f47400640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 0x7f7f48157780 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000e7f0 tx=0x7f7f3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000eea0 con 0x7f7f48160570 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000c940 con 0x7f7f48160570 2024-01-31T05:03:53.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4815e480 con 0x7f7f48160570 2024-01-31T05:03:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000ccd0 con 0x7f7f48160570 2024-01-31T05:03:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.804+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f48157cc0 con 0x7f7f48160570 2024-01-31T05:03:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.805+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4815dd20 con 0x7f7f48160570 2024-01-31T05:03:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.806+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7f30013470 con 0x7f7f48160570 2024-01-31T05:03:53.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.807+0000 7f7f3fe00640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7f2006af00 0x7f7f2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.808+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7f30089800 con 0x7f7f48160570 2024-01-31T05:03:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.808+0000 7f7f47e00640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7f2006af00 0x7f7f2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:53.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.808+0000 7f7f47e00640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7f2006af00 0x7f7f2006d3c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f38007ae0 tx=0x7f7f38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:53.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:53.812+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7f4815dd20 con 0x7f7f48160570 2024-01-31T05:03:54.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.010+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f4815dd20 con 0x7f7f48160570 2024-01-31T05:03:54.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.013+0000 7f7f3fe00640 1 -- 172.21.15.97:0/1604192248 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f7f4815dd20 con 0x7f7f48160570 2024-01-31T05:03:54.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:54.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.015+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7f2006af00 msgr2=0x7f7f2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7f2006af00 0x7f7f2006d3c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7f38007ae0 tx=0x7f7f38009040 comp rx=0 tx=0).stop 2024-01-31T05:03:54.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 msgr2=0x7f7f48157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 0x7f7f48157780 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000e7f0 tx=0x7f7f3000ecb0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 shutdown_connections 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480b2910 0x7f7f480ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7f2006af00 0x7f7f2006d3c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f48160570 0x7f7f48157780 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 --2- 172.21.15.97:0/1604192248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f28004d00 0x7f7f480cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 >> 172.21.15.97:0/1604192248 conn(0x7f7f480d3010 msgr2=0x7f7f480b6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 shutdown_connections 2024-01-31T05:03:54.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.016+0000 7f7f4d800640 1 -- 172.21.15.97:0/1604192248 wait complete. 2024-01-31T05:03:54.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:03:36.506793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:03:37.685194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:54.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.240+0000 7fc8b9c00640 1 Processor -- start 2024-01-31T05:03:54.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.240+0000 7fc8b9c00640 1 -- start start 2024-01-31T05:03:54.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.242+0000 7fc8b9c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8b406f150 0x7fc8b40af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.242+0000 7fc8b9c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b40b2ea0 con 0x7fc8b4160770 2024-01-31T05:03:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.242+0000 7fc8b9c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b4160b50 con 0x7fc8b40b48d0 2024-01-31T05:03:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.242+0000 7fc8b9c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b40b3420 con 0x7fc8b406f150 2024-01-31T05:03:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.242+0000 7fc8b3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc8b4160770 0x7fc8b406d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:54524/0 (socket says 172.21.15.97:54524) 2024-01-31T05:03:54.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.242+0000 7fc8b3e00640 1 -- 172.21.15.97:0/1933064621 learned_addr learned my addr 172.21.15.97:0/1933064621 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078579343 0 0) 0x7fc8b40b2ea0 con 0x7fc8b4160770 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b3400640 1 --2- 172.21.15.97:0/1933064621 >> v2:172.21.15.188:3300/0 conn(0x7fc8b406f150 0x7fc8b40af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc894003520 con 0x7fc8b4160770 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2336977326 0 0) 0x7fc894003520 con 0x7fc8b4160770 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 >> v2:172.21.15.188:3300/0 conn(0x7fc8b406f150 msgr2=0x7fc8b40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 --2- 172.21.15.97:0/1933064621 >> v2:172.21.15.188:3300/0 conn(0x7fc8b406f150 0x7fc8b40af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 >> v1:172.21.15.188:6789/0 conn(0x7fc8b40b48d0 legacy=0x7fc8b4160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4072120 con 0x7fc8b4160770 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fc8a4001670 con 0x7fc8b4160770 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 >> v1:172.21.15.97:6789/0 conn(0x7fc8b4160770 legacy=0x7fc8b406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8940039e0 0x7fc894003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc894004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc894005180 0x7fc89400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40b3420 con 0x7fc8940047c0 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89400c0b0 con 0x7fc894005180 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.243+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89400c270 con 0x7fc8940039e0 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b3400640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc894004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b9200640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc894005180 0x7fc89400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b3400640 1 -- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8940039e0 msgr2=0x7fc894003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b3400640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8940039e0 0x7fc894003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b3400640 1 -- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc894005180 msgr2=0x7fc89400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:54.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b3400640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc894005180 0x7fc89400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b3400640 1 -- 172.21.15.97:0/1933064621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894003520 con 0x7fc8940047c0 2024-01-31T05:03:54.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.244+0000 7fc8b9200640 1 -- 172.21.15.97:0/1933064621 reap_dead start 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.245+0000 7fc8b3400640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc894004c40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ac0 tx=0x7fc8940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0e92bb2f4e2746 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c02c6b0 con 0x7fc8940047c0 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c02c810 con 0x7fc8940047c0 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b2a00640 1 -- 172.21.15.97:0/1933064621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c02cba0 con 0x7fc8940047c0 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 -- 172.21.15.97:0/1933064621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 msgr2=0x7fc894004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc894004c40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ac0 tx=0x7fc8940057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 -- 172.21.15.97:0/1933064621 shutdown_connections 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/1933064621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc894004c40 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 -- 172.21.15.97:0/1933064621 >> 172.21.15.97:0/1933064621 conn(0x7fc8b40b1b00 msgr2=0x7fc8b40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 -- 172.21.15.97:0/1933064621 shutdown_connections 2024-01-31T05:03:54.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.246+0000 7fc8b9c00640 1 -- 172.21.15.97:0/1933064621 wait complete. 2024-01-31T05:03:54.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.248+0000 7fc8b9c00640 1 Processor -- start 2024-01-31T05:03:54.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.248+0000 7fc8b9c00640 1 -- start start 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b9c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc8b40c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b40c52b0 0x7fc8b415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8b415b2c0 0x7fc8b415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b9c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4157670 con 0x7fc8940047c0 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40c5810 con 0x7fc8b40c52b0 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40c5af0 con 0x7fc8b415b2c0 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc8b40c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.249+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc8b40c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42096/0 (socket says 172.21.15.97:42096) 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 -- 172.21.15.97:0/517313483 learned_addr learned my addr 172.21.15.97:0/517313483 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:54.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3400640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b40c52b0 0x7fc8b415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b9200640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8b415b2c0 0x7fc8b415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 -- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8b415b2c0 msgr2=0x7fc8b415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8b415b2c0 0x7fc8b415b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 -- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b40c52b0 msgr2=0x7fc8b415ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b40c52b0 0x7fc8b415ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 -- 172.21.15.97:0/517313483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40c5dd0 con 0x7fc8940047c0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3400640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b40c52b0 0x7fc8b415ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8b3e00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc8b40c4d70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c730 tx=0x7fc8a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40072d0 con 0x7fc8940047c0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.250+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a4007430 con 0x7fc8940047c0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.251+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004a20 con 0x7fc8940047c0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.251+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4157300 con 0x7fc8940047c0 2024-01-31T05:03:54.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.251+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b4155ba0 con 0x7fc8940047c0 2024-01-31T05:03:54.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.252+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b4067dc0 con 0x7fc8940047c0 2024-01-31T05:03:54.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.253+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc8a400a5e0 con 0x7fc8940047c0 2024-01-31T05:03:54.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.254+0000 7fc8abe00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88c06aed0 0x7fc88c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.254+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc8a40891c0 con 0x7fc8940047c0 2024-01-31T05:03:54.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.254+0000 7fc8b3400640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88c06aed0 0x7fc88c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.255+0000 7fc8b3400640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88c06aed0 0x7fc88c06d390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc89c036000 tx=0x7fc89c030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:54.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.259+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc8a4053e20 con 0x7fc8940047c0 2024-01-31T05:03:54.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.451+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8b40736b0 con 0x7fc8940047c0 2024-01-31T05:03:54.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.452+0000 7fc8abe00640 1 -- 172.21.15.97:0/517313483 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8a4057440 con 0x7fc8940047c0 2024-01-31T05:03:54.451 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:54.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.454+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88c06aed0 msgr2=0x7fc88c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.454+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88c06aed0 0x7fc88c06d390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc89c036000 tx=0x7fc89c030000 comp rx=0 tx=0).stop 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.454+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 msgr2=0x7fc8b40c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.454+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc8b40c4d70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400c730 tx=0x7fc8a400cbf0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 shutdown_connections 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8b415b2c0 0x7fc8b415b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc8b40c52b0 0x7fc8b415ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc88c06aed0 0x7fc88c06d390 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 --2- 172.21.15.97:0/517313483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8940047c0 0x7fc8b40c4d70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 >> 172.21.15.97:0/517313483 conn(0x7fc8b40b1b00 msgr2=0x7fc8b406db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 shutdown_connections 2024-01-31T05:03:54.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.455+0000 7fc8b9c00640 1 -- 172.21.15.97:0/517313483 wait complete. 2024-01-31T05:03:54.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:03:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.686+0000 7fe631c00640 1 Processor -- start 2024-01-31T05:03:54.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.686+0000 7fe631c00640 1 -- start start 2024-01-31T05:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe631c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe62c0ce1e0 0x7fe62c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe631c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe62c0cb7d0 0x7fe62c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe631c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62c161fc0 con 0x7fe62c161b10 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe631c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62c162c40 con 0x7fe62c0ce1e0 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe631c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62c1622a0 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe62aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe62c0ce1e0 0x7fe62c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe62aa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe62c0ce1e0 0x7fe62c0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42104/0 (socket says 172.21.15.97:42104) 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.688+0000 7fe62aa00640 1 -- 172.21.15.97:0/3862903616 learned_addr learned my addr 172.21.15.97:0/3862903616 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.689+0000 7fe62b400640 1 --2- 172.21.15.97:0/3862903616 >> v2:172.21.15.188:3300/0 conn(0x7fe62c0cb7d0 0x7fe62c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.689+0000 7fe62aa00640 1 -- 172.21.15.97:0/3862903616 >> v2:172.21.15.188:3300/0 conn(0x7fe62c0cb7d0 msgr2=0x7fe62c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.689+0000 7fe62aa00640 1 --2- 172.21.15.97:0/3862903616 >> v2:172.21.15.188:3300/0 conn(0x7fe62c0cb7d0 0x7fe62c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.689+0000 7fe62aa00640 1 -- 172.21.15.97:0/3862903616 >> v1:172.21.15.188:6789/0 conn(0x7fe62c161b10 legacy=0x7fe62c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.689+0000 7fe62aa00640 1 -- 172.21.15.97:0/3862903616 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c162580 con 0x7fe62c0ce1e0 2024-01-31T05:03:54.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.690+0000 7fe62aa00640 1 --2- 172.21.15.97:0/3862903616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c0b2fc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0ade50 tx=0x7fe61c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ce997cfe470ee4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:54.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.691+0000 7fe62a000640 1 -- 172.21.15.97:0/3862903616 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004230 con 0x7fe62c0ce1e0 2024-01-31T05:03:54.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.691+0000 7fe62a000640 1 -- 172.21.15.97:0/3862903616 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c004390 con 0x7fe62c0ce1e0 2024-01-31T05:03:54.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.691+0000 7fe62a000640 1 -- 172.21.15.97:0/3862903616 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c003810 con 0x7fe62c0ce1e0 2024-01-31T05:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.691+0000 7fe631c00640 1 -- 172.21.15.97:0/3862903616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0ce1e0 msgr2=0x7fe62c0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.691+0000 7fe631c00640 1 --2- 172.21.15.97:0/3862903616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c0b2fc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0ade50 tx=0x7fe61c02b860 comp rx=0 tx=0).stop 2024-01-31T05:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.692+0000 7fe631c00640 1 -- 172.21.15.97:0/3862903616 shutdown_connections 2024-01-31T05:03:54.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.692+0000 7fe631c00640 1 --2- 172.21.15.97:0/3862903616 >> v2:172.21.15.188:3300/0 conn(0x7fe62c0cb7d0 0x7fe62c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.692+0000 7fe631c00640 1 --2- 172.21.15.97:0/3862903616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c0b2fc0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.692+0000 7fe631c00640 1 -- 172.21.15.97:0/3862903616 >> 172.21.15.97:0/3862903616 conn(0x7fe62c0b0a10 msgr2=0x7fe62c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.692+0000 7fe631c00640 1 -- 172.21.15.97:0/3862903616 shutdown_connections 2024-01-31T05:03:54.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.692+0000 7fe631c00640 1 -- 172.21.15.97:0/3862903616 wait complete. 2024-01-31T05:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.694+0000 7fe631c00640 1 Processor -- start 2024-01-31T05:03:54.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.694+0000 7fe631c00640 1 -- start start 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.695+0000 7fe631c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 0x7fe62c164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.695+0000 7fe631c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c1ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.695+0000 7fe631c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62c161b10 0x7fe62c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.695+0000 7fe631c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1ed0a0 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.695+0000 7fe631c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1ec490 con 0x7fe62c0ce1e0 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.695+0000 7fe631c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1ec770 con 0x7fe62c161b10 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 0x7fe62c164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 0x7fe62c164370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42114/0 (socket says 172.21.15.97:42114) 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 -- 172.21.15.97:0/138255789 learned_addr learned my addr 172.21.15.97:0/138255789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe623e00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c1ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 -- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62c161b10 msgr2=0x7fe62c157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62b400640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62c161b10 0x7fe62c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62c161b10 0x7fe62c157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 -- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62c0ce1e0 msgr2=0x7fe62c1ebc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c1ebc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe62aa00640 1 -- 172.21.15.97:0/138255789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009460 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.696+0000 7fe623e00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c1ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:54.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.697+0000 7fe62aa00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 0x7fe62c164370 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe61c035040 tx=0x7fe61c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.697+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c004500 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.697+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c003c50 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.697+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c1eca50 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.697+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c006040 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.697+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c1ec180 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.698+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c1ed210 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.700+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe61c0061a0 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.701+0000 7fe622a00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe60406aed0 0x7fe60406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:54.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.701+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe61c0ab9e0 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.701+0000 7fe623e00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe60406aed0 0x7fe60406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:54.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.702+0000 7fe623e00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe60406aed0 0x7fe60406d390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe62c06c780 tx=0x7fe614009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:54.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.706+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe61c076750 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.892+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe62c0cbc20 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.893+0000 7fe622a00640 1 -- 172.21.15.97:0/138255789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe61c079d70 con 0x7fe62c0cb7d0 2024-01-31T05:03:54.893 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe60406aed0 msgr2=0x7fe60406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe60406aed0 0x7fe60406d390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe62c06c780 tx=0x7fe614009040 comp rx=0 tx=0).stop 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 msgr2=0x7fe62c164370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 0x7fe62c164370 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe61c035040 tx=0x7fe61c0046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 shutdown_connections 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe62c161b10 0x7fe62c157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe62c0ce1e0 0x7fe62c1ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe60406aed0 0x7fe60406d390 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 --2- 172.21.15.97:0/138255789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe62c0cb7d0 0x7fe62c164370 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 >> 172.21.15.97:0/138255789 conn(0x7fe62c0b0a10 msgr2=0x7fe62c06c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 shutdown_connections 2024-01-31T05:03:54.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:54.896+0000 7fe631c00640 1 -- 172.21.15.97:0/138255789 wait complete. 2024-01-31T05:03:54.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.114+0000 7fc329200640 1 Processor -- start 2024-01-31T05:03:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.115+0000 7fc329200640 1 -- start start 2024-01-31T05:03:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.116+0000 7fc329200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 0x7fc3240cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.116+0000 7fc329200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32415ec50 con 0x7fc3240d0de0 2024-01-31T05:03:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.116+0000 7fc329200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3240d0460 con 0x7fc32415d590 2024-01-31T05:03:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.116+0000 7fc329200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240cded0 con 0x7fc3240cfae0 2024-01-31T05:03:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc322800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc3240d0de0 0x7fc3240d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49556/0 (socket says 172.21.15.97:49556) 2024-01-31T05:03:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc322800640 1 -- 172.21.15.97:0/983191843 learned_addr learned my addr 172.21.15.97:0/983191843 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc321400640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 0x7fc3240cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542284109 0 0) 0x7fc32415ec50 con 0x7fc3240d0de0 2024-01-31T05:03:55.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc304003520 con 0x7fc3240d0de0 2024-01-31T05:03:55.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2122032827 0 0) 0x7fc304003520 con 0x7fc3240d0de0 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 msgr2=0x7fc3240cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 0x7fc3240cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 >> v1:172.21.15.188:6789/0 conn(0x7fc32415d590 legacy=0x7fc32415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32415d970 con 0x7fc3240d0de0 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fc31c0019f0 con 0x7fc3240d0de0 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 >> v1:172.21.15.97:6789/0 conn(0x7fc3240d0de0 legacy=0x7fc3240d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.117+0000 7fc320a00640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 0x7fc3040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc320a00640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc304003f20 0x7fc3040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc320a00640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc304004d00 0x7fc304005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3240cded0 con 0x7fc304003f20 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30400baf0 con 0x7fc304004d00 2024-01-31T05:03:55.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30400bcb0 con 0x7fc3240cfae0 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc321400640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc304003f20 0x7fc3040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc322800640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc304004d00 0x7fc304005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc321400640 1 -- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 msgr2=0x7fc3040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc321400640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3240cfae0 0x7fc3040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc321400640 1 -- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc304004d00 msgr2=0x7fc304005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc321400640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc304004d00 0x7fc304005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc321400640 1 -- 172.21.15.97:0/983191843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304003520 con 0x7fc304003f20 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.118+0000 7fc322800640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc304004d00 0x7fc304005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.119+0000 7fc321400640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc304003f20 0x7fc3040047c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ac0 tx=0x7fc3040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4121905d020d93 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.119+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c02c710 con 0x7fc304003f20 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.119+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c02c870 con 0x7fc304003f20 2024-01-31T05:03:55.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.119+0000 7fc320a00640 1 -- 172.21.15.97:0/983191843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c02cc00 con 0x7fc304003f20 2024-01-31T05:03:55.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc329200640 1 -- 172.21.15.97:0/983191843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc304003f20 msgr2=0x7fc3040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc329200640 1 --2- 172.21.15.97:0/983191843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc304003f20 0x7fc3040047c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009ac0 tx=0x7fc3040057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc322800640 1 -- 172.21.15.97:0/983191843 reap_dead start 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc329200640 1 -- 172.21.15.97:0/983191843 shutdown_connections 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc329200640 1 -- 172.21.15.97:0/983191843 >> 172.21.15.97:0/983191843 conn(0x7fc32409c560 msgr2=0x7fc3240d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc329200640 1 -- 172.21.15.97:0/983191843 shutdown_connections 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.120+0000 7fc329200640 1 -- 172.21.15.97:0/983191843 wait complete. 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.122+0000 7fc329200640 1 Processor -- start 2024-01-31T05:03:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.122+0000 7fc329200640 1 -- start start 2024-01-31T05:03:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc329200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc304003f20 0x7fc32409df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc329200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 0x7fc32415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc329200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3240a0770 0x7fc3240a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc329200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32409db50 con 0x7fc3240c9e10 2024-01-31T05:03:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc329200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32409e4b0 con 0x7fc3240a0770 2024-01-31T05:03:55.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc329200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32409ced0 con 0x7fc304003f20 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 0x7fc32415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 0x7fc32415b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34214/0 (socket says 172.21.15.97:34214) 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 -- 172.21.15.97:0/1714406818 learned_addr learned my addr 172.21.15.97:0/1714406818 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc322800640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3240a0770 0x7fc3240a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321e00640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc304003f20 0x7fc32409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 -- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc304003f20 msgr2=0x7fc32409df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc304003f20 0x7fc32409df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 -- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3240a0770 msgr2=0x7fc3240a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3240a0770 0x7fc3240a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321400640 1 -- 172.21.15.97:0/1714406818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32409d1b0 con 0x7fc3240c9e10 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc321e00640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc304003f20 0x7fc32409df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.123+0000 7fc322800640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3240a0770 0x7fc3240a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.124+0000 7fc321400640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 0x7fc32415b2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc30c037040 tx=0x7fc30c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:55.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.124+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c003810 con 0x7fc3240c9e10 2024-01-31T05:03:55.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.124+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c02de90 con 0x7fc3240c9e10 2024-01-31T05:03:55.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.124+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c003b90 con 0x7fc3240c9e10 2024-01-31T05:03:55.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.124+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32409d490 con 0x7fc3240c9e10 2024-01-31T05:03:55.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.124+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32415b800 con 0x7fc3240c9e10 2024-01-31T05:03:55.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.125+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc324067830 con 0x7fc3240c9e10 2024-01-31T05:03:55.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.127+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc30c003cf0 con 0x7fc3240c9e10 2024-01-31T05:03:55.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.127+0000 7fc31a000640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc2fc06aed0 0x7fc2fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.128+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc30c0ab7c0 con 0x7fc3240c9e10 2024-01-31T05:03:55.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.128+0000 7fc321e00640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc2fc06aed0 0x7fc2fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.128+0000 7fc321e00640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc2fc06aed0 0x7fc2fc06d390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc314001710 tx=0x7fc314009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:55.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.132+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc30c076680 con 0x7fc3240c9e10 2024-01-31T05:03:55.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.327+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc32415dc00 con 0x7fc3240c9e10 2024-01-31T05:03:55.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.327+0000 7fc31a000640 1 -- 172.21.15.97:0/1714406818 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc30c079ca0 con 0x7fc3240c9e10 2024-01-31T05:03:55.327 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:55.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc2fc06aed0 msgr2=0x7fc2fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc2fc06aed0 0x7fc2fc06d390 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc314001710 tx=0x7fc314009040 comp rx=0 tx=0).stop 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 msgr2=0x7fc32415b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 0x7fc32415b2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc30c037040 tx=0x7fc30c004730 comp rx=0 tx=0).stop 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 shutdown_connections 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3240a0770 0x7fc3240a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc2fc06aed0 0x7fc2fc06d390 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3240c9e10 0x7fc32415b2c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 --2- 172.21.15.97:0/1714406818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc304003f20 0x7fc32409df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 >> 172.21.15.97:0/1714406818 conn(0x7fc32409c560 msgr2=0x7fc3240d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 shutdown_connections 2024-01-31T05:03:55.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.330+0000 7fc329200640 1 -- 172.21.15.97:0/1714406818 wait complete. 2024-01-31T05:03:55.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:03:55.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.548+0000 7faeba800640 1 Processor -- start 2024-01-31T05:03:55.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.548+0000 7faeba800640 1 -- start start 2024-01-31T05:03:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeba800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faeb40ce680 0x7faeb40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeba800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 0x7faeb40b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeba800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb40b3f60 con 0x7faeb415b080 2024-01-31T05:03:55.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeba800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb40b41c0 con 0x7faeb40ce680 2024-01-31T05:03:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeba800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb415c540 con 0x7faeb40ae360 2024-01-31T05:03:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeb9400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faeb415b080 0x7faeb40cdf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49568/0 (socket says 172.21.15.97:49568) 2024-01-31T05:03:55.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeb9400640 1 -- 172.21.15.97:0/3280356831 learned_addr learned my addr 172.21.15.97:0/3280356831 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3280356831 >> v2:172.21.15.188:3300/0 conn(0x7faeb40ce680 0x7faeb40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeb9e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 0x7faeb40b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.550+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091254918 0 0) 0x7faeb40b3f60 con 0x7faeb415b080 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae94003520 con 0x7faeb415b080 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 948726415 0 0) 0x7fae94003520 con 0x7faeb415b080 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 msgr2=0x7faeb40b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 0x7faeb40b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 >> v2:172.21.15.188:3300/0 conn(0x7faeb40ce680 msgr2=0x7faeb40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3280356831 >> v2:172.21.15.188:3300/0 conn(0x7faeb40ce680 0x7faeb40b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb415b780 con 0x7faeb415b080 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7faea4001670 con 0x7faeb415b080 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 >> v1:172.21.15.97:6789/0 conn(0x7faeb415b080 legacy=0x7faeb40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 0x7fae940039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7fae940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94004d00 0x7fae94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40b41c0 con 0x7fae94003f60 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb415c540 con 0x7fae94004d00 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9400b9b0 con 0x7faeb40ae360 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.551+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7fae940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb9e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94004d00 0x7fae94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb8a00640 1 -- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 msgr2=0x7fae940039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40ae360 0x7fae940039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb8a00640 1 -- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94004d00 msgr2=0x7fae94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94004d00 0x7fae94005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb8a00640 1 -- 172.21.15.97:0/3280356831 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94003520 con 0x7fae94003f60 2024-01-31T05:03:55.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.552+0000 7faeb9e00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae94004d00 0x7fae94005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7fae940047c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009ac0 tx=0x7fae940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fee536fcf98296f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c02c710 con 0x7fae94003f60 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c02c870 con 0x7fae94003f60 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeb3e00640 1 -- 172.21.15.97:0/3280356831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c02cc00 con 0x7fae94003f60 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeba800640 1 -- 172.21.15.97:0/3280356831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 msgr2=0x7fae940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeba800640 1 --2- 172.21.15.97:0/3280356831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7fae940047c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009ac0 tx=0x7fae940057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.553+0000 7faeb9e00640 1 -- 172.21.15.97:0/3280356831 reap_dead start 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.554+0000 7faeba800640 1 -- 172.21.15.97:0/3280356831 shutdown_connections 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.554+0000 7faeba800640 1 -- 172.21.15.97:0/3280356831 >> 172.21.15.97:0/3280356831 conn(0x7faeb406ad40 msgr2=0x7faeb40b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.554+0000 7faeba800640 1 -- 172.21.15.97:0/3280356831 shutdown_connections 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.554+0000 7faeba800640 1 -- 172.21.15.97:0/3280356831 wait complete. 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.555+0000 7faeba800640 1 Processor -- start 2024-01-31T05:03:55.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.556+0000 7faeba800640 1 -- start start 2024-01-31T05:03:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.556+0000 7faeba800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7faeb414a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.556+0000 7faeba800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb4159300 0x7faeb4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.556+0000 7faeba800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2120 0x7faeb40b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.556+0000 7faeba800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40b2ae0 con 0x7fae94003f60 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.556+0000 7faeba800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4159850 con 0x7faeb4159300 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeba800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4159b30 con 0x7faeb40b2120 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7faeb414a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7faeb414a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34224/0 (socket says 172.21.15.97:34224) 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 -- 172.21.15.97:0/3583723362 learned_addr learned my addr 172.21.15.97:0/3583723362 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 -- 172.21.15.97:0/3583723362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2120 msgr2=0x7faeb40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2120 0x7faeb40b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 -- 172.21.15.97:0/3583723362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb4159300 msgr2=0x7faeb4155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb4159300 0x7faeb4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 -- 172.21.15.97:0/3583723362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4159e10 con 0x7fae94003f60 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb9400640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7faeb414a4e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faea400c700 tx=0x7faea400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.557+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40074e0 con 0x7fae94003f60 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.558+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4007660 con 0x7fae94003f60 2024-01-31T05:03:55.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.558+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4004a20 con 0x7fae94003f60 2024-01-31T05:03:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.558+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb415a0f0 con 0x7fae94003f60 2024-01-31T05:03:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.558+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb414aa20 con 0x7fae94003f60 2024-01-31T05:03:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.559+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb40b37e0 con 0x7fae94003f60 2024-01-31T05:03:55.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.560+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7faea4004b80 con 0x7fae94003f60 2024-01-31T05:03:55.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.561+0000 7faeb1600640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae8c06af00 0x7fae8c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:55.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.561+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faea4088e90 con 0x7fae94003f60 2024-01-31T05:03:55.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.561+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae8c06af00 0x7fae8c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:55.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.562+0000 7faeb8a00640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae8c06af00 0x7fae8c06d3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae9c036000 tx=0x7fae9c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:55.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.566+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7faea4053b80 con 0x7fae94003f60 2024-01-31T05:03:55.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.773+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faeb40676b0 con 0x7fae94003f60 2024-01-31T05:03:55.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.773+0000 7faeb1600640 1 -- 172.21.15.97:0/3583723362 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7faea4018020 con 0x7fae94003f60 2024-01-31T05:03:55.773 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae8c06af00 msgr2=0x7fae8c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae8c06af00 0x7fae8c06d3c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae9c036000 tx=0x7fae9c02e000 comp rx=0 tx=0).stop 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 msgr2=0x7faeb414a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7faeb414a4e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faea400c700 tx=0x7faea400cbc0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 shutdown_connections 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeb40b2120 0x7faeb40b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeb4159300 0x7faeb4155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae8c06af00 0x7fae8c06d3c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 --2- 172.21.15.97:0/3583723362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae94003f60 0x7faeb414a4e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 >> 172.21.15.97:0/3583723362 conn(0x7faeb406ad40 msgr2=0x7faeb40b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:55.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 shutdown_connections 2024-01-31T05:03:55.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:55.776+0000 7faeba800640 1 -- 172.21.15.97:0/3583723362 wait complete. 2024-01-31T05:03:55.795 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:03:55.795 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:03:55.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:03:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.057+0000 7f9bc2e00640 1 Processor -- start 2024-01-31T05:03:56.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.057+0000 7f9bc2e00640 1 -- start start 2024-01-31T05:03:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.059+0000 7f9bc2e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9bbc0b2e10 0x7f9bbc06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.059+0000 7f9bc2e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc06cba0 con 0x7f9bbc06c350 2024-01-31T05:03:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.059+0000 7f9bc2e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc06ce80 con 0x7f9bbc0cb7d0 2024-01-31T05:03:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.059+0000 7f9bc2e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc0b32a0 con 0x7f9bbc0b2e10 2024-01-31T05:03:56.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.060+0000 7f9bc0e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9bbc06c350 0x7f9bbc0ccd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49582/0 (socket says 172.21.15.97:49582) 2024-01-31T05:03:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.060+0000 7f9bc0e00640 1 -- 172.21.15.97:0/2531589897 learned_addr learned my addr 172.21.15.97:0/2531589897 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.060+0000 7f9bb3e00640 1 --2- 172.21.15.97:0/2531589897 >> v2:172.21.15.188:3300/0 conn(0x7f9bbc0b2e10 0x7f9bbc06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.060+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553717850 0 0) 0x7f9bbc06cba0 con 0x7f9bbc06c350 2024-01-31T05:03:56.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.060+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9c003520 con 0x7f9bbc06c350 2024-01-31T05:03:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347640858 0 0) 0x7f9b9c003520 con 0x7f9bbc06c350 2024-01-31T05:03:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 >> v2:172.21.15.188:3300/0 conn(0x7f9bbc0b2e10 msgr2=0x7f9bbc06dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:56.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 --2- 172.21.15.97:0/2531589897 >> v2:172.21.15.188:3300/0 conn(0x7f9bbc0b2e10 0x7f9bbc06dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 >> v1:172.21.15.188:6789/0 conn(0x7f9bbc0cb7d0 legacy=0x7f9bbc0cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc06d160 con 0x7f9bbc06c350 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f9bb40019f0 con 0x7f9bbc06c350 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 >> v1:172.21.15.97:6789/0 conn(0x7f9bbc06c350 legacy=0x7f9bbc0ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0039c0 0x7f9b9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0047c0 0x7f9b9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b9c005180 0x7f9b9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00c050 con 0x7f9b9c0039c0 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00c1c0 con 0x7f9b9c0047c0 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00c380 con 0x7f9b9c005180 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0039c0 0x7f9b9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.061+0000 7f9bb3e00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0047c0 0x7f9b9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bc0e00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b9c005180 0x7f9b9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bbbe00640 1 -- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b9c005180 msgr2=0x7f9b9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:56.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b9c005180 0x7f9b9c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bbbe00640 1 -- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0047c0 msgr2=0x7f9b9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:56.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0047c0 0x7f9b9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bbbe00640 1 -- 172.21.15.97:0/2531589897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c003520 con 0x7f9b9c0039c0 2024-01-31T05:03:56.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.062+0000 7f9bc0e00640 1 -- 172.21.15.97:0/2531589897 reap_dead start 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.063+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0039c0 0x7f9b9c003de0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001750 tx=0x7f9b9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f200971bc182cc77 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.063+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0052d0 con 0x7f9b9c0039c0 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.063+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac02e050 con 0x7f9b9c0039c0 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.063+0000 7f9bbb400640 1 -- 172.21.15.97:0/2531589897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003830 con 0x7f9b9c0039c0 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 -- 172.21.15.97:0/2531589897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0039c0 msgr2=0x7f9b9c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0039c0 0x7f9b9c003de0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001750 tx=0x7f9b9c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 -- 172.21.15.97:0/2531589897 shutdown_connections 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/2531589897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b9c0039c0 0x7f9b9c003de0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 -- 172.21.15.97:0/2531589897 >> 172.21.15.97:0/2531589897 conn(0x7f9bbc0b0a10 msgr2=0x7f9bbc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 -- 172.21.15.97:0/2531589897 shutdown_connections 2024-01-31T05:03:56.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.064+0000 7f9bc2e00640 1 -- 172.21.15.97:0/2531589897 wait complete. 2024-01-31T05:03:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.066+0000 7f9bc2e00640 1 Processor -- start 2024-01-31T05:03:56.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.066+0000 7f9bc2e00640 1 -- start start 2024-01-31T05:03:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc2e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0039c0 0x7f9bbc1647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc2e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 0x7f9bbc1fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc2e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bbc0cee00 0x7f9bbc0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc2e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0cf7f0 con 0x7f9bbc0999c0 2024-01-31T05:03:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc2e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc099e40 con 0x7f9b9c0039c0 2024-01-31T05:03:56.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc2e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc09a0a0 con 0x7f9bbc0cee00 2024-01-31T05:03:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 0x7f9bbc1fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 0x7f9bbc1fa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34242/0 (socket says 172.21.15.97:34242) 2024-01-31T05:03:56.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bb3e00640 1 -- 172.21.15.97:0/486419444 learned_addr learned my addr 172.21.15.97:0/486419444 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0039c0 0x7f9bbc1647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bc0e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bbc0cee00 0x7f9bbc0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.067+0000 7f9bb3e00640 1 -- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bbc0cee00 msgr2=0x7f9bbc0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb3e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bbc0cee00 0x7f9bbc0cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb3e00640 1 -- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0039c0 msgr2=0x7f9bbc1647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb3e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0039c0 0x7f9bbc1647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb3e00640 1 -- 172.21.15.97:0/486419444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc09a380 con 0x7f9bbc0999c0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb3e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 0x7f9bbc1fa5a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4011a70 tx=0x7f9ba4011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4009680 con 0x7f9bbc0999c0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400fb50 con 0x7f9bbc0999c0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400fee0 con 0x7f9bbc0999c0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc09a660 con 0x7f9bbc0999c0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.068+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc09ac20 con 0x7f9bbc0999c0 2024-01-31T05:03:56.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.070+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc0677c0 con 0x7f9bbc0999c0 2024-01-31T05:03:56.070 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.070+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9ba4014440 con 0x7f9bbc0999c0 2024-01-31T05:03:56.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.071+0000 7f9bb8c00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9b9406af20 0x7f9b9406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.071+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9ba408a090 con 0x7f9bbc0999c0 2024-01-31T05:03:56.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.072+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9b9406af20 0x7f9b9406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.072+0000 7f9bbbe00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9b9406af20 0x7f9b9406d3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9bac033000 tx=0x7f9bac009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.076+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9ba4054d70 con 0x7f9bbc0999c0 2024-01-31T05:03:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.270+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bbc0ae1d0 con 0x7f9bbc0999c0 2024-01-31T05:03:56.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.273+0000 7f9bb8c00640 1 -- 172.21.15.97:0/486419444 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f9ba4058390 con 0x7f9bbc0999c0 2024-01-31T05:03:56.273 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:03:56.273 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:03:56.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.275+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9b9406af20 msgr2=0x7f9b9406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.275+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9b9406af20 0x7f9b9406d3e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9bac033000 tx=0x7f9bac009030 comp rx=0 tx=0).stop 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.275+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 msgr2=0x7f9bbc1fa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.275+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 0x7f9bbc1fa5a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4011a70 tx=0x7f9ba4011f30 comp rx=0 tx=0).stop 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 shutdown_connections 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9bbc0cee00 0x7f9bbc0cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9b9406af20 0x7f9b9406d3e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bbc0999c0 0x7f9bbc1fa5a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 --2- 172.21.15.97:0/486419444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b9c0039c0 0x7f9bbc1647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 >> 172.21.15.97:0/486419444 conn(0x7f9bbc0b0a10 msgr2=0x7f9bbc070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 shutdown_connections 2024-01-31T05:03:56.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.276+0000 7f9bc2e00640 1 -- 172.21.15.97:0/486419444 wait complete. 2024-01-31T05:03:56.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:03:36.506793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:03:37.685194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:03:56.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:03:56.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.509+0000 7fd14b200640 1 Processor -- start 2024-01-31T05:03:56.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.509+0000 7fd14b200640 1 -- start start 2024-01-31T05:03:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd14b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd14415d690 0x7fd1440ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd14b200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd144161020 0x7fd1441614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd14b200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14415db40 con 0x7fd14406d580 2024-01-31T05:03:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd14b200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14415e7c0 con 0x7fd144161020 2024-01-31T05:03:56.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd14b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14415de20 con 0x7fd14415d690 2024-01-31T05:03:56.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd149200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd144161020 0x7fd1441614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd149200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd144161020 0x7fd1441614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34256/0 (socket says 172.21.15.97:34256) 2024-01-31T05:03:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd149200640 1 -- 172.21.15.97:0/3189936398 learned_addr learned my addr 172.21.15.97:0/3189936398 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.511+0000 7fd13b200640 1 --2- 172.21.15.97:0/3189936398 >> v2:172.21.15.188:3300/0 conn(0x7fd14415d690 0x7fd1440ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.512+0000 7fd149200640 1 -- 172.21.15.97:0/3189936398 >> v2:172.21.15.188:3300/0 conn(0x7fd14415d690 msgr2=0x7fd1440ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.512+0000 7fd149200640 1 --2- 172.21.15.97:0/3189936398 >> v2:172.21.15.188:3300/0 conn(0x7fd14415d690 0x7fd1440ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.512+0000 7fd149200640 1 -- 172.21.15.97:0/3189936398 >> v1:172.21.15.188:6789/0 conn(0x7fd14406d580 legacy=0x7fd14406d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.512+0000 7fd149200640 1 -- 172.21.15.97:0/3189936398 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14415e100 con 0x7fd144161020 2024-01-31T05:03:56.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.513+0000 7fd149200640 1 --2- 172.21.15.97:0/3189936398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1441614d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009e50 tx=0x7fd13c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33ab13a17ab51ec server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.513+0000 7fd143400640 1 -- 172.21.15.97:0/3189936398 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c02cb00 con 0x7fd144161020 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.513+0000 7fd143400640 1 -- 172.21.15.97:0/3189936398 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c02cc60 con 0x7fd144161020 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.513+0000 7fd143400640 1 -- 172.21.15.97:0/3189936398 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0343f0 con 0x7fd144161020 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 -- 172.21.15.97:0/3189936398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 msgr2=0x7fd1441614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 --2- 172.21.15.97:0/3189936398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1441614d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009e50 tx=0x7fd13c02bf60 comp rx=0 tx=0).stop 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 -- 172.21.15.97:0/3189936398 shutdown_connections 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 --2- 172.21.15.97:0/3189936398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1441614d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 --2- 172.21.15.97:0/3189936398 >> v2:172.21.15.188:3300/0 conn(0x7fd14415d690 0x7fd1440ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 -- 172.21.15.97:0/3189936398 >> 172.21.15.97:0/3189936398 conn(0x7fd1440b0a10 msgr2=0x7fd1440b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 -- 172.21.15.97:0/3189936398 shutdown_connections 2024-01-31T05:03:56.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.514+0000 7fd14b200640 1 -- 172.21.15.97:0/3189936398 wait complete. 2024-01-31T05:03:56.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.516+0000 7fd14b200640 1 Processor -- start 2024-01-31T05:03:56.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.516+0000 7fd14b200640 1 -- start start 2024-01-31T05:03:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd14b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14406d580 0x7fd144149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd14b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14415d690 0x7fd144071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd14b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1440719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd14b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441f3fd0 con 0x7fd144161020 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd14b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441f3740 con 0x7fd14415d690 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd14b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd144149e70 con 0x7fd14406d580 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd149200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1440719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd149200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1440719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34264/0 (socket says 172.21.15.97:34264) 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd149200640 1 -- 172.21.15.97:0/507666265 learned_addr learned my addr 172.21.15.97:0/507666265 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd13b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14415d690 0x7fd144071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.517+0000 7fd143e00640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14406d580 0x7fd144149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd149200640 1 -- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14406d580 msgr2=0x7fd144149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd149200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14406d580 0x7fd144149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd149200640 1 -- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14415d690 msgr2=0x7fd144071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd149200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14415d690 0x7fd144071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd149200640 1 -- 172.21.15.97:0/507666265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c009b30 con 0x7fd144161020 2024-01-31T05:03:56.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd143e00640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14406d580 0x7fd144149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:56.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd149200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1440719b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009e50 tx=0x7fd13c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c009510 con 0x7fd144161020 2024-01-31T05:03:56.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd13c03e450 con 0x7fd144161020 2024-01-31T05:03:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c03e800 con 0x7fd144161020 2024-01-31T05:03:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.518+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1441f3930 con 0x7fd144161020 2024-01-31T05:03:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.519+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1441f3bf0 con 0x7fd144161020 2024-01-31T05:03:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.520+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1441f4110 con 0x7fd144161020 2024-01-31T05:03:56.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.521+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd13c03e960 con 0x7fd144161020 2024-01-31T05:03:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.522+0000 7fd140c00640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd11c06aed0 0x7fd11c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.522+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd13c0ac5e0 con 0x7fd144161020 2024-01-31T05:03:56.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.522+0000 7fd143e00640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd11c06aed0 0x7fd11c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.522+0000 7fd143e00640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd11c06aed0 0x7fd11c06d390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd134001610 tx=0x7fd134009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.526+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd1441f4110 con 0x7fd144161020 2024-01-31T05:03:56.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.727+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1441f4110 con 0x7fd144161020 2024-01-31T05:03:56.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.728+0000 7fd140c00640 1 -- 172.21.15.97:0/507666265 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1441f4110 con 0x7fd144161020 2024-01-31T05:03:56.728 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:03:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd11c06aed0 msgr2=0x7fd11c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd11c06aed0 0x7fd11c06d390 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd134001610 tx=0x7fd134009040 comp rx=0 tx=0).stop 2024-01-31T05:03:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 msgr2=0x7fd1440719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1440719b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009e50 tx=0x7fd13c004a40 comp rx=0 tx=0).stop 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 shutdown_connections 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd11c06aed0 0x7fd11c06d390 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd144161020 0x7fd1440719b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14415d690 0x7fd144071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 --2- 172.21.15.97:0/507666265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14406d580 0x7fd144149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 >> 172.21.15.97:0/507666265 conn(0x7fd1440b0a10 msgr2=0x7fd144153980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 shutdown_connections 2024-01-31T05:03:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.731+0000 7fd14b200640 1 -- 172.21.15.97:0/507666265 wait complete. 2024-01-31T05:03:56.751 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:03:56.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.965+0000 7f96d0600640 1 Processor -- start 2024-01-31T05:03:56.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.965+0000 7f96d0600640 1 -- start start 2024-01-31T05:03:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96d0600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96c815a350 0x7f96c80abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96d0600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c815a730 0x7f96c806da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96d0600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c815abc0 con 0x7f96c806dfc0 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96d0600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c80b2c90 con 0x7f96c815a350 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96d0600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c815aea0 con 0x7f96c815a730 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96cdc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96c815a350 0x7f96c80abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96cdc00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96c815a350 0x7f96c80abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34276/0 (socket says 172.21.15.97:34276) 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96cdc00640 1 -- 172.21.15.97:0/3156545873 learned_addr learned my addr 172.21.15.97:0/3156545873 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96cdc00640 1 -- 172.21.15.97:0/3156545873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c815a730 msgr2=0x7f96c806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:56.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96cd200640 1 --2- 172.21.15.97:0/3156545873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c815a730 0x7f96c806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.967+0000 7f96cdc00640 1 --2- 172.21.15.97:0/3156545873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c815a730 0x7f96c806da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.968+0000 7f96cdc00640 1 -- 172.21.15.97:0/3156545873 >> v1:172.21.15.188:6789/0 conn(0x7f96c806dfc0 legacy=0x7f96c80cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:56.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.968+0000 7f96cdc00640 1 -- 172.21.15.97:0/3156545873 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c815b180 con 0x7f96c815a350 2024-01-31T05:03:56.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.969+0000 7f96cdc00640 1 --2- 172.21.15.97:0/3156545873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c815a350 0x7f96c80abcf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009850 tx=0x7f96b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb306bc663288814 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.969+0000 7f96c7e00640 1 -- 172.21.15.97:0/3156545873 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8004230 con 0x7f96c815a350 2024-01-31T05:03:56.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.969+0000 7f96c7e00640 1 -- 172.21.15.97:0/3156545873 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b8004390 con 0x7f96c815a350 2024-01-31T05:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.969+0000 7f96c7e00640 1 -- 172.21.15.97:0/3156545873 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8031830 con 0x7f96c815a350 2024-01-31T05:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 -- 172.21.15.97:0/3156545873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c815a350 msgr2=0x7f96c80abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 --2- 172.21.15.97:0/3156545873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c815a350 0x7f96c80abcf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009850 tx=0x7f96b802b8d0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 -- 172.21.15.97:0/3156545873 shutdown_connections 2024-01-31T05:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 --2- 172.21.15.97:0/3156545873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c815a730 0x7f96c806da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 --2- 172.21.15.97:0/3156545873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c815a350 0x7f96c80abcf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 -- 172.21.15.97:0/3156545873 >> 172.21.15.97:0/3156545873 conn(0x7f96c80b0a10 msgr2=0x7f96c80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 -- 172.21.15.97:0/3156545873 shutdown_connections 2024-01-31T05:03:56.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.970+0000 7f96d0600640 1 -- 172.21.15.97:0/3156545873 wait complete. 2024-01-31T05:03:56.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.971+0000 7f96d0600640 1 Processor -- start 2024-01-31T05:03:56.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.972+0000 7f96d0600640 1 -- start start 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96d0600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 0x7f96c80c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96d0600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96c815a350 0x7f96c814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96d0600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c80c81b0 0x7f96c80cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96d0600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80cafb0 con 0x7f96c806dfc0 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96d0600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c814c0b0 con 0x7f96c815a350 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96d0600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80ca330 con 0x7f96c80c81b0 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 0x7f96c80c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 0x7f96c80c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34292/0 (socket says 172.21.15.97:34292) 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 -- 172.21.15.97:0/3851776699 learned_addr learned my addr 172.21.15.97:0/3851776699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 -- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c80c81b0 msgr2=0x7f96c80cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c80c81b0 0x7f96c80cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 -- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96c815a350 msgr2=0x7f96c814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cd200640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96c815a350 0x7f96c814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96c815a350 0x7f96c814bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cdc00640 1 -- 172.21.15.97:0/3851776699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8009460 con 0x7f96c806dfc0 2024-01-31T05:03:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.973+0000 7f96cd200640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96c815a350 0x7f96c814bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.974+0000 7f96cdc00640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 0x7f96c80c9c90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f96b8000c00 tx=0x7f96b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.974+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8031830 con 0x7f96c806dfc0 2024-01-31T05:03:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.974+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b803f070 con 0x7f96c806dfc0 2024-01-31T05:03:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.974+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8031bd0 con 0x7f96c806dfc0 2024-01-31T05:03:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.974+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c80ca610 con 0x7f96c806dfc0 2024-01-31T05:03:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.974+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c80cabd0 con 0x7f96c806dfc0 2024-01-31T05:03:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.975+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c8067830 con 0x7f96c806dfc0 2024-01-31T05:03:56.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.976+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f96b8041030 con 0x7f96c806dfc0 2024-01-31T05:03:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.977+0000 7f96c5600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f96a006af00 0x7f96a006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.977+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f96b80ab9d0 con 0x7f96c806dfc0 2024-01-31T05:03:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.977+0000 7f96cd200640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f96a006af00 0x7f96a006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:56.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.978+0000 7f96cd200640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f96a006af00 0x7f96a006d3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f96c80b2360 tx=0x7f96b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:56.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:56.982+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f96b80767f0 con 0x7f96c806dfc0 2024-01-31T05:03:57.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.176+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96c8067830 con 0x7f96c806dfc0 2024-01-31T05:03:57.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.177+0000 7f96c5600640 1 -- 172.21.15.97:0/3851776699 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96b8079e10 con 0x7f96c806dfc0 2024-01-31T05:03:57.176 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:03:57.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f96a006af00 msgr2=0x7f96a006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f96a006af00 0x7f96a006d3c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f96c80b2360 tx=0x7f96b0008040 comp rx=0 tx=0).stop 2024-01-31T05:03:57.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 msgr2=0x7f96c80c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 0x7f96c80c9c90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f96b8000c00 tx=0x7f96b80046c0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 shutdown_connections 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96c80c81b0 0x7f96c80cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96c815a350 0x7f96c814bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f96a006af00 0x7f96a006d3c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 --2- 172.21.15.97:0/3851776699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96c806dfc0 0x7f96c80c9c90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.179+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 >> 172.21.15.97:0/3851776699 conn(0x7f96c80b0a10 msgr2=0x7f96c80cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.180+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 shutdown_connections 2024-01-31T05:03:57.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.180+0000 7f96d0600640 1 -- 172.21.15.97:0/3851776699 wait complete. 2024-01-31T05:03:57.197 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:03:57.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.410+0000 7fb5d1600640 1 Processor -- start 2024-01-31T05:03:57.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.410+0000 7fb5d1600640 1 -- start start 2024-01-31T05:03:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5d1600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5d1600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5cc06c810 0x7fb5cc0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5d1600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc15e7b0 con 0x7fb5cc071440 2024-01-31T05:03:57.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5d1600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0b4130 con 0x7fb5cc06c810 2024-01-31T05:03:57.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5d1600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc15e4d0 con 0x7fb5cc1576b0 2024-01-31T05:03:57.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5c3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb5cc071440 0x7fb5cc06c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49590/0 (socket says 172.21.15.97:49590) 2024-01-31T05:03:57.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5c3e00640 1 -- 172.21.15.97:0/3044420678 learned_addr learned my addr 172.21.15.97:0/3044420678 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:57.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5d0c00640 1 --2- 172.21.15.97:0/3044420678 >> v2:172.21.15.97:3300/0 conn(0x7fb5cc06c810 0x7fb5cc0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5cbe00640 1 --2- 172.21.15.97:0/3044420678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.412+0000 7fb5cb400640 1 -- 172.21.15.97:0/3044420678 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822903538 0 0) 0x7fb5cc15e7b0 con 0x7fb5cc071440 2024-01-31T05:03:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.413+0000 7fb5d0c00640 1 -- 172.21.15.97:0/3044420678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 msgr2=0x7fb5cc070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.413+0000 7fb5d0c00640 1 --2- 172.21.15.97:0/3044420678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.413+0000 7fb5d0c00640 1 -- 172.21.15.97:0/3044420678 >> v1:172.21.15.97:6789/0 conn(0x7fb5cc071440 legacy=0x7fb5cc06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.413+0000 7fb5d0c00640 1 -- 172.21.15.97:0/3044420678 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc070420 con 0x7fb5cc06c810 2024-01-31T05:03:57.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.414+0000 7fb5d0c00640 1 --2- 172.21.15.97:0/3044420678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc0b1c10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009d90 tx=0x7fb5c402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492f7231ef27ea1f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:57.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.414+0000 7fb5cb400640 1 -- 172.21.15.97:0/3044420678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c402cc50 con 0x7fb5cc06c810 2024-01-31T05:03:57.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.414+0000 7fb5cb400640 1 -- 172.21.15.97:0/3044420678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c402cdb0 con 0x7fb5cc06c810 2024-01-31T05:03:57.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.414+0000 7fb5cb400640 1 -- 172.21.15.97:0/3044420678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4034660 con 0x7fb5cc06c810 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 -- 172.21.15.97:0/3044420678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 msgr2=0x7fb5cc0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 --2- 172.21.15.97:0/3044420678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc0b1c10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009d90 tx=0x7fb5c402bea0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 -- 172.21.15.97:0/3044420678 shutdown_connections 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 --2- 172.21.15.97:0/3044420678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc0b1c10 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 --2- 172.21.15.97:0/3044420678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 -- 172.21.15.97:0/3044420678 >> 172.21.15.97:0/3044420678 conn(0x7fb5cc06ad40 msgr2=0x7fb5cc0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 -- 172.21.15.97:0/3044420678 shutdown_connections 2024-01-31T05:03:57.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.415+0000 7fb5d1600640 1 -- 172.21.15.97:0/3044420678 wait complete. 2024-01-31T05:03:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.417+0000 7fb5d1600640 1 Processor -- start 2024-01-31T05:03:57.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.417+0000 7fb5d1600640 1 -- start start 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5d1600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5d1600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc071440 0x7fb5cc0ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5d1600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5d1600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0adbc0 con 0x7fb5cc06c810 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5d1600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc143c70 con 0x7fb5cc071440 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5d1600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0acf40 con 0x7fb5cc1576b0 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc143730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34320/0 (socket says 172.21.15.97:34320) 2024-01-31T05:03:57.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 -- 172.21.15.97:0/17554075 learned_addr learned my addr 172.21.15.97:0/17554075 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 -- 172.21.15.97:0/17554075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 msgr2=0x7fb5cc0ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 -- 172.21.15.97:0/17554075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc071440 msgr2=0x7fb5cc0ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc071440 0x7fb5cc0ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.418+0000 7fb5cbe00640 1 -- 172.21.15.97:0/17554075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c4009a70 con 0x7fb5cc06c810 2024-01-31T05:03:57.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.419+0000 7fb5cbe00640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc143730 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d5f0 tx=0x7fb5bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.419+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0125f0 con 0x7fb5cc06c810 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.419+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc012750 con 0x7fb5cc06c810 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.419+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc012ae0 con 0x7fb5cc06c810 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.419+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc0ad220 con 0x7fb5cc06c810 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.419+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc0ad7e0 con 0x7fb5cc06c810 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.420+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc0676b0 con 0x7fb5cc06c810 2024-01-31T05:03:57.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.421+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb5bc012c40 con 0x7fb5cc06c810 2024-01-31T05:03:57.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.422+0000 7fb5c8c00640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5a406aed0 0x7fb5a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.422+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb5bc089cc0 con 0x7fb5cc06c810 2024-01-31T05:03:57.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.422+0000 7fb5c3e00640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5a406aed0 0x7fb5a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.423+0000 7fb5c3e00640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5a406aed0 0x7fb5a406d390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4007fd0 tx=0x7fb5b4007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:57.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.426+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb5bc090080 con 0x7fb5cc06c810 2024-01-31T05:03:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.619+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5cc0b1550 con 0x7fb5cc06c810 2024-01-31T05:03:57.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.620+0000 7fb5c8c00640 1 -- 172.21.15.97:0/17554075 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5bc0549b0 con 0x7fb5cc06c810 2024-01-31T05:03:57.619 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:03:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.622+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5a406aed0 msgr2=0x7fb5a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.622+0000 7fb5d1600640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5a406aed0 0x7fb5a406d390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4007fd0 tx=0x7fb5b4007920 comp rx=0 tx=0).stop 2024-01-31T05:03:57.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.622+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 msgr2=0x7fb5cc143730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.622+0000 7fb5d1600640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc143730 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d5f0 tx=0x7fb5bc00dab0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.622+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 shutdown_connections 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.622+0000 7fb5d1600640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5cc1576b0 0x7fb5cc0ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.623+0000 7fb5d1600640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5cc071440 0x7fb5cc0ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.623+0000 7fb5d1600640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb5a406aed0 0x7fb5a406d390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.623+0000 7fb5d1600640 1 --2- 172.21.15.97:0/17554075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc06c810 0x7fb5cc143730 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.623+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 >> 172.21.15.97:0/17554075 conn(0x7fb5cc06ad40 msgr2=0x7fb5cc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.623+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 shutdown_connections 2024-01-31T05:03:57.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.623+0000 7fb5d1600640 1 -- 172.21.15.97:0/17554075 wait complete. 2024-01-31T05:03:57.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:03:57.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.841+0000 7f03b8400640 1 Processor -- start 2024-01-31T05:03:57.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.842+0000 7f03b8400640 1 -- start start 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b8400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f03b00ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b8400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f03b015ef80 0x7f03b015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b8400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b015f920 con 0x7f03b015b4b0 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b8400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00cd1b0 con 0x7f03b015ef80 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b8400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00cf740 con 0x7f03b00cc730 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b5a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f03b015b4b0 0x7f03b00cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49592/0 (socket says 172.21.15.97:49592) 2024-01-31T05:03:57.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.843+0000 7f03b5a00640 1 -- 172.21.15.97:0/737649533 learned_addr learned my addr 172.21.15.97:0/737649533 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03b5000640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f03b00ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03b6400640 1 --2- 172.21.15.97:0/737649533 >> v2:172.21.15.188:3300/0 conn(0x7f03b015ef80 0x7f03b015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923685748 0 0) 0x7f03b015f920 con 0x7f03b015b4b0 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0390003520 con 0x7f03b015b4b0 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707238358 0 0) 0x7f0390003520 con 0x7f03b015b4b0 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 msgr2=0x7f03b00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f03b00ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 >> v2:172.21.15.188:3300/0 conn(0x7f03b015ef80 msgr2=0x7f03b015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 --2- 172.21.15.97:0/737649533 >> v2:172.21.15.188:3300/0 conn(0x7f03b015ef80 0x7f03b015f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b015b890 con 0x7f03b015b4b0 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f0398001670 con 0x7f03b015b4b0 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03b5000640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f03b00ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 >> v1:172.21.15.97:6789/0 conn(0x7f03b015b4b0 legacy=0x7f03b00cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f0390003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0390003eb0 0x7f03900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f0390005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00cd1b0 con 0x7f0390004d00 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00cf740 con 0x7f0390003eb0 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.844+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039000b9b0 con 0x7f03b00cc730 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b6400640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f0390005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b5a00640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f0390003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b5000640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0390003eb0 0x7f03900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b6400640 1 -- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 msgr2=0x7f0390003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:57.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b6400640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b00cc730 0x7f0390003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b6400640 1 -- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0390003eb0 msgr2=0x7f03900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:03:57.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b6400640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0390003eb0 0x7f03900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b6400640 1 -- 172.21.15.97:0/737649533 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390003520 con 0x7f0390004d00 2024-01-31T05:03:57.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.845+0000 7f03b5000640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0390003eb0 0x7f03900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.846+0000 7f03b6400640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f0390005180 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ac0 tx=0x7f03900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac718935f20b09af server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.846+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a802c710 con 0x7f0390004d00 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.846+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a802c870 con 0x7f0390004d00 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.846+0000 7f03afe00640 1 -- 172.21.15.97:0/737649533 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a802cc00 con 0x7f0390004d00 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b8400640 1 -- 172.21.15.97:0/737649533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 msgr2=0x7f0390005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b8400640 1 --2- 172.21.15.97:0/737649533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f0390005180 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009ac0 tx=0x7f03900057f0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b6400640 1 -- 172.21.15.97:0/737649533 reap_dead start 2024-01-31T05:03:57.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b8400640 1 -- 172.21.15.97:0/737649533 shutdown_connections 2024-01-31T05:03:57.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b8400640 1 -- 172.21.15.97:0/737649533 >> 172.21.15.97:0/737649533 conn(0x7f03b00b2b90 msgr2=0x7f03b00b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:57.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b8400640 1 -- 172.21.15.97:0/737649533 shutdown_connections 2024-01-31T05:03:57.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.847+0000 7f03b8400640 1 -- 172.21.15.97:0/737649533 wait complete. 2024-01-31T05:03:57.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.849+0000 7f03b8400640 1 Processor -- start 2024-01-31T05:03:57.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.849+0000 7f03b8400640 1 -- start start 2024-01-31T05:03:57.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b8400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f03b00c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b8400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b0155cf0 0x7f03b00ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b8400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b00ca9b0 0x7f03b006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b8400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00c4bd0 con 0x7f0390004d00 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b8400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00c5470 con 0x7f03b00ca9b0 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b8400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0156170 con 0x7f03b0155cf0 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f03b00c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b5a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f03b00c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34340/0 (socket says 172.21.15.97:34340) 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.850+0000 7f03b5a00640 1 -- 172.21.15.97:0/631092781 learned_addr learned my addr 172.21.15.97:0/631092781 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b5a00640 1 -- 172.21.15.97:0/631092781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b0155cf0 msgr2=0x7f03b00ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b5a00640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b0155cf0 0x7f03b00ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b5a00640 1 -- 172.21.15.97:0/631092781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b00ca9b0 msgr2=0x7f03b006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b5a00640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b00ca9b0 0x7f03b006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:57.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b5a00640 1 -- 172.21.15.97:0/631092781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00cae50 con 0x7f0390004d00 2024-01-31T05:03:57.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b5a00640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f03b00c5900 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0398001eb0 tx=0x7f039800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:57.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800c7f0 con 0x7f0390004d00 2024-01-31T05:03:57.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039800c970 con 0x7f0390004d00 2024-01-31T05:03:57.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800cd20 con 0x7f0390004d00 2024-01-31T05:03:57.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.851+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b00c4e20 con 0x7f0390004d00 2024-01-31T05:03:57.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.852+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b01569e0 con 0x7f0390004d00 2024-01-31T05:03:57.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.853+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b0156ba0 con 0x7f0390004d00 2024-01-31T05:03:57.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.854+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f039801d460 con 0x7f0390004d00 2024-01-31T05:03:57.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.854+0000 7f03ad600640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f038806af20 0x7f038806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:03:57.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.855+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0398089710 con 0x7f0390004d00 2024-01-31T05:03:57.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.855+0000 7f03b5000640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f038806af20 0x7f038806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:03:57.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.855+0000 7f03b5000640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f038806af20 0x7f038806d3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f03b006d9a0 tx=0x7f03a000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:03:57.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:57.859+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f0398054500 con 0x7f0390004d00 2024-01-31T05:03:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.065+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03b0067830 con 0x7f0390004d00 2024-01-31T05:03:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.066+0000 7f03ad600640 1 -- 172.21.15.97:0/631092781 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0398057b20 con 0x7f0390004d00 2024-01-31T05:03:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:03:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.068+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f038806af20 msgr2=0x7f038806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.068+0000 7f03b8400640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f038806af20 0x7f038806d3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f03b006d9a0 tx=0x7f03a000b040 comp rx=0 tx=0).stop 2024-01-31T05:03:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.068+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 msgr2=0x7f03b00c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:03:58.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.068+0000 7f03b8400640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f03b00c5900 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0398001eb0 tx=0x7f039800ed50 comp rx=0 tx=0).stop 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 shutdown_connections 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b00ca9b0 0x7f03b006c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b0155cf0 0x7f03b00ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f038806af20 0x7f038806d3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 --2- 172.21.15.97:0/631092781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0390004d00 0x7f03b00c5900 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 >> 172.21.15.97:0/631092781 conn(0x7f03b00b2b90 msgr2=0x7f03b00aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 shutdown_connections 2024-01-31T05:03:58.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:03:58.069+0000 7f03b8400640 1 -- 172.21.15.97:0/631092781 wait complete. 2024-01-31T05:03:58.087 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:03:58.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:03:58.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:03:58.172 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[58403]: fuse finished with error 0 and tester_r 0 2024-01-31T05:03:58.179 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:04.281 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:04.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:04.281 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:04:04.340 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:04:04.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.554+0000 7f6129400640 1 Processor -- start 2024-01-31T05:04:04.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.554+0000 7f6129400640 1 -- start start 2024-01-31T05:04:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6129400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6124157bd0 0x7f61240ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6129400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612415e560 con 0x7f612415e140 2024-01-31T05:04:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6129400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61240ad9a0 con 0x7f61240cf620 2024-01-31T05:04:04.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6129400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612415ea70 con 0x7f6124157bd0 2024-01-31T05:04:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6122800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6124157bd0 0x7f61240ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6123200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f612415e140 0x7f612415ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49594/0 (socket says 172.21.15.97:49594) 2024-01-31T05:04:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.556+0000 7f6123200640 1 -- 172.21.15.97:0/3201635205 learned_addr learned my addr 172.21.15.97:0/3201635205 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.557+0000 7f6122800640 1 -- 172.21.15.97:0/3201635205 >> v1:172.21.15.188:6789/0 conn(0x7f61240cf620 legacy=0x7f612415da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:04.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.557+0000 7f6122800640 1 -- 172.21.15.97:0/3201635205 >> v1:172.21.15.97:6789/0 conn(0x7f612415e140 legacy=0x7f612415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:04.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.557+0000 7f6122800640 1 -- 172.21.15.97:0/3201635205 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612415e790 con 0x7f6124157bd0 2024-01-31T05:04:04.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.558+0000 7f6122800640 1 --2- 172.21.15.97:0/3201635205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6124157bd0 0x7f61240ade30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f61140097a0 tx=0x7f611402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f9faffd1b4fb26 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:04.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.558+0000 7f6121400640 1 -- 172.21.15.97:0/3201635205 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114031830 con 0x7f6124157bd0 2024-01-31T05:04:04.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.558+0000 7f6121400640 1 -- 172.21.15.97:0/3201635205 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6114031990 con 0x7f6124157bd0 2024-01-31T05:04:04.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.558+0000 7f6121400640 1 -- 172.21.15.97:0/3201635205 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114031d20 con 0x7f6124157bd0 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 -- 172.21.15.97:0/3201635205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6124157bd0 msgr2=0x7f61240ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 --2- 172.21.15.97:0/3201635205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6124157bd0 0x7f61240ade30 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f61140097a0 tx=0x7f611402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 -- 172.21.15.97:0/3201635205 shutdown_connections 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 --2- 172.21.15.97:0/3201635205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6124157bd0 0x7f61240ade30 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 -- 172.21.15.97:0/3201635205 >> 172.21.15.97:0/3201635205 conn(0x7f6124066490 msgr2=0x7f6124066870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 -- 172.21.15.97:0/3201635205 shutdown_connections 2024-01-31T05:04:04.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.559+0000 7f6129400640 1 -- 172.21.15.97:0/3201635205 wait complete. 2024-01-31T05:04:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.561+0000 7f6129400640 1 Processor -- start 2024-01-31T05:04:04.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.561+0000 7f6129400640 1 -- start start 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6129400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61240cf620 0x7f6124144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6129400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124157bd0 0x7f61240af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6129400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 0x7f61241f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6129400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6124144a90 con 0x7f612415e140 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6129400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61240b0160 con 0x7f6124157bd0 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6129400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61240b0440 con 0x7f61240cf620 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6123200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 0x7f61241f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6123200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 0x7f61241f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34352/0 (socket says 172.21.15.97:34352) 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6123200640 1 -- 172.21.15.97:0/769164854 learned_addr learned my addr 172.21.15.97:0/769164854 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6122800640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61240cf620 0x7f6124144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.563+0000 7f6121e00640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124157bd0 0x7f61240af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6123200640 1 -- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61240cf620 msgr2=0x7f6124144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6123200640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61240cf620 0x7f6124144550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6123200640 1 -- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124157bd0 msgr2=0x7f61240af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6123200640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124157bd0 0x7f61240af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6123200640 1 -- 172.21.15.97:0/769164854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114009400 con 0x7f612415e140 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6123200640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 0x7f61241f6420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f611c00ca80 tx=0x7f611c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00ac30 con 0x7f612415e140 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c00ad90 con 0x7f612415e140 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61240b0720 con 0x7f612415e140 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c012600 con 0x7f612415e140 2024-01-31T05:04:04.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.564+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61240b0ce0 con 0x7f612415e140 2024-01-31T05:04:04.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.566+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61240677c0 con 0x7f612415e140 2024-01-31T05:04:04.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.566+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f611c007070 con 0x7f612415e140 2024-01-31T05:04:04.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.568+0000 7f611aa00640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f60fc06aed0 0x7f60fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:04.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.568+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f611c089540 con 0x7f612415e140 2024-01-31T05:04:04.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.568+0000 7f6122800640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f60fc06aed0 0x7f60fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:04.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.569+0000 7f6122800640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f60fc06aed0 0x7f60fc06d390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6114036000 tx=0x7f611403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:04.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.573+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f611c054380 con 0x7f612415e140 2024-01-31T05:04:04.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.802+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6124067ff0 con 0x7f612415e140 2024-01-31T05:04:04.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.803+0000 7f611aa00640 1 -- 172.21.15.97:0/769164854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f611c0579a0 con 0x7f612415e140 2024-01-31T05:04:04.803 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f60fc06aed0 msgr2=0x7f60fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f60fc06aed0 0x7f60fc06d390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6114036000 tx=0x7f611403c040 comp rx=0 tx=0).stop 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 msgr2=0x7f61241f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 0x7f61241f6420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f611c00ca80 tx=0x7f611c00cf40 comp rx=0 tx=0).stop 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 shutdown_connections 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f60fc06aed0 0x7f60fc06d390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f612415e140 0x7f61241f6420 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6124157bd0 0x7f61240af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 --2- 172.21.15.97:0/769164854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61240cf620 0x7f6124144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:04.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 >> 172.21.15.97:0/769164854 conn(0x7f6124066490 msgr2=0x7f61240b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:04.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 shutdown_connections 2024-01-31T05:04:04.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:04.806+0000 7f6129400640 1 -- 172.21.15.97:0/769164854 wait complete. 2024-01-31T05:04:04.825 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:04:04.826 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:04:04.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:04.827 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:04:04.881 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:04:04.881 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:04:04.881 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:04:04.882 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42279sec preferred_lft 42279sec 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.883 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:04:04.884 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether ca:84:a9:c5:b2:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:04.884 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c884:a9ff:fec5:b219/64 scope link 2024-01-31T05:04:04.884 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:04.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:04.885 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:04:04.970 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:04:04.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:04.974 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:04:05.028 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:04:05.029 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:04:05.029 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:04:05.029 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:04:05.029 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:04:05.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:05.030 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:04:05.030 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:04:05.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:04:05.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:04:05.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:04:05.030 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:04:05.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:05.216 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:04:05.265 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:04:05.265 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:04:05.266 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:04:05.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:05.267 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:04:05.364 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:04:05.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:04:05.364 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:05.418 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:04:05.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:04:05.419 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:05.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:04:05.561 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:04:05.616 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:04:05.617 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:05.618 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:05.619 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:05.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:05.620 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:04:05.671 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:04:05.671 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:04:05.675 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:04:05.747 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:04:05.788 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:04:05.788 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:04:05.789 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:04:05.790 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:05.791 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:05.791 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:05.791 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:05.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:05.792 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:04:05.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:04:05.858+0000 7fc3a4640480 -1 init, newargv = 0x559d3be36bf0 newargc=15 2024-01-31T05:04:05.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[59366]: starting ceph client 2024-01-31T05:04:05.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[59366]: starting fuse 2024-01-31T05:04:05.877 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:04:05.877 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:05.878 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:04:05.879 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:05.880 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:04:05.880 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:04:06.072 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.59366.asok 2024-01-31T05:04:06.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:04:06.088 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.59366.asok 2024-01-31T05:04:06.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:06.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.59366.asok status 2024-01-31T05:04:06.325 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:06.325 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:04:06.325 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "59366", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6510, 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:04:06.326 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6510 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 516195736 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 516195736 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6510 192.168.0.1:0/516195736", 2024-01-31T05:04:06.327 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/516195736", 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 27, 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 55, 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:04:06.328 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:06.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:06.340 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:04:06.340 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:06.392 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:04:06.392 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:06.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:06.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:06.486 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> import errno 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> if False: 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> ret = os.read(fd, 1024) 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> else: 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> os.write(fd, b'"'"'content'"'"') 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> if e.errno != errno.EPERM: 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> raise 2024-01-31T05:04:06.487 DEBUG:teuthology.orchestra.run.smithi097:> else: 2024-01-31T05:04:06.488 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("client does not check permission of data pool") 2024-01-31T05:04:06.488 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:04:06.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:04:06.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-01-31T05:04:06.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.919+0000 7f742b600640 1 Processor -- start 2024-01-31T05:04:06.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.919+0000 7f742b600640 1 -- start start 2024-01-31T05:04:06.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.920+0000 7f742b600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240cbf20 0x7f74240722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:06.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.920+0000 7f742b600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74240abcf0 0x7f74240ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:06.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.920+0000 7f742b600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742406cd20 con 0x7f742415b470 2024-01-31T05:04:06.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.920+0000 7f742b600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240ce970 con 0x7f74240abcf0 2024-01-31T05:04:06.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.920+0000 7f742b600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742415b850 con 0x7f74240cbf20 2024-01-31T05:04:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.921+0000 7f741be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240cbf20 0x7f74240722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.921+0000 7f7428c00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f742415b470 0x7f74240cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:59650/0 (socket says 172.21.15.97:59650) 2024-01-31T05:04:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.921+0000 7f7428c00640 1 -- 172.21.15.97:0/1263155328 learned_addr learned my addr 172.21.15.97:0/1263155328 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:06.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.921+0000 7f7429600640 1 --2- 172.21.15.97:0/1263155328 >> v2:172.21.15.188:3300/0 conn(0x7f74240abcf0 0x7f74240ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.922+0000 7f741be00640 1 -- 172.21.15.97:0/1263155328 >> v2:172.21.15.188:3300/0 conn(0x7f74240abcf0 msgr2=0x7f74240ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.922+0000 7f741be00640 1 --2- 172.21.15.97:0/1263155328 >> v2:172.21.15.188:3300/0 conn(0x7f74240abcf0 0x7f74240ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.922+0000 7f741be00640 1 -- 172.21.15.97:0/1263155328 >> v1:172.21.15.188:6789/0 conn(0x7f742415b470 legacy=0x7f74240cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.922+0000 7f741be00640 1 -- 172.21.15.97:0/1263155328 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240ceb60 con 0x7f74240cbf20 2024-01-31T05:04:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.922+0000 7f7429600640 1 --2- 172.21.15.97:0/1263155328 >> v2:172.21.15.188:3300/0 conn(0x7f74240abcf0 0x7f74240ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:04:06.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.923+0000 7f741be00640 1 --2- 172.21.15.97:0/1263155328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240cbf20 0x7f74240722c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097d0 tx=0x7f740c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79c88ff4d5aa1e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f7423e00640 1 -- 172.21.15.97:0/1263155328 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c031800 con 0x7f74240cbf20 2024-01-31T05:04:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f7423e00640 1 -- 172.21.15.97:0/1263155328 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c031960 con 0x7f74240cbf20 2024-01-31T05:04:06.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f7423e00640 1 -- 172.21.15.97:0/1263155328 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c031cf0 con 0x7f74240cbf20 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 -- 172.21.15.97:0/1263155328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240cbf20 msgr2=0x7f74240722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 --2- 172.21.15.97:0/1263155328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240cbf20 0x7f74240722c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097d0 tx=0x7f740c02b8a0 comp rx=0 tx=0).stop 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 -- 172.21.15.97:0/1263155328 shutdown_connections 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 --2- 172.21.15.97:0/1263155328 >> v2:172.21.15.188:3300/0 conn(0x7f74240abcf0 0x7f74240ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 --2- 172.21.15.97:0/1263155328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240cbf20 0x7f74240722c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 -- 172.21.15.97:0/1263155328 >> 172.21.15.97:0/1263155328 conn(0x7f74240b0a10 msgr2=0x7f74240b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 -- 172.21.15.97:0/1263155328 shutdown_connections 2024-01-31T05:04:06.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.924+0000 7f742b600640 1 -- 172.21.15.97:0/1263155328 wait complete. 2024-01-31T05:04:06.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.926+0000 7f742b600640 1 Processor -- start 2024-01-31T05:04:06.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.926+0000 7f742b600640 1 -- start start 2024-01-31T05:04:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.927+0000 7f742b600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240abcf0 0x7f7424154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.927+0000 7f742b600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240cbf20 0x7f7424153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.927+0000 7f742b600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 0x7f7424145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.927+0000 7f742b600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240b3dc0 con 0x7f742415b470 2024-01-31T05:04:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.927+0000 7f742b600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240b2ed0 con 0x7f74240cbf20 2024-01-31T05:04:06.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.927+0000 7f742b600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74240b31b0 con 0x7f74240abcf0 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 0x7f7424145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 0x7f7424145470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53962/0 (socket says 172.21.15.97:53962) 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 -- 172.21.15.97:0/85383212 learned_addr learned my addr 172.21.15.97:0/85383212 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7428c00640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240abcf0 0x7f7424154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f741be00640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240cbf20 0x7f7424153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 -- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240abcf0 msgr2=0x7f7424154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240abcf0 0x7f7424154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 -- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240cbf20 msgr2=0x7f7424153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240cbf20 0x7f7424153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.928+0000 7f7429600640 1 -- 172.21.15.97:0/85383212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f741c009a70 con 0x7f742415b470 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.929+0000 7f7429600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 0x7f7424145470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f741c00ef90 tx=0x7f741c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:06.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.929+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c009d70 con 0x7f742415b470 2024-01-31T05:04:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.929+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c00f040 con 0x7f742415b470 2024-01-31T05:04:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.929+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0146c0 con 0x7f742415b470 2024-01-31T05:04:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.929+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c009400 con 0x7f742415b470 2024-01-31T05:04:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.929+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74240b3770 con 0x7f742415b470 2024-01-31T05:04:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.930+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7424067830 con 0x7f742415b470 2024-01-31T05:04:06.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.931+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f741c014820 con 0x7f742415b470 2024-01-31T05:04:06.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.932+0000 7f7421600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f73fc06aed0 0x7f73fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:06.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.932+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f741c08a750 con 0x7f742415b470 2024-01-31T05:04:06.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.932+0000 7f7428c00640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f73fc06aed0 0x7f73fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:06.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.933+0000 7f7428c00640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f73fc06aed0 0x7f73fc06d390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f74140058d0 tx=0x7f7414009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:06.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:06.937+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f741c055570 con 0x7f742415b470 2024-01-31T05:04:07.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.202+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7f74240ce4a0 con 0x7f742415b470 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.206+0000 7f7421600640 1 -- 172.21.15.97:0/85383212 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 183+0+140 (secure 0 0 0) 0x7f741c0180a0 con 0x7f742415b470 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow w pool=cephfs_data" 2024-01-31T05:04:07.206 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f73fc06aed0 msgr2=0x7f73fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f73fc06aed0 0x7f73fc06d390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f74140058d0 tx=0x7f7414009040 comp rx=0 tx=0).stop 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 msgr2=0x7f7424145470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 0x7f7424145470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f741c00ef90 tx=0x7f741c009f90 comp rx=0 tx=0).stop 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 shutdown_connections 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f73fc06aed0 0x7f73fc06d390 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742415b470 0x7f7424145470 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240cbf20 0x7f7424153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 --2- 172.21.15.97:0/85383212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74240abcf0 0x7f7424154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.209+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 >> 172.21.15.97:0/85383212 conn(0x7f74240b0a10 msgr2=0x7f742415dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.210+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 shutdown_connections 2024-01-31T05:04:07.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.210+0000 7f742b600640 1 -- 172.21.15.97:0/85383212 wait complete. 2024-01-31T05:04:07.228 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:04:07.228 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:07.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:07.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.484+0000 7fd998000640 1 Processor -- start 2024-01-31T05:04:07.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.484+0000 7fd998000640 1 -- start start 2024-01-31T05:04:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.485+0000 7fd998000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9900b1d10 0x7fd9900b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.485+0000 7fd998000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 0x7fd99015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.485+0000 7fd998000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9900b2630 con 0x7fd99015a600 2024-01-31T05:04:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.485+0000 7fd998000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99006c8b0 con 0x7fd9900b1d10 2024-01-31T05:04:07.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.485+0000 7fd998000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99015b120 con 0x7fd99006e0a0 2024-01-31T05:04:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.486+0000 7fd996000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 0x7fd99015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.486+0000 7fd996000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 0x7fd99015aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:58636/0 (socket says 172.21.15.97:58636) 2024-01-31T05:04:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.486+0000 7fd996000640 1 -- 172.21.15.97:0/309204874 learned_addr learned my addr 172.21.15.97:0/309204874 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:07.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.486+0000 7fd995600640 1 --2- 172.21.15.97:0/309204874 >> v2:172.21.15.188:3300/0 conn(0x7fd9900b1d10 0x7fd9900b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.487+0000 7fd996000640 1 -- 172.21.15.97:0/309204874 >> v2:172.21.15.188:3300/0 conn(0x7fd9900b1d10 msgr2=0x7fd9900b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.487+0000 7fd996000640 1 --2- 172.21.15.97:0/309204874 >> v2:172.21.15.188:3300/0 conn(0x7fd9900b1d10 0x7fd9900b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.487+0000 7fd996000640 1 -- 172.21.15.97:0/309204874 >> v1:172.21.15.188:6789/0 conn(0x7fd99015a600 legacy=0x7fd99015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.487+0000 7fd996000640 1 -- 172.21.15.97:0/309204874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99015b400 con 0x7fd99006e0a0 2024-01-31T05:04:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.488+0000 7fd996000640 1 --2- 172.21.15.97:0/309204874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 0x7fd99015aa60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd988009e50 tx=0x7fd98802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651f6fc35148c756 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.489+0000 7fd98fe00640 1 -- 172.21.15.97:0/309204874 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98802cb00 con 0x7fd99006e0a0 2024-01-31T05:04:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.489+0000 7fd98fe00640 1 -- 172.21.15.97:0/309204874 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98802cc60 con 0x7fd99006e0a0 2024-01-31T05:04:07.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.489+0000 7fd98fe00640 1 -- 172.21.15.97:0/309204874 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880343f0 con 0x7fd99006e0a0 2024-01-31T05:04:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 -- 172.21.15.97:0/309204874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 msgr2=0x7fd99015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 --2- 172.21.15.97:0/309204874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 0x7fd99015aa60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd988009e50 tx=0x7fd98802bf60 comp rx=0 tx=0).stop 2024-01-31T05:04:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 -- 172.21.15.97:0/309204874 shutdown_connections 2024-01-31T05:04:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 --2- 172.21.15.97:0/309204874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd99006e0a0 0x7fd99015aa60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 --2- 172.21.15.97:0/309204874 >> v2:172.21.15.188:3300/0 conn(0x7fd9900b1d10 0x7fd9900b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 -- 172.21.15.97:0/309204874 >> 172.21.15.97:0/309204874 conn(0x7fd9900b0a10 msgr2=0x7fd9900b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 -- 172.21.15.97:0/309204874 shutdown_connections 2024-01-31T05:04:07.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.490+0000 7fd998000640 1 -- 172.21.15.97:0/309204874 wait complete. 2024-01-31T05:04:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.492+0000 7fd998000640 1 Processor -- start 2024-01-31T05:04:07.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.492+0000 7fd998000640 1 -- start start 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd998000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 0x7fd99009bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd998000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9900b1d10 0x7fd9901f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd998000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd99015a600 0x7fd990156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd998000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9900999c0 con 0x7fd99006e0a0 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd998000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99009c1a0 con 0x7fd99015a600 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd998000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901f6a10 con 0x7fd9900b1d10 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd995600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 0x7fd99009bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd995600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 0x7fd99009bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53966/0 (socket says 172.21.15.97:53966) 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd995600640 1 -- 172.21.15.97:0/3479300321 learned_addr learned my addr 172.21.15.97:0/3479300321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:07.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd994c00640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9900b1d10 0x7fd9901f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.493+0000 7fd996000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd99015a600 0x7fd990156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd995600640 1 -- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9900b1d10 msgr2=0x7fd9901f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd995600640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9900b1d10 0x7fd9901f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd995600640 1 -- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd99015a600 msgr2=0x7fd990156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd995600640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd99015a600 0x7fd990156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd995600640 1 -- 172.21.15.97:0/3479300321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd988009b30 con 0x7fd99006e0a0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd995600640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 0x7fd99009bc60 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd98000ea10 tx=0x7fd98000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980009670 con 0x7fd99006e0a0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98000ca00 con 0x7fd99006e0a0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98000cdb0 con 0x7fd99006e0a0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9900adb80 con 0x7fd99006e0a0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.494+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd990157050 con 0x7fd99006e0a0 2024-01-31T05:04:07.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.496+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9900677c0 con 0x7fd99006e0a0 2024-01-31T05:04:07.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.497+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd980013470 con 0x7fd99006e0a0 2024-01-31T05:04:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.498+0000 7fd98d600640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd96806aed0 0x7fd96806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.498+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd9800899e0 con 0x7fd99006e0a0 2024-01-31T05:04:07.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.498+0000 7fd994c00640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd96806aed0 0x7fd96806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.499+0000 7fd994c00640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd96806aed0 0x7fd96806d390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd99015d710 tx=0x7fd978009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:07.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.502+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd9800546d0 con 0x7fd99006e0a0 2024-01-31T05:04:07.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.697+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd99006e740 con 0x7fd99006e0a0 2024-01-31T05:04:07.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.700+0000 7fd98d600640 1 -- 172.21.15.97:0/3479300321 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7fd980017090 con 0x7fd99006e0a0 2024-01-31T05:04:07.700 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:07.700 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:07.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.702+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd96806aed0 msgr2=0x7fd96806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.702+0000 7fd998000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd96806aed0 0x7fd96806d390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd99015d710 tx=0x7fd978009040 comp rx=0 tx=0).stop 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 msgr2=0x7fd99009bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 0x7fd99009bc60 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd98000ea10 tx=0x7fd98000eed0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 shutdown_connections 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd99015a600 0x7fd990156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9900b1d10 0x7fd9901f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd96806aed0 0x7fd96806d390 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 --2- 172.21.15.97:0/3479300321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99006e0a0 0x7fd99009bc60 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 >> 172.21.15.97:0/3479300321 conn(0x7fd9900b0a10 msgr2=0x7fd9900b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 shutdown_connections 2024-01-31T05:04:07.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.703+0000 7fd998000640 1 -- 172.21.15.97:0/3479300321 wait complete. 2024-01-31T05:04:07.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:03:36.506793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:03:37.685194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:04:07.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:04:07.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.929+0000 7f0d4fe00640 1 Processor -- start 2024-01-31T05:04:07.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.929+0000 7f0d4fe00640 1 -- start start 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.930+0000 7f0d4fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d4815a350 0x7f0d480abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.930+0000 7f0d4fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d480cde10 0x7f0d480ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.930+0000 7f0d4fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4815abc0 con 0x7f0d4815a730 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.930+0000 7f0d4fe00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d480ce7e0 con 0x7f0d4815a350 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.930+0000 7f0d4fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d480b2c90 con 0x7f0d480cde10 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.930+0000 7f0d4d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d4815a350 0x7f0d480abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.931+0000 7f0d4d400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d4815a350 0x7f0d480abcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53978/0 (socket says 172.21.15.97:53978) 2024-01-31T05:04:07.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.931+0000 7f0d4d400640 1 -- 172.21.15.97:0/853802709 learned_addr learned my addr 172.21.15.97:0/853802709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.931+0000 7f0d4d400640 1 -- 172.21.15.97:0/853802709 >> v2:172.21.15.188:3300/0 conn(0x7f0d480cde10 msgr2=0x7f0d480ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.931+0000 7f0d4d400640 1 --2- 172.21.15.97:0/853802709 >> v2:172.21.15.188:3300/0 conn(0x7f0d480cde10 0x7f0d480ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.931+0000 7f0d4d400640 1 -- 172.21.15.97:0/853802709 >> v1:172.21.15.188:6789/0 conn(0x7f0d4815a730 legacy=0x7f0d4806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:07.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.931+0000 7f0d4d400640 1 -- 172.21.15.97:0/853802709 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4815aea0 con 0x7f0d4815a350 2024-01-31T05:04:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.932+0000 7f0d4d400640 1 --2- 172.21.15.97:0/853802709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4815a350 0x7f0d480abcf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ac260 tx=0x7f0d3802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26fb104bbe28543 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.932+0000 7f0d47e00640 1 -- 172.21.15.97:0/853802709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d380317d0 con 0x7f0d4815a350 2024-01-31T05:04:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.932+0000 7f0d47e00640 1 -- 172.21.15.97:0/853802709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38031930 con 0x7f0d4815a350 2024-01-31T05:04:07.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.932+0000 7f0d47e00640 1 -- 172.21.15.97:0/853802709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38031cc0 con 0x7f0d4815a350 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 -- 172.21.15.97:0/853802709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4815a350 msgr2=0x7f0d480abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/853802709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4815a350 0x7f0d480abcf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ac260 tx=0x7f0d3802b870 comp rx=0 tx=0).stop 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 -- 172.21.15.97:0/853802709 shutdown_connections 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/853802709 >> v2:172.21.15.188:3300/0 conn(0x7f0d480cde10 0x7f0d480ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/853802709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d4815a350 0x7f0d480abcf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 -- 172.21.15.97:0/853802709 >> 172.21.15.97:0/853802709 conn(0x7f0d480b0a10 msgr2=0x7f0d480b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:07.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 -- 172.21.15.97:0/853802709 shutdown_connections 2024-01-31T05:04:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.933+0000 7f0d4fe00640 1 -- 172.21.15.97:0/853802709 wait complete. 2024-01-31T05:04:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.935+0000 7f0d4fe00640 1 Processor -- start 2024-01-31T05:04:07.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.935+0000 7f0d4fe00640 1 -- start start 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 0x7f0d4809abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d4815a350 0x7f0d4809c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d48150710 0x7f0d48148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4fe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48150e80 con 0x7f0d480cde10 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4809b1c0 con 0x7f0d48150710 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4809b4a0 con 0x7f0d4815a350 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 0x7f0d4809abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 0x7f0d4809abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53980/0 (socket says 172.21.15.97:53980) 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.936+0000 7f0d4d400640 1 -- 172.21.15.97:0/1579903605 learned_addr learned my addr 172.21.15.97:0/1579903605 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4de00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d48150710 0x7f0d48148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4d400640 1 -- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d4815a350 msgr2=0x7f0d4809c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4d400640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d4815a350 0x7f0d4809c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4d400640 1 -- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d48150710 msgr2=0x7f0d48148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4d400640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d48150710 0x7f0d48148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4d400640 1 -- 172.21.15.97:0/1579903605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d38009470 con 0x7f0d480cde10 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4de00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d48150710 0x7f0d48148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4d400640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 0x7f0d4809abe0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38032870 con 0x7f0d480cde10 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d380329d0 con 0x7f0d480cde10 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38032d60 con 0x7f0d480cde10 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.937+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4809b780 con 0x7f0d480cde10 2024-01-31T05:04:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.938+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4809bd40 con 0x7f0d480cde10 2024-01-31T05:04:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.939+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4809bf30 con 0x7f0d480cde10 2024-01-31T05:04:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.940+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f0d38030070 con 0x7f0d480cde10 2024-01-31T05:04:07.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.940+0000 7f0d45600640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0d2006af00 0x7f0d2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:07.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.941+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0d380ab370 con 0x7f0d480cde10 2024-01-31T05:04:07.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.941+0000 7f0d4ca00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0d2006af00 0x7f0d2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:07.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.941+0000 7f0d4ca00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0d2006af00 0x7f0d2006d3c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0d481514e0 tx=0x7f0d30006ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:07.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:07.945+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f0d38076100 con 0x7f0d480cde10 2024-01-31T05:04:08.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.138+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d4809bf30 con 0x7f0d480cde10 2024-01-31T05:04:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.138+0000 7f0d45600640 1 -- 172.21.15.97:0/1579903605 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d38079720 con 0x7f0d480cde10 2024-01-31T05:04:08.138 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0d2006af00 msgr2=0x7f0d2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0d2006af00 0x7f0d2006d3c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0d481514e0 tx=0x7f0d30006ba0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 msgr2=0x7f0d4809abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 0x7f0d4809abe0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d380046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 shutdown_connections 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d48150710 0x7f0d48148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d4815a350 0x7f0d4809c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0d2006af00 0x7f0d2006d3c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 --2- 172.21.15.97:0/1579903605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d480cde10 0x7f0d4809abe0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 >> 172.21.15.97:0/1579903605 conn(0x7f0d480b0a10 msgr2=0x7f0d480cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 shutdown_connections 2024-01-31T05:04:08.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.141+0000 7f0d4fe00640 1 -- 172.21.15.97:0/1579903605 wait complete. 2024-01-31T05:04:08.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:04:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.367+0000 7f595f200640 1 Processor -- start 2024-01-31T05:04:08.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.368+0000 7f595f200640 1 -- start start 2024-01-31T05:04:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.369+0000 7f595f200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59580abcf0 0x7f59580722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.369+0000 7f595f200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595806cd20 con 0x7f59580cbf20 2024-01-31T05:04:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.369+0000 7f595f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580ceb60 con 0x7f595815b470 2024-01-31T05:04:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.369+0000 7f595f200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595815b850 con 0x7f59580abcf0 2024-01-31T05:04:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.369+0000 7f594f200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f59580cbf20 0x7f595806d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58938/0 (socket says 172.21.15.97:58938) 2024-01-31T05:04:08.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.369+0000 7f594f200640 1 -- 172.21.15.97:0/2290972134 learned_addr learned my addr 172.21.15.97:0/2290972134 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843414653 0 0) 0x7f595806cd20 con 0x7f59580cbf20 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5938003520 con 0x7f59580cbf20 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f595d200640 1 --2- 172.21.15.97:0/2290972134 >> v2:172.21.15.188:3300/0 conn(0x7f59580abcf0 0x7f59580722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 146814272 0 0) 0x7f5938003520 con 0x7f59580cbf20 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 >> v2:172.21.15.188:3300/0 conn(0x7f59580abcf0 msgr2=0x7f59580722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 --2- 172.21.15.97:0/2290972134 >> v2:172.21.15.188:3300/0 conn(0x7f59580abcf0 0x7f59580722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 >> v1:172.21.15.188:6789/0 conn(0x7f595815b470 legacy=0x7f59580cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595806c3f0 con 0x7f59580cbf20 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f5940001670 con 0x7f59580cbf20 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 >> v1:172.21.15.97:6789/0 conn(0x7f59580cbf20 legacy=0x7f595806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938003970 0x7f5938003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59380047c0 0x7f5938004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5938005180 0x7f593800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595815b850 con 0x7f5938005180 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593800c0d0 con 0x7f5938003970 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.370+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593800c290 con 0x7f59380047c0 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f595d200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5938005180 0x7f593800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f5957e00640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938003970 0x7f5938003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f595d200640 1 -- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59380047c0 msgr2=0x7f5938004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f594f200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59380047c0 0x7f5938004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f595d200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59380047c0 0x7f5938004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f595d200640 1 -- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938003970 msgr2=0x7f5938003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f595d200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938003970 0x7f5938003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f595d200640 1 -- 172.21.15.97:0/2290972134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5938003520 con 0x7f5938005180 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.371+0000 7f5957e00640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938003970 0x7f5938003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:08.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.372+0000 7f595d200640 1 -- 172.21.15.97:0/2290972134 reap_dead start 2024-01-31T05:04:08.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.372+0000 7f595d200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5938005180 0x7f593800b980 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5950007330 tx=0x7f59380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f176c33792402ee server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595002ccd0 con 0x7f5938005180 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595002ce30 con 0x7f5938005180 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f5957400640 1 -- 172.21.15.97:0/2290972134 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950034680 con 0x7f5938005180 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f595f200640 1 -- 172.21.15.97:0/2290972134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5938005180 msgr2=0x7f593800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f595f200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5938005180 0x7f593800b980 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5950007330 tx=0x7f59380057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f595f200640 1 -- 172.21.15.97:0/2290972134 shutdown_connections 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f595f200640 1 --2- 172.21.15.97:0/2290972134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5938005180 0x7f593800b980 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f595f200640 1 -- 172.21.15.97:0/2290972134 >> 172.21.15.97:0/2290972134 conn(0x7f59580b0a10 msgr2=0x7f59580b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.373+0000 7f595f200640 1 -- 172.21.15.97:0/2290972134 shutdown_connections 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.374+0000 7f595f200640 1 -- 172.21.15.97:0/2290972134 wait complete. 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.375+0000 7f595f200640 1 Processor -- start 2024-01-31T05:04:08.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.375+0000 7f595f200640 1 -- start start 2024-01-31T05:04:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.376+0000 7f595f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938005180 0x7f59581526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.376+0000 7f595f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5958153100 0x7f5958152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.376+0000 7f595f200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 0x7f5958156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.376+0000 7f595f200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5958152560 con 0x7f5958156a70 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.376+0000 7f595f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5958157440 con 0x7f5938005180 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.376+0000 7f595f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5958153580 con 0x7f5958153100 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 0x7f5958156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 0x7f5958156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53986/0 (socket says 172.21.15.97:53986) 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 -- 172.21.15.97:0/2494679321 learned_addr learned my addr 172.21.15.97:0/2494679321 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f5957e00640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938005180 0x7f59581526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f594f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5958153100 0x7f5958152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 -- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5958153100 msgr2=0x7f5958152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5958153100 0x7f5958152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 -- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938005180 msgr2=0x7f59581526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938005180 0x7f59581526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f595d200640 1 -- 172.21.15.97:0/2494679321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958156460 con 0x7f5958156a70 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.377+0000 7f5957e00640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938005180 0x7f59581526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.378+0000 7f595d200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 0x7f5958156f00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5950007330 tx=0x7f5950004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.378+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595003e610 con 0x7f5958156a70 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.378+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595003e770 con 0x7f5958156a70 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.378+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595003eae0 con 0x7f5958156a70 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.378+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59581537e0 con 0x7f5958156a70 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.378+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5958153da0 con 0x7f5958156a70 2024-01-31T05:04:08.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.379+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958067830 con 0x7f5958156a70 2024-01-31T05:04:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.380+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f595003b050 con 0x7f5958156a70 2024-01-31T05:04:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.381+0000 7f5954c00640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f593006aed0 0x7f593006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.381+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f59500aad70 con 0x7f5958156a70 2024-01-31T05:04:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.381+0000 7f5957e00640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f593006aed0 0x7f593006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.382+0000 7f5957e00640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f593006aed0 0x7f593006d390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5958159240 tx=0x7f5948009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:08.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.386+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5950075b50 con 0x7f5958156a70 2024-01-31T05:04:08.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.577+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59580cbc20 con 0x7f5958156a70 2024-01-31T05:04:08.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.577+0000 7f5954c00640 1 -- 172.21.15.97:0/2494679321 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5950079170 con 0x7f5958156a70 2024-01-31T05:04:08.577 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f593006aed0 msgr2=0x7f593006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f593006aed0 0x7f593006d390 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5958159240 tx=0x7f5948009040 comp rx=0 tx=0).stop 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 msgr2=0x7f5958156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 0x7f5958156f00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5950007330 tx=0x7f5950004ab0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 shutdown_connections 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f593006aed0 0x7f593006d390 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5958156a70 0x7f5958156f00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5958153100 0x7f5958152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 --2- 172.21.15.97:0/2494679321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5938005180 0x7f59581526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.580+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 >> 172.21.15.97:0/2494679321 conn(0x7f59580b0a10 msgr2=0x7f59580ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.581+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 shutdown_connections 2024-01-31T05:04:08.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.581+0000 7f595f200640 1 -- 172.21.15.97:0/2494679321 wait complete. 2024-01-31T05:04:08.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:04:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.802+0000 7ff27c200640 1 Processor -- start 2024-01-31T05:04:08.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.803+0000 7ff27c200640 1 -- start start 2024-01-31T05:04:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.804+0000 7ff27c200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff27415d590 0x7ff27415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.804+0000 7ff27c200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff274161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.804+0000 7ff27c200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740cded0 con 0x7ff2740cfae0 2024-01-31T05:04:08.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.804+0000 7ff27c200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27415d970 con 0x7ff27415d590 2024-01-31T05:04:08.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.804+0000 7ff27c200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740cf210 con 0x7ff2740cff20 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.804+0000 7ff27a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff274161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff278e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff2740cfae0 0x7ff274161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:59710/0 (socket says 172.21.15.97:59710) 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff278e00640 1 -- 172.21.15.97:0/834235529 learned_addr learned my addr 172.21.15.97:0/834235529 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff279800640 1 --2- 172.21.15.97:0/834235529 >> v2:172.21.15.188:3300/0 conn(0x7ff27415d590 0x7ff27415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff279800640 1 -- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 msgr2=0x7ff274161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff279800640 1 --2- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff274161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff279800640 1 -- 172.21.15.97:0/834235529 >> v1:172.21.15.188:6789/0 conn(0x7ff2740cfae0 legacy=0x7ff274161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff279800640 1 -- 172.21.15.97:0/834235529 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740ce170 con 0x7ff27415d590 2024-01-31T05:04:08.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.805+0000 7ff27a200640 1 --2- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff274161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:04:08.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.807+0000 7ff279800640 1 --2- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff27415e540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff264009d90 tx=0x7ff26402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9027254bb395689 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.807+0000 7ff273e00640 1 -- 172.21.15.97:0/834235529 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26402cce0 con 0x7ff27415d590 2024-01-31T05:04:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.807+0000 7ff273e00640 1 -- 172.21.15.97:0/834235529 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26402ce40 con 0x7ff27415d590 2024-01-31T05:04:08.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.807+0000 7ff273e00640 1 -- 172.21.15.97:0/834235529 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264034660 con 0x7ff27415d590 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 -- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 msgr2=0x7ff27415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 --2- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff27415e540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff264009d90 tx=0x7ff26402bf30 comp rx=0 tx=0).stop 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 -- 172.21.15.97:0/834235529 shutdown_connections 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 --2- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff274161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 --2- 172.21.15.97:0/834235529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff27415e540 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 -- 172.21.15.97:0/834235529 >> 172.21.15.97:0/834235529 conn(0x7ff27409c560 msgr2=0x7ff2740d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 -- 172.21.15.97:0/834235529 shutdown_connections 2024-01-31T05:04:08.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.808+0000 7ff27c200640 1 -- 172.21.15.97:0/834235529 wait complete. 2024-01-31T05:04:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.810+0000 7ff27c200640 1 Processor -- start 2024-01-31T05:04:08.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.810+0000 7ff27c200640 1 -- start start 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff27c200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 0x7ff27415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff27c200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff2740c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff27c200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff2740c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff27c200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740c7790 con 0x7ff2740cfae0 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff27c200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740c7a70 con 0x7ff27415d590 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff27c200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740c7d50 con 0x7ff2740cff20 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.811+0000 7ff279800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 0x7ff27415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 0x7ff27415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53992/0 (socket says 172.21.15.97:53992) 2024-01-31T05:04:08.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 -- 172.21.15.97:0/2599390436 learned_addr learned my addr 172.21.15.97:0/2599390436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff27a200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff2740c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff278e00640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff2740c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 -- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 msgr2=0x7ff2740c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff2740c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 -- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 msgr2=0x7ff2740c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff2740c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 -- 172.21.15.97:0/2599390436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c009400 con 0x7ff2740cfae0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff278e00640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff2740c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.812+0000 7ff279800640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 0x7ff27415ba50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff264037040 tx=0x7ff264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.813+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26403d600 con 0x7ff2740cfae0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.813+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26403d760 con 0x7ff2740cfae0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.813+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff264009a70 con 0x7ff2740cfae0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.813+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26403db10 con 0x7ff2740cfae0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.813+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2740c8310 con 0x7ff2740cfae0 2024-01-31T05:04:08.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.814+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2740c8500 con 0x7ff2740cfae0 2024-01-31T05:04:08.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.815+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff26403b040 con 0x7ff2740cfae0 2024-01-31T05:04:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.816+0000 7ff271600640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff24c06aed0 0x7ff24c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.817+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff2640ab590 con 0x7ff2740cfae0 2024-01-31T05:04:08.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.817+0000 7ff278e00640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff24c06aed0 0x7ff24c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:08.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.817+0000 7ff278e00640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff24c06aed0 0x7ff24c06d390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff27415bf90 tx=0x7ff25c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:08.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:08.821+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff2640763b0 con 0x7ff2740cfae0 2024-01-31T05:04:09.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.015+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff274067830 con 0x7ff2740cfae0 2024-01-31T05:04:09.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.017+0000 7ff271600640 1 -- 172.21.15.97:0/2599390436 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2640799d0 con 0x7ff2740cfae0 2024-01-31T05:04:09.016 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:04:09.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff24c06aed0 msgr2=0x7ff24c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff24c06aed0 0x7ff24c06d390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff27415bf90 tx=0x7ff25c009040 comp rx=0 tx=0).stop 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 msgr2=0x7ff27415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 0x7ff27415ba50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff264037040 tx=0x7ff264004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 shutdown_connections 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff27415d590 0x7ff2740c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2740cff20 0x7ff2740c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff24c06aed0 0x7ff24c06d390 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 --2- 172.21.15.97:0/2599390436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2740cfae0 0x7ff27415ba50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.019+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 >> 172.21.15.97:0/2599390436 conn(0x7ff27409c560 msgr2=0x7ff27406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.020+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 shutdown_connections 2024-01-31T05:04:09.020 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.020+0000 7ff27c200640 1 -- 172.21.15.97:0/2599390436 wait complete. 2024-01-31T05:04:09.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:04:09.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.234+0000 7faa41000640 1 Processor -- start 2024-01-31T05:04:09.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.234+0000 7faa41000640 1 -- start start 2024-01-31T05:04:09.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.236+0000 7faa41000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 0x7faa3c0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.236+0000 7faa41000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa3c0ae360 0x7faa3c0b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.236+0000 7faa41000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3c0b3f60 con 0x7faa3c15b080 2024-01-31T05:04:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.236+0000 7faa41000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa3c0b41c0 con 0x7faa3c0ae360 2024-01-31T05:04:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.236+0000 7faa41000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c15c540 con 0x7faa3c0ce680 2024-01-31T05:04:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.237+0000 7faa32800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 0x7faa3c0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.237+0000 7faa32800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 0x7faa3c0b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:58702/0 (socket says 172.21.15.97:58702) 2024-01-31T05:04:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.237+0000 7faa32800640 1 -- 172.21.15.97:0/3926016935 learned_addr learned my addr 172.21.15.97:0/3926016935 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.237+0000 7faa3be00640 1 --2- 172.21.15.97:0/3926016935 >> v2:172.21.15.188:3300/0 conn(0x7faa3c0ae360 0x7faa3c0b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.238+0000 7faa32800640 1 -- 172.21.15.97:0/3926016935 >> v2:172.21.15.188:3300/0 conn(0x7faa3c0ae360 msgr2=0x7faa3c0b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.238+0000 7faa32800640 1 --2- 172.21.15.97:0/3926016935 >> v2:172.21.15.188:3300/0 conn(0x7faa3c0ae360 0x7faa3c0b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.238+0000 7faa32800640 1 -- 172.21.15.97:0/3926016935 >> v1:172.21.15.188:6789/0 conn(0x7faa3c15b080 legacy=0x7faa3c0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.238+0000 7faa32800640 1 -- 172.21.15.97:0/3926016935 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c15b780 con 0x7faa3c0ce680 2024-01-31T05:04:09.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.238+0000 7faa3be00640 1 --2- 172.21.15.97:0/3926016935 >> v2:172.21.15.188:3300/0 conn(0x7faa3c0ae360 0x7faa3c0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-31T05:04:09.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.239+0000 7faa32800640 1 --2- 172.21.15.97:0/3926016935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 0x7faa3c0b3a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa24009870 tx=0x7faa2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2d4f232a9878b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:09.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.239+0000 7faa3aa00640 1 -- 172.21.15.97:0/3926016935 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24030840 con 0x7faa3c0ce680 2024-01-31T05:04:09.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.239+0000 7faa3aa00640 1 -- 172.21.15.97:0/3926016935 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa240309a0 con 0x7faa3c0ce680 2024-01-31T05:04:09.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.239+0000 7faa3aa00640 1 -- 172.21.15.97:0/3926016935 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24030d30 con 0x7faa3c0ce680 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 -- 172.21.15.97:0/3926016935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 msgr2=0x7faa3c0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 --2- 172.21.15.97:0/3926016935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 0x7faa3c0b3a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa24009870 tx=0x7faa2402b980 comp rx=0 tx=0).stop 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 -- 172.21.15.97:0/3926016935 shutdown_connections 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 --2- 172.21.15.97:0/3926016935 >> v2:172.21.15.188:3300/0 conn(0x7faa3c0ae360 0x7faa3c0b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 --2- 172.21.15.97:0/3926016935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ce680 0x7faa3c0b3a20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 -- 172.21.15.97:0/3926016935 >> 172.21.15.97:0/3926016935 conn(0x7faa3c06ad40 msgr2=0x7faa3c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 -- 172.21.15.97:0/3926016935 shutdown_connections 2024-01-31T05:04:09.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.240+0000 7faa41000640 1 -- 172.21.15.97:0/3926016935 wait complete. 2024-01-31T05:04:09.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.242+0000 7faa41000640 1 Processor -- start 2024-01-31T05:04:09.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.242+0000 7faa41000640 1 -- start start 2024-01-31T05:04:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa41000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ae360 0x7faa3c09a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa41000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 0x7faa3c1486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa41000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa3c15b080 0x7faa3c09b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa41000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c1590f0 con 0x7faa3c0ce680 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa41000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c09a1b0 con 0x7faa3c15b080 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa41000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c148c30 con 0x7faa3c0ae360 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa32800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 0x7faa3c1486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa32800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 0x7faa3c1486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54008/0 (socket says 172.21.15.97:54008) 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.243+0000 7faa32800640 1 -- 172.21.15.97:0/1818952706 learned_addr learned my addr 172.21.15.97:0/1818952706 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa3be00640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa3c15b080 0x7faa3c09b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa3b400640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ae360 0x7faa3c09a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa32800640 1 -- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ae360 msgr2=0x7faa3c09a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa32800640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ae360 0x7faa3c09a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa32800640 1 -- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa3c15b080 msgr2=0x7faa3c09b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa32800640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa3c15b080 0x7faa3c09b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa32800640 1 -- 172.21.15.97:0/1818952706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa34009a70 con 0x7faa3c0ce680 2024-01-31T05:04:09.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.244+0000 7faa32800640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 0x7faa3c1486f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7faa24000c00 tx=0x7faa24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.245+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24039c90 con 0x7faa3c0ce680 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.245+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa24039df0 con 0x7faa3c0ce680 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.245+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa240094d0 con 0x7faa3c0ce680 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.245+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa24043640 con 0x7faa3c0ce680 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.245+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c09bf90 con 0x7faa3c0ce680 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.246+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c09a360 con 0x7faa3c0ce680 2024-01-31T05:04:09.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.247+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7faa240437a0 con 0x7faa3c0ce680 2024-01-31T05:04:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.248+0000 7faa33e00640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa1406aed0 0x7faa1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.248+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7faa2403e070 con 0x7faa3c0ce680 2024-01-31T05:04:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.249+0000 7faa3b400640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa1406aed0 0x7faa1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.249+0000 7faa3b400640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa1406aed0 0x7faa1406d390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001640 tx=0x7faa2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:09.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.253+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7faa3c09a360 con 0x7faa3c0ce680 2024-01-31T05:04:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.459+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa3c09a320 con 0x7faa3c0ce680 2024-01-31T05:04:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.460+0000 7faa33e00640 1 -- 172.21.15.97:0/1818952706 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7faa24077610 con 0x7faa3c0ce680 2024-01-31T05:04:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:04:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa1406aed0 msgr2=0x7faa1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa1406aed0 0x7faa1406d390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001640 tx=0x7faa2c009040 comp rx=0 tx=0).stop 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 msgr2=0x7faa3c1486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 0x7faa3c1486f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7faa24000c00 tx=0x7faa24004730 comp rx=0 tx=0).stop 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 shutdown_connections 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa3c15b080 0x7faa3c09b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7faa1406aed0 0x7faa1406d390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa3c0ce680 0x7faa3c1486f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 --2- 172.21.15.97:0/1818952706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa3c0ae360 0x7faa3c09a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 >> 172.21.15.97:0/1818952706 conn(0x7faa3c06ad40 msgr2=0x7faa3c1548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 shutdown_connections 2024-01-31T05:04:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.463+0000 7faa41000640 1 -- 172.21.15.97:0/1818952706 wait complete. 2024-01-31T05:04:09.481 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:04:09.481 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:09.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:09.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.789+0000 7f134a800640 1 Processor -- start 2024-01-31T05:04:09.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.789+0000 7f134a800640 1 -- start start 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.790+0000 7f134a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f13440cb810 0x7f13440cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.790+0000 7f134a800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13440ccd00 0x7f13440cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.790+0000 7f134a800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134406d1a0 con 0x7f134415b400 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.790+0000 7f134a800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13440cc960 con 0x7f13440cb810 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.790+0000 7f134a800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134406c600 con 0x7f13440ccd00 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f13440cb810 0x7f13440cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f13440cb810 0x7f13440cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54022/0 (socket says 172.21.15.97:54022) 2024-01-31T05:04:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 -- 172.21.15.97:0/2450968013 learned_addr learned my addr 172.21.15.97:0/2450968013 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 -- 172.21.15.97:0/2450968013 >> v2:172.21.15.188:3300/0 conn(0x7f13440ccd00 msgr2=0x7f13440cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 --2- 172.21.15.97:0/2450968013 >> v2:172.21.15.188:3300/0 conn(0x7f13440ccd00 0x7f13440cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 -- 172.21.15.97:0/2450968013 >> v1:172.21.15.188:6789/0 conn(0x7f134415b400 legacy=0x7f134406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.791+0000 7f1343400640 1 -- 172.21.15.97:0/2450968013 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440cd6b0 con 0x7f13440cb810 2024-01-31T05:04:09.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.793+0000 7f1343400640 1 --2- 172.21.15.97:0/2450968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13440cb810 0x7f13440cbbf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f134415fe30 tx=0x7f133402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586cafe0ad6c5b57 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:09.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.793+0000 7f1342a00640 1 -- 172.21.15.97:0/2450968013 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334031870 con 0x7f13440cb810 2024-01-31T05:04:09.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.793+0000 7f1342a00640 1 -- 172.21.15.97:0/2450968013 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13340319d0 con 0x7f13440cb810 2024-01-31T05:04:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.793+0000 7f1342a00640 1 -- 172.21.15.97:0/2450968013 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334031d60 con 0x7f13440cb810 2024-01-31T05:04:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 -- 172.21.15.97:0/2450968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13440cb810 msgr2=0x7f13440cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 --2- 172.21.15.97:0/2450968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13440cb810 0x7f13440cbbf0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f134415fe30 tx=0x7f133402b910 comp rx=0 tx=0).stop 2024-01-31T05:04:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 -- 172.21.15.97:0/2450968013 shutdown_connections 2024-01-31T05:04:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 --2- 172.21.15.97:0/2450968013 >> v2:172.21.15.188:3300/0 conn(0x7f13440ccd00 0x7f13440cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 --2- 172.21.15.97:0/2450968013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13440cb810 0x7f13440cbbf0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 -- 172.21.15.97:0/2450968013 >> 172.21.15.97:0/2450968013 conn(0x7f13440b0a10 msgr2=0x7f13440b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 -- 172.21.15.97:0/2450968013 shutdown_connections 2024-01-31T05:04:09.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.794+0000 7f134a800640 1 -- 172.21.15.97:0/2450968013 wait complete. 2024-01-31T05:04:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.796+0000 7f134a800640 1 Processor -- start 2024-01-31T05:04:09.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.796+0000 7f134a800640 1 -- start start 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f134a800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13440cb810 0x7f134414b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f134a800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13440ccd00 0x7f134415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f134a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 0x7f13440c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f134a800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440c9e20 con 0x7f134415b400 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f134a800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134415e240 con 0x7f13440cb810 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f134a800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134409a660 con 0x7f13440ccd00 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f1343e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 0x7f13440c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f1343e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 0x7f13440c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54024/0 (socket says 172.21.15.97:54024) 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f1343e00640 1 -- 172.21.15.97:0/3476676071 learned_addr learned my addr 172.21.15.97:0/3476676071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.797+0000 7f1343400640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13440cb810 0x7f134414b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f133be00640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13440ccd00 0x7f134415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f1343e00640 1 -- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13440ccd00 msgr2=0x7f134415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f1343e00640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13440ccd00 0x7f134415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f1343e00640 1 -- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13440cb810 msgr2=0x7f134414b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f1343e00640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13440cb810 0x7f134414b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f1343e00640 1 -- 172.21.15.97:0/3476676071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340094d0 con 0x7f134415b400 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f1343e00640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 0x7f13440c82b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f133c00b970 tx=0x7f133c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c00cb40 con 0x7f134415b400 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c00ccc0 con 0x7f134415b400 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134409a940 con 0x7f134415b400 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c014680 con 0x7f134415b400 2024-01-31T05:04:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.798+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134409af00 con 0x7f134415b400 2024-01-31T05:04:09.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.799+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f134414be10 con 0x7f134415b400 2024-01-31T05:04:09.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.801+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f133c01d3b0 con 0x7f134415b400 2024-01-31T05:04:09.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.801+0000 7f133b400640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f131c06af20 0x7f131c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:09.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.801+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f133c0896d0 con 0x7f134415b400 2024-01-31T05:04:09.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.802+0000 7f1343400640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f131c06af20 0x7f131c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:09.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.802+0000 7f1343400640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f131c06af20 0x7f131c06d3e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1334036000 tx=0x7f133403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:09.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:09.807+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f133c0544d0 con 0x7f134415b400 2024-01-31T05:04:10.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.006+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f134409b0f0 con 0x7f134415b400 2024-01-31T05:04:10.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.007+0000 7f133b400640 1 -- 172.21.15.97:0/3476676071 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f133c057af0 con 0x7f134415b400 2024-01-31T05:04:10.007 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:10.007 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f131c06af20 msgr2=0x7f131c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f131c06af20 0x7f131c06d3e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1334036000 tx=0x7f133403c040 comp rx=0 tx=0).stop 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 msgr2=0x7f13440c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 0x7f13440c82b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f133c00b970 tx=0x7f133c00be30 comp rx=0 tx=0).stop 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 shutdown_connections 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f131c06af20 0x7f131c06d3e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134415b400 0x7f13440c82b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13440ccd00 0x7f134415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 --2- 172.21.15.97:0/3476676071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13440cb810 0x7f134414b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 >> 172.21.15.97:0/3476676071 conn(0x7f13440b0a10 msgr2=0x7f13440aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:10.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 shutdown_connections 2024-01-31T05:04:10.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.010+0000 7f134a800640 1 -- 172.21.15.97:0/3476676071 wait complete. 2024-01-31T05:04:10.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:03:36.506793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:03:37.685194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:04:10.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:04:10.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.242+0000 7fc415c00640 1 Processor -- start 2024-01-31T05:04:10.242 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.242+0000 7fc415c00640 1 -- start start 2024-01-31T05:04:10.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.244+0000 7fc415c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc41015b080 0x7fc4100b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.244+0000 7fc415c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4100ae360 0x7fc4100cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.244+0000 7fc415c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4100ce4b0 0x7fc41015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.244+0000 7fc415c00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100b41c0 con 0x7fc4100ae360 2024-01-31T05:04:10.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.244+0000 7fc415c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100ce920 con 0x7fc4100ce4b0 2024-01-31T05:04:10.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.244+0000 7fc415c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100b3f60 con 0x7fc41015b080 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4100ae360 0x7fc4100cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4100ae360 0x7fc4100cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54036/0 (socket says 172.21.15.97:54036) 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 -- 172.21.15.97:0/1719294169 learned_addr learned my addr 172.21.15.97:0/1719294169 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc415200640 1 --2- 172.21.15.97:0/1719294169 >> v2:172.21.15.188:3300/0 conn(0x7fc4100ce4b0 0x7fc41015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc40fe00640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc41015b080 0x7fc4100b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 -- 172.21.15.97:0/1719294169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc41015b080 msgr2=0x7fc4100b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc41015b080 0x7fc4100b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 -- 172.21.15.97:0/1719294169 >> v2:172.21.15.188:3300/0 conn(0x7fc4100ce4b0 msgr2=0x7fc41015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 --2- 172.21.15.97:0/1719294169 >> v2:172.21.15.188:3300/0 conn(0x7fc4100ce4b0 0x7fc41015eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc407200640 1 -- 172.21.15.97:0/1719294169 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41015c540 con 0x7fc4100ae360 2024-01-31T05:04:10.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc415200640 1 --2- 172.21.15.97:0/1719294169 >> v2:172.21.15.188:3300/0 conn(0x7fc4100ce4b0 0x7fc41015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:10.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.245+0000 7fc40fe00640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc41015b080 0x7fc4100b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.246+0000 7fc407200640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4100ae360 0x7fc4100cdf70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc4100cfd20 tx=0x7fc3f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f30da9784170ba server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.246+0000 7fc40f400640 1 -- 172.21.15.97:0/1719294169 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8031640 con 0x7fc4100ae360 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc40f400640 1 -- 172.21.15.97:0/1719294169 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f80317a0 con 0x7fc4100ae360 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc40f400640 1 -- 172.21.15.97:0/1719294169 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8031b30 con 0x7fc4100ae360 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 -- 172.21.15.97:0/1719294169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4100ae360 msgr2=0x7fc4100cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4100ae360 0x7fc4100cdf70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc4100cfd20 tx=0x7fc3f802b8f0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 -- 172.21.15.97:0/1719294169 shutdown_connections 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 --2- 172.21.15.97:0/1719294169 >> v2:172.21.15.188:3300/0 conn(0x7fc4100ce4b0 0x7fc41015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4100ae360 0x7fc4100cdf70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 --2- 172.21.15.97:0/1719294169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc41015b080 0x7fc4100b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 -- 172.21.15.97:0/1719294169 >> 172.21.15.97:0/1719294169 conn(0x7fc41006ad40 msgr2=0x7fc4100b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:10.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 -- 172.21.15.97:0/1719294169 shutdown_connections 2024-01-31T05:04:10.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.247+0000 7fc415c00640 1 -- 172.21.15.97:0/1719294169 wait complete. 2024-01-31T05:04:10.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.249+0000 7fc415c00640 1 Processor -- start 2024-01-31T05:04:10.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.249+0000 7fc415c00640 1 -- start start 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4100ae360 0x7fc4101531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ce4b0 0x7fc4100b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 0x7fc4100b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410152150 con 0x7fc41015b080 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410152430 con 0x7fc4100ae360 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc410152710 con 0x7fc4100ce4b0 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 0x7fc4100b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 0x7fc4100b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54044/0 (socket says 172.21.15.97:54044) 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.250+0000 7fc415200640 1 -- 172.21.15.97:0/4207422317 learned_addr learned my addr 172.21.15.97:0/4207422317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc40fe00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4100ae360 0x7fc4101531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc407200640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ce4b0 0x7fc4100b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415200640 1 -- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ce4b0 msgr2=0x7fc4100b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415200640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ce4b0 0x7fc4100b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415200640 1 -- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4100ae360 msgr2=0x7fc4101531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415200640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4100ae360 0x7fc4101531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415200640 1 -- 172.21.15.97:0/4207422317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80094d0 con 0x7fc41015b080 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415200640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 0x7fc4100b2f70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc40800cef0 tx=0x7fc40800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800d840 con 0x7fc41015b080 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40800d9c0 con 0x7fc41015b080 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4101529f0 con 0x7fc41015b080 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.251+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800dd70 con 0x7fc41015b080 2024-01-31T05:04:10.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.252+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc410153730 con 0x7fc41015b080 2024-01-31T05:04:10.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.253+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410152f10 con 0x7fc41015b080 2024-01-31T05:04:10.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.254+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc40801e3b0 con 0x7fc41015b080 2024-01-31T05:04:10.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.254+0000 7fc40cc00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3e806aed0 0x7fc3e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.254+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc40808a280 con 0x7fc41015b080 2024-01-31T05:04:10.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.255+0000 7fc40fe00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3e806aed0 0x7fc3e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.255+0000 7fc40fe00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3e806aed0 0x7fc3e806d390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc410156ad0 tx=0x7fc400008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:10.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.259+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc410152f10 con 0x7fc41015b080 2024-01-31T05:04:10.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.436+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc410152f10 con 0x7fc41015b080 2024-01-31T05:04:10.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.437+0000 7fc40cc00640 1 -- 172.21.15.97:0/4207422317 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc410152f10 con 0x7fc41015b080 2024-01-31T05:04:10.437 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:04:10.439 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.439+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3e806aed0 msgr2=0x7fc3e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.439+0000 7fc415c00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3e806aed0 0x7fc3e806d390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc410156ad0 tx=0x7fc400008040 comp rx=0 tx=0).stop 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.439+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 msgr2=0x7fc4100b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.439+0000 7fc415c00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 0x7fc4100b2f70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc40800cef0 tx=0x7fc40800a710 comp rx=0 tx=0).stop 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 shutdown_connections 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc3e806aed0 0x7fc3e806d390 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc41015b080 0x7fc4100b2f70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4100ce4b0 0x7fc4100b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 --2- 172.21.15.97:0/4207422317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4100ae360 0x7fc4101531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 >> 172.21.15.97:0/4207422317 conn(0x7fc41006ad40 msgr2=0x7fc410153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 shutdown_connections 2024-01-31T05:04:10.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.440+0000 7fc415c00640 1 -- 172.21.15.97:0/4207422317 wait complete. 2024-01-31T05:04:10.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:04:10.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.666+0000 7f8f0fa00640 1 Processor -- start 2024-01-31T05:04:10.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.666+0000 7f8f0fa00640 1 -- start start 2024-01-31T05:04:10.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.668+0000 7f8f0fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8f080722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.668+0000 7f8f0fa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0806cd20 con 0x7f8f080abcf0 2024-01-31T05:04:10.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.668+0000 7f8f0fa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f080ceb60 con 0x7f8f0815b470 2024-01-31T05:04:10.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.668+0000 7f8f0fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815b850 con 0x7f8f080cbf20 2024-01-31T05:04:10.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.668+0000 7f8f0da00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8f080abcf0 0x7f8f0806d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58942/0 (socket says 172.21.15.97:58942) 2024-01-31T05:04:10.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.668+0000 7f8f0da00640 1 -- 172.21.15.97:0/2701533534 learned_addr learned my addr 172.21.15.97:0/2701533534 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:10.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07e00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8f080722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187482698 0 0) 0x7f8f0806cd20 con 0x7f8f080abcf0 2024-01-31T05:04:10.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee8003520 con 0x7f8f080abcf0 2024-01-31T05:04:10.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1686658554 0 0) 0x7f8ee8003520 con 0x7f8f080abcf0 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 msgr2=0x7f8f080722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8f080722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 >> v1:172.21.15.188:6789/0 conn(0x7f8f0815b470 legacy=0x7f8f080cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0806c3f0 con 0x7f8f080abcf0 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f8f000019f0 con 0x7f8f080abcf0 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07e00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8f080722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 >> v1:172.21.15.97:6789/0 conn(0x7f8f080abcf0 legacy=0x7f8f0806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8ee80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee8003f90 0x7f8ee80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee8004d00 0x7f8ee8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815b850 con 0x7f8ee8004d00 2024-01-31T05:04:10.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee800bae0 con 0x7f8ee8003f90 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.669+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee800bca0 con 0x7f8f080cbf20 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0da00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee8004d00 0x7f8ee8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0d000640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8ee80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0da00640 1 -- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 msgr2=0x7f8ee80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0da00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f080cbf20 0x7f8ee80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0da00640 1 -- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee8003f90 msgr2=0x7f8ee80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0da00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ee8003f90 0x7f8ee80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.670+0000 7f8f0da00640 1 -- 172.21.15.97:0/2701533534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8003520 con 0x7f8ee8004d00 2024-01-31T05:04:10.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.671+0000 7f8f0da00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee8004d00 0x7f8ee8005180 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000a800 tx=0x7f8ee80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe6b927a9731b11 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:10.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.671+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000072d0 con 0x7f8ee8004d00 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.671+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0002f070 con 0x7f8ee8004d00 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.671+0000 7f8f07400640 1 -- 172.21.15.97:0/2701533534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000076f0 con 0x7f8ee8004d00 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2701533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee8004d00 msgr2=0x7f8ee8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2701533534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ee8004d00 0x7f8ee8005180 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000a800 tx=0x7f8ee80057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0da00640 1 -- 172.21.15.97:0/2701533534 reap_dead start 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2701533534 shutdown_connections 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2701533534 >> 172.21.15.97:0/2701533534 conn(0x7f8f080b0a10 msgr2=0x7f8f080b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2701533534 shutdown_connections 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.672+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2701533534 wait complete. 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.674+0000 7f8f0fa00640 1 Processor -- start 2024-01-31T05:04:10.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.674+0000 7f8f0fa00640 1 -- start start 2024-01-31T05:04:10.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f0fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ee8004d00 0x7f8f0809c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f0fa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 0x7f8f0815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f0fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f080b2c70 0x7f8f080b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f0fa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f080c8f90 con 0x7f8f0815b9d0 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f0fa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815c390 con 0x7f8f080b2c70 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f0fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f080c8380 con 0x7f8ee8004d00 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 0x7f8f0815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f07e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 0x7f8f0815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54060/0 (socket says 172.21.15.97:54060) 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.675+0000 7f8f07e00640 1 -- 172.21.15.97:0/2634475400 learned_addr learned my addr 172.21.15.97:0/2634475400 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f07e00640 1 -- 172.21.15.97:0/2634475400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ee8004d00 msgr2=0x7f8f0809c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f07e00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ee8004d00 0x7f8f0809c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f07e00640 1 -- 172.21.15.97:0/2634475400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f080b2c70 msgr2=0x7f8f080b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f07e00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f080b2c70 0x7f8f080b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f07e00640 1 -- 172.21.15.97:0/2634475400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f080c8660 con 0x7f8f0815b9d0 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f07e00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 0x7f8f0815be50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000e5c0 tx=0x7f8ef000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0004230 con 0x7f8f0815b9d0 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0004390 con 0x7f8f0815b9d0 2024-01-31T05:04:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0003850 con 0x7f8f0815b9d0 2024-01-31T05:04:10.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.676+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f080c8940 con 0x7f8f0815b9d0 2024-01-31T05:04:10.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.677+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0809c6e0 con 0x7f8f0815b9d0 2024-01-31T05:04:10.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.678+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f08067830 con 0x7f8f0815b9d0 2024-01-31T05:04:10.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.679+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8ef00039b0 con 0x7f8f0815b9d0 2024-01-31T05:04:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.679+0000 7f8f04c00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8ee006af00 0x7f8ee006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.680+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8ef0089290 con 0x7f8f0815b9d0 2024-01-31T05:04:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.680+0000 7f8f0d000640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8ee006af00 0x7f8ee006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:10.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.680+0000 7f8f0d000640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8ee006af00 0x7f8ee006d3c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8007fd0 tx=0x7f8ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:10.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.684+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f8ef0053f80 con 0x7f8f0815b9d0 2024-01-31T05:04:10.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.880+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f080c90d0 con 0x7f8f0815b9d0 2024-01-31T05:04:10.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.881+0000 7f8f04c00640 1 -- 172.21.15.97:0/2634475400 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f080c90d0 con 0x7f8f0815b9d0 2024-01-31T05:04:10.881 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8ee006af00 msgr2=0x7f8ee006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8ee006af00 0x7f8ee006d3c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8007fd0 tx=0x7f8ef8009040 comp rx=0 tx=0).stop 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 msgr2=0x7f8f0815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 0x7f8f0815be50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000e5c0 tx=0x7f8ef000ea80 comp rx=0 tx=0).stop 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 shutdown_connections 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f080b2c70 0x7f8f080b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f8ee006af00 0x7f8ee006d3c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f0815b9d0 0x7f8f0815be50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 --2- 172.21.15.97:0/2634475400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ee8004d00 0x7f8f0809c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:10.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 >> 172.21.15.97:0/2634475400 conn(0x7f8f080b0a10 msgr2=0x7f8f080b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:10.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 shutdown_connections 2024-01-31T05:04:10.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:10.884+0000 7f8f0fa00640 1 -- 172.21.15.97:0/2634475400 wait complete. 2024-01-31T05:04:10.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:04:11.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.111+0000 7fa698e00640 1 Processor -- start 2024-01-31T05:04:11.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.111+0000 7fa698e00640 1 -- start start 2024-01-31T05:04:11.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa698e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa69406d580 0x7fa69406d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa698e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940ceb40 con 0x7fa69415d690 2024-01-31T05:04:11.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa698e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69406dea0 con 0x7fa694161020 2024-01-31T05:04:11.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa698e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69415e7c0 con 0x7fa69406d580 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa691e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa69406d580 0x7fa69406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa691e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa69406d580 0x7fa69406d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54068/0 (socket says 172.21.15.97:54068) 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa691e00640 1 -- 172.21.15.97:0/1341342286 learned_addr learned my addr 172.21.15.97:0/1341342286 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.113+0000 7fa691400640 1 --1- 172.21.15.97:0/1341342286 >> v1:172.21.15.97:6789/0 conn(0x7fa69415d690 0x7fa6940ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58944/0 (socket says 172.21.15.97:58944) 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.114+0000 7fa691e00640 1 -- 172.21.15.97:0/1341342286 >> v1:172.21.15.188:6789/0 conn(0x7fa694161020 legacy=0x7fa6941614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.114+0000 7fa691e00640 1 -- 172.21.15.97:0/1341342286 >> v1:172.21.15.97:6789/0 conn(0x7fa69415d690 legacy=0x7fa6940ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.114+0000 7fa691e00640 1 -- 172.21.15.97:0/1341342286 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69415db40 con 0x7fa69406d580 2024-01-31T05:04:11.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.114+0000 7fa690a00640 1 -- 172.21.15.97:0/1341342286 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90094830 0 0) 0x7fa6940ceb40 con 0x7fa69415d690 2024-01-31T05:04:11.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.115+0000 7fa691e00640 1 --2- 172.21.15.97:0/1341342286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa69406d960 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa684009850 tx=0x7fa68402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9fa8c87986c910 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.115+0000 7fa690a00640 1 -- 172.21.15.97:0/1341342286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684004230 con 0x7fa69406d580 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.115+0000 7fa690a00640 1 -- 172.21.15.97:0/1341342286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa684004390 con 0x7fa69406d580 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.115+0000 7fa690a00640 1 -- 172.21.15.97:0/1341342286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684030840 con 0x7fa69406d580 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 -- 172.21.15.97:0/1341342286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 msgr2=0x7fa69406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 --2- 172.21.15.97:0/1341342286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa69406d960 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa684009850 tx=0x7fa68402b980 comp rx=0 tx=0).stop 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 -- 172.21.15.97:0/1341342286 shutdown_connections 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 --2- 172.21.15.97:0/1341342286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa69406d960 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 -- 172.21.15.97:0/1341342286 >> 172.21.15.97:0/1341342286 conn(0x7fa6940b0a10 msgr2=0x7fa6940b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 -- 172.21.15.97:0/1341342286 shutdown_connections 2024-01-31T05:04:11.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.116+0000 7fa698e00640 1 -- 172.21.15.97:0/1341342286 wait complete. 2024-01-31T05:04:11.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.117+0000 7fa698e00640 1 Processor -- start 2024-01-31T05:04:11.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.118+0000 7fa698e00640 1 -- start start 2024-01-31T05:04:11.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.118+0000 7fa698e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa694151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.118+0000 7fa698e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa69415d690 0x7fa6940c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa698e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa694161020 0x7fa694071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa698e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c8fc0 con 0x7fa69406d580 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa698e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940999c0 con 0x7fa694161020 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa698e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6941524d0 con 0x7fa69415d690 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa694151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa694151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54074/0 (socket says 172.21.15.97:54074) 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 -- 172.21.15.97:0/1306481596 learned_addr learned my addr 172.21.15.97:0/1306481596 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 -- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa69415d690 msgr2=0x7fa6940c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa69415d690 0x7fa6940c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 -- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa694161020 msgr2=0x7fa694071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa692800640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa694161020 0x7fa694071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa694161020 0x7fa694071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.119+0000 7fa691e00640 1 -- 172.21.15.97:0/1306481596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684009460 con 0x7fa69406d580 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.120+0000 7fa691e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa694151f90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa684000c00 tx=0x7fa6840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.120+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68403d070 con 0x7fa69406d580 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.120+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa684004500 con 0x7fa69406d580 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.120+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684030e40 con 0x7fa69406d580 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.120+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6940c8a00 con 0x7fa69406d580 2024-01-31T05:04:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.120+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa694099bd0 con 0x7fa69406d580 2024-01-31T05:04:11.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.121+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa694067830 con 0x7fa69406d580 2024-01-31T05:04:11.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.122+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa684003610 con 0x7fa69406d580 2024-01-31T05:04:11.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.123+0000 7fa68a000640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa66c06aed0 0x7fa66c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.123+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa6840aba40 con 0x7fa69406d580 2024-01-31T05:04:11.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.123+0000 7fa691400640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa66c06aed0 0x7fa66c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.124+0000 7fa691400640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa66c06aed0 0x7fa66c06d390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa69406c870 tx=0x7fa67c007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:11.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.127+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa684076730 con 0x7fa69406d580 2024-01-31T05:04:11.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.327+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6940cc6c0 con 0x7fa69406d580 2024-01-31T05:04:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.327+0000 7fa68a000640 1 -- 172.21.15.97:0/1306481596 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa684079d50 con 0x7fa69406d580 2024-01-31T05:04:11.327 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:04:11.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa66c06aed0 msgr2=0x7fa66c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa66c06aed0 0x7fa66c06d390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa69406c870 tx=0x7fa67c007980 comp rx=0 tx=0).stop 2024-01-31T05:04:11.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 msgr2=0x7fa694151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa694151f90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa684000c00 tx=0x7fa6840046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 shutdown_connections 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa694161020 0x7fa694071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa69415d690 0x7fa6940c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa66c06aed0 0x7fa66c06d390 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 --2- 172.21.15.97:0/1306481596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa69406d580 0x7fa694151f90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 >> 172.21.15.97:0/1306481596 conn(0x7fa6940b0a10 msgr2=0x7fa6940afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 shutdown_connections 2024-01-31T05:04:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.330+0000 7fa698e00640 1 -- 172.21.15.97:0/1306481596 wait complete. 2024-01-31T05:04:11.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:04:11.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.558+0000 7f2ce8e00640 1 Processor -- start 2024-01-31T05:04:11.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.558+0000 7f2ce8e00640 1 -- start start 2024-01-31T05:04:11.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.559+0000 7f2ce8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415b4b0 0x7f2ce40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.559+0000 7f2ce8e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce40cf740 con 0x7f2ce40cc730 2024-01-31T05:04:11.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.559+0000 7f2ce8e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce415b890 con 0x7f2ce415f690 2024-01-31T05:04:11.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.559+0000 7f2ce8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40cd1b0 con 0x7f2ce415b4b0 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce1400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ce40cc730 0x7f2ce415ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:58950/0 (socket says 172.21.15.97:58950) 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce1400640 1 -- 172.21.15.97:0/3894952339 learned_addr learned my addr 172.21.15.97:0/3894952339 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce1e00640 1 --2- 172.21.15.97:0/3894952339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415b4b0 0x7f2ce40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce0a00640 1 -- 172.21.15.97:0/3894952339 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361817737 0 0) 0x7f2ce40cf740 con 0x7f2ce40cc730 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce0a00640 1 -- 172.21.15.97:0/3894952339 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc4003520 con 0x7f2ce40cc730 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce1e00640 1 -- 172.21.15.97:0/3894952339 >> v1:172.21.15.188:6789/0 conn(0x7f2ce415f690 legacy=0x7f2ce40b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce1e00640 1 -- 172.21.15.97:0/3894952339 >> v1:172.21.15.97:6789/0 conn(0x7f2ce40cc730 legacy=0x7f2ce415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce0a00640 1 -- 172.21.15.97:0/3894952339 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945203280 0 0) 0x7f2cc4003520 con 0x7f2ce40cc730 2024-01-31T05:04:11.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.560+0000 7f2ce1e00640 1 -- 172.21.15.97:0/3894952339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce40d0110 con 0x7f2ce415b4b0 2024-01-31T05:04:11.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.562+0000 7f2ce1e00640 1 --2- 172.21.15.97:0/3894952339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415b4b0 0x7f2ce40cfbd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0097d0 tx=0x7f2ccc02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87d661fb781e5c14 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:11.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.562+0000 7f2ce0a00640 1 -- 172.21.15.97:0/3894952339 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40019f0 con 0x7f2ce415b4b0 2024-01-31T05:04:11.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.562+0000 7f2ce0a00640 1 -- 172.21.15.97:0/3894952339 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc02e050 con 0x7f2ce415b4b0 2024-01-31T05:04:11.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.562+0000 7f2ce0a00640 1 -- 172.21.15.97:0/3894952339 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc030730 con 0x7f2ce415b4b0 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 -- 172.21.15.97:0/3894952339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415b4b0 msgr2=0x7f2ce40cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/3894952339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415b4b0 0x7f2ce40cfbd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0097d0 tx=0x7f2ccc02b800 comp rx=0 tx=0).stop 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 -- 172.21.15.97:0/3894952339 shutdown_connections 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/3894952339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415b4b0 0x7f2ce40cfbd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 -- 172.21.15.97:0/3894952339 >> 172.21.15.97:0/3894952339 conn(0x7f2ce40b2b90 msgr2=0x7f2ce40b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 -- 172.21.15.97:0/3894952339 shutdown_connections 2024-01-31T05:04:11.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.563+0000 7f2ce8e00640 1 -- 172.21.15.97:0/3894952339 wait complete. 2024-01-31T05:04:11.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.565+0000 7f2ce8e00640 1 Processor -- start 2024-01-31T05:04:11.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.565+0000 7f2ce8e00640 1 -- start start 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce8e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 0x7f2ce40c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415b4b0 0x7f2ce4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415f690 0x7f2ce41e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce8e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce415a4f0 con 0x7f2ce40cc730 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce8e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40c6ff0 con 0x7f2ce415b4b0 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40c63e0 con 0x7f2ce415f690 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 0x7f2ce40c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 0x7f2ce40c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54088/0 (socket says 172.21.15.97:54088) 2024-01-31T05:04:11.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.566+0000 7f2ce1e00640 1 -- 172.21.15.97:0/2146518796 learned_addr learned my addr 172.21.15.97:0/2146518796 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:11.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1400640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415b4b0 0x7f2ce4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce2800640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415f690 0x7f2ce41e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1e00640 1 -- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415f690 msgr2=0x7f2ce41e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415f690 0x7f2ce41e6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1e00640 1 -- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415b4b0 msgr2=0x7f2ce4159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415b4b0 0x7f2ce4159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1e00640 1 -- 172.21.15.97:0/2146518796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009400 con 0x7f2ce40cc730 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2ce1e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 0x7f2ce40c7480 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc035040 tx=0x7f2ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.567+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc03c880 con 0x7f2ce40cc730 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.568+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc03c9e0 con 0x7f2ce40cc730 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.568+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce40c66c0 con 0x7f2ce40cc730 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.568+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc03cd50 con 0x7f2ce40cc730 2024-01-31T05:04:11.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.568+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce40c6c80 con 0x7f2ce40cc730 2024-01-31T05:04:11.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.569+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce40cf0b0 con 0x7f2ce40cc730 2024-01-31T05:04:11.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.570+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2ccc039040 con 0x7f2ce40cc730 2024-01-31T05:04:11.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.571+0000 7f2cda000640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cbc06aed0 0x7f2cbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:11.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.571+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2ccc042030 con 0x7f2ce40cc730 2024-01-31T05:04:11.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.571+0000 7f2ce1400640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cbc06aed0 0x7f2cbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:11.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.572+0000 7f2ce1400640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cbc06aed0 0x7f2cbc06d390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40c5fc0 tx=0x7f2cd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:11.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.575+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2ccc0761b0 con 0x7f2ce40cc730 2024-01-31T05:04:11.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.781+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ce4067830 con 0x7f2ce40cc730 2024-01-31T05:04:11.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.782+0000 7f2cda000640 1 -- 172.21.15.97:0/2146518796 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ccc0797d0 con 0x7f2ce40cc730 2024-01-31T05:04:11.782 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cbc06aed0 msgr2=0x7f2cbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cbc06aed0 0x7f2cbc06d390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2ce40c5fc0 tx=0x7f2cd4009040 comp rx=0 tx=0).stop 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 msgr2=0x7f2ce40c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 0x7f2ce40c7480 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc035040 tx=0x7f2ccc0046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 shutdown_connections 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce415f690 0x7f2ce41e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415b4b0 0x7f2ce4159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2cbc06aed0 0x7f2cbc06d390 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 --2- 172.21.15.97:0/2146518796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40cc730 0x7f2ce40c7480 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 >> 172.21.15.97:0/2146518796 conn(0x7f2ce40b2b90 msgr2=0x7f2ce40b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 shutdown_connections 2024-01-31T05:04:11.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:11.785+0000 7f2ce8e00640 1 -- 172.21.15.97:0/2146518796 wait complete. 2024-01-31T05:04:11.802 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:04:11.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:11.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:11.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[59366]: fuse finished with error 0 and tester_r 0 2024-01-31T05:04:11.886 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:13.302 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:13.310 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:17.988 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:17.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:17.988 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:04:18.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-31T05:04:18.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.255+0000 7f1e73200640 1 Processor -- start 2024-01-31T05:04:18.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.255+0000 7f1e73200640 1 -- start start 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.256+0000 7f1e73200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e7415ec70 0x7f1e74071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e73200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e740723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e73200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e74070400 con 0x7f1e7406cb90 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e73200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740ad8c0 con 0x7f1e7415ec70 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e73200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740ca580 con 0x7f1e740ae3b0 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e72800640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1e7406cb90 0x7f1e7415f050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39668/0 (socket says 172.21.15.97:39668) 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e72800640 1 -- 172.21.15.97:0/2947731040 learned_addr learned my addr 172.21.15.97:0/2947731040 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e71e00640 1 --2- 172.21.15.97:0/2947731040 >> v2:172.21.15.188:3300/0 conn(0x7f1e7415ec70 0x7f1e74071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e69200640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e740723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963785614 0 0) 0x7f1e74070400 con 0x7f1e7406cb90 2024-01-31T05:04:18.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.257+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e54003520 con 0x7f1e7406cb90 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2984358708 0 0) 0x7f1e54003520 con 0x7f1e7406cb90 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 msgr2=0x7f1e740723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e740723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 >> v2:172.21.15.188:3300/0 conn(0x7f1e7415ec70 msgr2=0x7f1e74071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 --2- 172.21.15.97:0/2947731040 >> v2:172.21.15.188:3300/0 conn(0x7f1e7415ec70 0x7f1e74071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74070650 con 0x7f1e7406cb90 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f1e6c001670 con 0x7f1e7406cb90 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 >> v1:172.21.15.97:6789/0 conn(0x7f1e7406cb90 legacy=0x7f1e7415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e54003970 0x7f1e54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54004d00 0x7f1e54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740ad8c0 con 0x7f1e54004d00 2024-01-31T05:04:18.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740ca580 con 0x7f1e54003970 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5400ba00 con 0x7f1e740ae3b0 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.258+0000 7f1e72800640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54004d00 0x7f1e54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e71e00640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e54003970 0x7f1e54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e69200640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e72800640 1 -- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 msgr2=0x7f1e540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e72800640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e72800640 1 -- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e54003970 msgr2=0x7f1e54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e72800640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e54003970 0x7f1e54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e72800640 1 -- 172.21.15.97:0/2947731040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54003520 con 0x7f1e54004d00 2024-01-31T05:04:18.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.259+0000 7f1e69200640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e740ae3b0 0x7f1e540047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:18.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.260+0000 7f1e72800640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54004d00 0x7f1e54005180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c290 tx=0x7f1e540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ffb423091bcba3 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.260+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c005ad0 con 0x7f1e54004d00 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.260+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005c30 con 0x7f1e54004d00 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.260+0000 7f1e71400640 1 -- 172.21.15.97:0/2947731040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c02c8d0 con 0x7f1e54004d00 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e73200640 1 -- 172.21.15.97:0/2947731040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54004d00 msgr2=0x7f1e54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e73200640 1 --2- 172.21.15.97:0/2947731040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54004d00 0x7f1e54005180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c290 tx=0x7f1e540057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e72800640 1 -- 172.21.15.97:0/2947731040 reap_dead start 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e73200640 1 -- 172.21.15.97:0/2947731040 shutdown_connections 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e73200640 1 -- 172.21.15.97:0/2947731040 >> 172.21.15.97:0/2947731040 conn(0x7f1e740cec40 msgr2=0x7f1e740cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e73200640 1 -- 172.21.15.97:0/2947731040 shutdown_connections 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.261+0000 7f1e73200640 1 -- 172.21.15.97:0/2947731040 wait complete. 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.263+0000 7f1e73200640 1 Processor -- start 2024-01-31T05:04:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.263+0000 7f1e73200640 1 -- start start 2024-01-31T05:04:18.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e73200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e54004d00 0x7f1e741f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e73200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e741f6c00 0x7f1e740ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e73200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 0x7f1e740cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e73200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740cde50 con 0x7f1e740cd460 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e73200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e741f7080 con 0x7f1e741f6c00 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e73200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c8350 con 0x7f1e54004d00 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e72800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 0x7f1e740cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e72800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 0x7f1e740cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39774/0 (socket says 172.21.15.97:39774) 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e72800640 1 -- 172.21.15.97:0/2482999006 learned_addr learned my addr 172.21.15.97:0/2482999006 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e72800640 1 -- 172.21.15.97:0/2482999006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e54004d00 msgr2=0x7f1e741f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.264+0000 7f1e72800640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e54004d00 0x7f1e741f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e72800640 1 -- 172.21.15.97:0/2482999006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e741f6c00 msgr2=0x7f1e740ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e72800640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e741f6c00 0x7f1e740ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e72800640 1 -- 172.21.15.97:0/2482999006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740c8630 con 0x7f1e740cd460 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e72800640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 0x7f1e740cd910 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c02c540 tx=0x7f1e6c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c03c040 con 0x7f1e740cd460 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c02cce0 con 0x7f1e740cd460 2024-01-31T05:04:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c0072b0 con 0x7f1e740cd460 2024-01-31T05:04:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e740c8910 con 0x7f1e740cd460 2024-01-31T05:04:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.265+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e740c8ed0 con 0x7f1e740cd460 2024-01-31T05:04:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.266+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e740730c0 con 0x7f1e740cd460 2024-01-31T05:04:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.267+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f1e6c032070 con 0x7f1e740cd460 2024-01-31T05:04:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.268+0000 7f1e6aa00640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e4c06af00 0x7f1e4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.269+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1e6c0aa850 con 0x7f1e740cd460 2024-01-31T05:04:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.269+0000 7f1e71e00640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e4c06af00 0x7f1e4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.269+0000 7f1e71e00640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e4c06af00 0x7f1e4c06d3c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009fa0 tx=0x7f1e64009750 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:18.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.273+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1e740730c0 con 0x7f1e740cd460 2024-01-31T05:04:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.510+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1e740730c0 con 0x7f1e740cd460 2024-01-31T05:04:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.511+0000 7f1e6aa00640 1 -- 172.21.15.97:0/2482999006 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1e740730c0 con 0x7f1e740cd460 2024-01-31T05:04:18.510 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e4c06af00 msgr2=0x7f1e4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e4c06af00 0x7f1e4c06d3c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009fa0 tx=0x7f1e64009750 comp rx=0 tx=0).stop 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 msgr2=0x7f1e740cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 0x7f1e740cd910 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c02c540 tx=0x7f1e6c001a00 comp rx=0 tx=0).stop 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 shutdown_connections 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f1e4c06af00 0x7f1e4c06d3c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e740cd460 0x7f1e740cd910 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e741f6c00 0x7f1e740ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 --2- 172.21.15.97:0/2482999006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e54004d00 0x7f1e741f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 >> 172.21.15.97:0/2482999006 conn(0x7f1e740cec40 msgr2=0x7f1e740ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 shutdown_connections 2024-01-31T05:04:18.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:18.514+0000 7f1e73200640 1 -- 172.21.15.97:0/2482999006 wait complete. 2024-01-31T05:04:18.533 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-31T05:04:18.533 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-31T05:04:18.533 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-01-31T05:04:18.533 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-31T05:04:18.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:18.534 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-31T05:04:18.534 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-31T05:04:18.534 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-31T05:04:18.534 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:04:18.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:18.534 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-01-31T05:04:18.587 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42266sec preferred_lft 42266sec 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-31T05:04:18.588 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::dc28:44ff:fe78:74b/64 scope link 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether aa:ed:50:85:d3:39 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a8ed:50ff:fe85:d339/64 scope link 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether ca:84:a9:c5:b2:19 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c884:a9ff:fec5:b219/64 scope link 2024-01-31T05:04:18.589 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-01-31T05:04:18.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:18.590 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-31T05:04:18.677 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-01-31T05:04:18.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:18.682 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-01-31T05:04:18.734 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-01-31T05:04:18.734 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-31T05:04:18.734 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-31T05:04:18.734 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-31T05:04:18.734 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-31T05:04:18.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:18.735 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2024-01-31T05:04:18.735 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-01-31T05:04:18.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-31T05:04:18.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-31T05:04:18.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-31T05:04:18.736 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-01-31T05:04:18.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:18.914 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:04:18.961 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:04:18.961 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:04:18.962 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:04:18.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:18.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-01-31T05:04:19.054 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-31T05:04:19.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:04:19.054 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:19.110 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-31T05:04:19.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-31T05:04:19.110 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:19.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:04:19.262 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:04:19.315 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:04:19.316 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:04:19.317 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:19.318 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:19.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:19.321 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:04:19.371 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-31T05:04:19.372 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-31T05:04:19.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo modprobe fuse 2024-01-31T05:04:19.453 DEBUG:teuthology.orchestra.run.smithi097:> cat /proc/self/mounts | awk '{print $2}' 2024-01-31T05:04:19.494 INFO:teuthology.orchestra.run.smithi097.stdout:/proc 2024-01-31T05:04:19.494 INFO:teuthology.orchestra.run.smithi097.stdout:/sys 2024-01-31T05:04:19.494 INFO:teuthology.orchestra.run.smithi097.stdout:/dev 2024-01-31T05:04:19.494 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/security 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/shm 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/pts 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/run 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/cgroup 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/pstore 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/ 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/selinux 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/proc/sys/fs/binfmt_misc 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/mqueue 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/dev/hugepages 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/debug 2024-01-31T05:04:19.495 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/tracing 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/fs/fuse/connections 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/sys/kernel/config 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-sysctl.service 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/nfs/rpc_pipefs 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/run/user/1000 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-0 2024-01-31T05:04:19.496 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-1 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-2 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/var/lib/ceph/osd/ceph-3 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:19.497 INFO:teuthology.orchestra.run.smithi097.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:19.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:19.498 DEBUG:teuthology.orchestra.run.smithi097:> ls /sys/fs/fuse/connections 2024-01-31T05:04:19.567 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:2024-01-31T05:04:19.568+0000 7ffbba652480 -1 init, newargv = 0x56164a3c00c0 newargc=15 2024-01-31T05:04:19.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[60347]: starting ceph client 2024-01-31T05:04:19.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[60347]: starting fuse 2024-01-31T05:04:19.584 INFO:teuthology.orchestra.run.smithi097.stdout:40 2024-01-31T05:04:19.584 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> sudo stdin-killer -- python3 -c ' 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> import glob 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> import re 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> import subprocess 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> def _find_admin_socket(client_name): 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> files = glob.glob(asok_path) 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> # Given a non-glob path, it better be there 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> if "*" not in asok_path: 2024-01-31T05:04:19.585 DEBUG:teuthology.orchestra.run.smithi097:> assert(len(files) == 1) 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> return files[0] 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> for f in files: 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> contents = proc_f.read() 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> if mountpoint in contents: 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> return f 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> print(_find_admin_socket("client.0")) 2024-01-31T05:04:19.586 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:04:19.786 INFO:teuthology.orchestra.run.smithi097.stdout:/var/run/ceph/ceph-client.0.60347.asok 2024-01-31T05:04:19.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-31T05:04:19.803 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60347.asok 2024-01-31T05:04:19.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:19.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60347.asok status 2024-01-31T05:04:20.033 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:20.033 INFO:teuthology.orchestra.run.smithi097.stdout: "metadata": { 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_sha1": "661946e7044723130c03627323f6a879a9848cb5", 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "entity_id": "0", 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": "smithi097", 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "pid": "60347", 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "root": "/" 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:20.034 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_count": 0, 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "dentry_pinned_count": 0, 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6579, 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "inst": { 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "name": { 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "client", 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "num": 6579 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 400886276 2024-01-31T05:04:20.035 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": { 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "192.168.0.1:0", 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 400886276 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "inst_str": "client.6579 192.168.0.1:0/400886276", 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "addr_str": "192.168.0.1:0/400886276", 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "inode_count": 1, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_epoch": 27, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch": 55, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_epoch_barrier": 0, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "blocklisted": false, 2024-01-31T05:04:20.036 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs" 2024-01-31T05:04:20.037 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:20.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:20.048 DEBUG:teuthology.orchestra.run.smithi097:> stat --file-system '--printf=%T 2024-01-31T05:04:20.048 DEBUG:teuthology.orchestra.run.smithi097:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:20.101 INFO:teuthology.orchestra.run.smithi097.stdout:fuseblk 2024-01-31T05:04:20.101 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:20.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:20.101 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:20.192 DEBUG:teuthology.orchestra.run.smithi097:> stdin-killer -- python3 -c ' 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> import os 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> import errno 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> try: 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> if True: 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> ret = os.read(fd, 1024) 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> else: 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> os.write(fd, b'"'"'content'"'"') 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> except OSError as e: 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> if e.errno != errno.EPERM: 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> raise 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> else: 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> raise RuntimeError("client does not check permission of data pool") 2024-01-31T05:04:20.193 DEBUG:teuthology.orchestra.run.smithi097:> ' 2024-01-31T05:04:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:Traceback (most recent call last): 2024-01-31T05:04:20.337 INFO:teuthology.orchestra.run.smithi097.stderr: File "", line 15, in 2024-01-31T05:04:20.337 INFO:teuthology.orchestra.run.smithi097.stderr:RuntimeError: client does not check permission of data pool 2024-01-31T05:04:20.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2024-01-31T05:04:20.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:20.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-01-31T05:04:20.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.614+0000 7f347d400640 1 Processor -- start 2024-01-31T05:04:20.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.614+0000 7f347d400640 1 -- start start 2024-01-31T05:04:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.616+0000 7f347d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f347815d450 0x7f34780cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.616+0000 7f347d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 0x7f34780cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.616+0000 7f347d400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780d2190 con 0x7f34780d1680 2024-01-31T05:04:20.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.616+0000 7f347d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34780b3060 con 0x7f347815d450 2024-01-31T05:04:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.616+0000 7f347d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780b2a30 con 0x7f34780d39f0 2024-01-31T05:04:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.616+0000 7f3477400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 0x7f34780cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 0x7f34780cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59452/0 (socket says 172.21.15.97:59452) 2024-01-31T05:04:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477400640 1 -- 172.21.15.97:0/2078068671 learned_addr learned my addr 172.21.15.97:0/2078068671 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f347ca00640 1 --1- 172.21.15.97:0/2078068671 >> v1:172.21.15.188:6789/0 conn(0x7f34780d1680 0x7f34780cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:44796/0 (socket says 172.21.15.97:44796) 2024-01-31T05:04:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477e00640 1 --2- 172.21.15.97:0/2078068671 >> v2:172.21.15.188:3300/0 conn(0x7f347815d450 0x7f34780cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477400640 1 -- 172.21.15.97:0/2078068671 >> v2:172.21.15.188:3300/0 conn(0x7f347815d450 msgr2=0x7f34780cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477400640 1 --2- 172.21.15.97:0/2078068671 >> v2:172.21.15.188:3300/0 conn(0x7f347815d450 0x7f34780cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477400640 1 -- 172.21.15.97:0/2078068671 >> v1:172.21.15.188:6789/0 conn(0x7f34780d1680 legacy=0x7f34780cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.617+0000 7f3477400640 1 -- 172.21.15.97:0/2078068671 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780d1c30 con 0x7f34780d39f0 2024-01-31T05:04:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.619+0000 7f3477400640 1 --2- 172.21.15.97:0/2078068671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 0x7f34780cfe70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3460009870 tx=0x7f346002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46d5be0fda46e7c server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.620+0000 7f3476a00640 1 -- 172.21.15.97:0/2078068671 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460030840 con 0x7f34780d39f0 2024-01-31T05:04:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.620+0000 7f3476a00640 1 -- 172.21.15.97:0/2078068671 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34600309a0 con 0x7f34780d39f0 2024-01-31T05:04:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.620+0000 7f3476a00640 1 -- 172.21.15.97:0/2078068671 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460030d30 con 0x7f34780d39f0 2024-01-31T05:04:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 -- 172.21.15.97:0/2078068671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 msgr2=0x7f34780cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 --2- 172.21.15.97:0/2078068671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 0x7f34780cfe70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3460009870 tx=0x7f346002b980 comp rx=0 tx=0).stop 2024-01-31T05:04:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 -- 172.21.15.97:0/2078068671 shutdown_connections 2024-01-31T05:04:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 --2- 172.21.15.97:0/2078068671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d39f0 0x7f34780cfe70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 --2- 172.21.15.97:0/2078068671 >> v2:172.21.15.188:3300/0 conn(0x7f347815d450 0x7f34780cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 -- 172.21.15.97:0/2078068671 >> 172.21.15.97:0/2078068671 conn(0x7f347809c400 msgr2=0x7f34780d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 -- 172.21.15.97:0/2078068671 shutdown_connections 2024-01-31T05:04:20.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.621+0000 7f347d400640 1 -- 172.21.15.97:0/2078068671 wait complete. 2024-01-31T05:04:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.623+0000 7f347d400640 1 Processor -- start 2024-01-31T05:04:20.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.623+0000 7f347d400640 1 -- start start 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d1680 0x7f34780a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347d400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 0x7f34780ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f347815d450 0x7f34781f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347d400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781f36b0 con 0x7f34780d39f0 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780b5c60 con 0x7f347815d450 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780ca830 con 0x7f34780d1680 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 0x7f34780ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:20.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 0x7f34780ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39786/0 (socket says 172.21.15.97:39786) 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 -- 172.21.15.97:0/3013233653 learned_addr learned my addr 172.21.15.97:0/3013233653 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477e00640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d1680 0x7f34780a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347ca00640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f347815d450 0x7f34781f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 -- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d1680 msgr2=0x7f34780a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d1680 0x7f34780a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 -- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f347815d450 msgr2=0x7f34781f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f347815d450 0x7f34781f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f3477400640 1 -- 172.21.15.97:0/3013233653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600094d0 con 0x7f34780d39f0 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.625+0000 7f347ca00640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f347815d450 0x7f34781f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:20.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.626+0000 7f3477400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 0x7f34780ca1a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3460000c00 tx=0x7f3460004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.626+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460039c90 con 0x7f34780d39f0 2024-01-31T05:04:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.626+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460039df0 con 0x7f34780d39f0 2024-01-31T05:04:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.626+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34780cab10 con 0x7f34780d39f0 2024-01-31T05:04:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.626+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460043640 con 0x7f34780d39f0 2024-01-31T05:04:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.626+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34780cb0d0 con 0x7f34780d39f0 2024-01-31T05:04:20.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.628+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34780cb260 con 0x7f34780d39f0 2024-01-31T05:04:20.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.628+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f34600437a0 con 0x7f34780d39f0 2024-01-31T05:04:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.630+0000 7f346fe00640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f345006af00 0x7f345006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.630+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f346003e070 con 0x7f34780d39f0 2024-01-31T05:04:20.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.630+0000 7f3477e00640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f345006af00 0x7f345006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:20.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.631+0000 7f3477e00640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f345006af00 0x7f345006d3c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3468009a30 tx=0x7f3468008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:20.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.634+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3460077530 con 0x7f34780d39f0 2024-01-31T05:04:20.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.899+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f34780676b0 con 0x7f34780d39f0 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.902+0000 7f346fe00640 1 -- 172.21.15.97:0/3013233653 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+141 (secure 0 0 0) 0x7f346007ab50 con 0x7f34780d39f0 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDC07llepy9KxAAwqU0F1ptrkjfsIP8gRjC4A== 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw pool=cephfs_data" 2024-01-31T05:04:20.902 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f345006af00 msgr2=0x7f345006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f345006af00 0x7f345006d3c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3468009a30 tx=0x7f3468008040 comp rx=0 tx=0).stop 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 msgr2=0x7f34780ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 0x7f34780ca1a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3460000c00 tx=0x7f3460004730 comp rx=0 tx=0).stop 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 shutdown_connections 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f347815d450 0x7f34781f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f345006af00 0x7f345006d3c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34780d39f0 0x7f34780ca1a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.905+0000 7f347d400640 1 --2- 172.21.15.97:0/3013233653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34780d1680 0x7f34780a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.906+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 >> 172.21.15.97:0/3013233653 conn(0x7f347809c400 msgr2=0x7f347806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.906+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 shutdown_connections 2024-01-31T05:04:20.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:20.906+0000 7f347d400640 1 -- 172.21.15.97:0/3013233653 wait complete. 2024-01-31T05:04:20.924 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-31T05:04:20.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:04:21.029 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-31T05:04:21.146 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-31T05:04:21.146 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:04:21.146 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:21.203 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:21.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.422+0000 7ff735e00640 1 Processor -- start 2024-01-31T05:04:21.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.422+0000 7ff735e00640 1 -- start start 2024-01-31T05:04:21.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff735e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7300cb810 0x7ff7300cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff735e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73006d1a0 con 0x7ff7300ccd00 2024-01-31T05:04:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff735e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73006c600 con 0x7ff73015b400 2024-01-31T05:04:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff735e00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7300cc960 con 0x7ff7300cb810 2024-01-31T05:04:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff72ea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7300cb810 0x7ff7300cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff72f400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff7300ccd00 0x7ff7300cd170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39684/0 (socket says 172.21.15.97:39684) 2024-01-31T05:04:21.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.424+0000 7ff72f400640 1 -- 172.21.15.97:0/2243667622 learned_addr learned my addr 172.21.15.97:0/2243667622 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:21.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.425+0000 7ff72e000640 1 -- 172.21.15.97:0/2243667622 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879039770 0 0) 0x7ff73006d1a0 con 0x7ff7300ccd00 2024-01-31T05:04:21.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.425+0000 7ff72ea00640 1 -- 172.21.15.97:0/2243667622 >> v1:172.21.15.188:6789/0 conn(0x7ff73015b400 legacy=0x7ff73006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.425+0000 7ff72ea00640 1 -- 172.21.15.97:0/2243667622 >> v1:172.21.15.97:6789/0 conn(0x7ff7300ccd00 legacy=0x7ff7300cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.425+0000 7ff72ea00640 1 -- 172.21.15.97:0/2243667622 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300cc130 con 0x7ff7300cb810 2024-01-31T05:04:21.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.426+0000 7ff72ea00640 1 --2- 172.21.15.97:0/2243667622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7300cb810 0x7ff7300cbbf0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff7200097a0 tx=0x7ff72002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301027c029037219 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:21.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.426+0000 7ff72e000640 1 -- 172.21.15.97:0/2243667622 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720030610 con 0x7ff7300cb810 2024-01-31T05:04:21.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.426+0000 7ff72e000640 1 -- 172.21.15.97:0/2243667622 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720030770 con 0x7ff7300cb810 2024-01-31T05:04:21.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.426+0000 7ff72e000640 1 -- 172.21.15.97:0/2243667622 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720030b00 con 0x7ff7300cb810 2024-01-31T05:04:21.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 -- 172.21.15.97:0/2243667622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7300cb810 msgr2=0x7ff7300cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 --2- 172.21.15.97:0/2243667622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7300cb810 0x7ff7300cbbf0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff7200097a0 tx=0x7ff72002b960 comp rx=0 tx=0).stop 2024-01-31T05:04:21.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 -- 172.21.15.97:0/2243667622 shutdown_connections 2024-01-31T05:04:21.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 --2- 172.21.15.97:0/2243667622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7300cb810 0x7ff7300cbbf0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 -- 172.21.15.97:0/2243667622 >> 172.21.15.97:0/2243667622 conn(0x7ff7300b0a10 msgr2=0x7ff7300b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:21.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 -- 172.21.15.97:0/2243667622 shutdown_connections 2024-01-31T05:04:21.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.427+0000 7ff735e00640 1 -- 172.21.15.97:0/2243667622 wait complete. 2024-01-31T05:04:21.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.428+0000 7ff735e00640 1 Processor -- start 2024-01-31T05:04:21.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.429+0000 7ff735e00640 1 -- start start 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff735e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300cb810 0x7ff7301528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff735e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7300ccd00 0x7ff7300999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff735e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 0x7ff73009bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff735e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff730152e00 con 0x7ff73015b400 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff735e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73009a4e0 con 0x7ff7300ccd00 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff735e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73009a7c0 con 0x7ff7300cb810 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 0x7ff73009bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 0x7ff73009bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39806/0 (socket says 172.21.15.97:39806) 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 -- 172.21.15.97:0/3249147488 learned_addr learned my addr 172.21.15.97:0/3249147488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff727e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7300ccd00 0x7ff7300999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72ea00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300cb810 0x7ff7301528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 -- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300cb810 msgr2=0x7ff7301528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300cb810 0x7ff7301528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 -- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7300ccd00 msgr2=0x7ff7300999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7300ccd00 0x7ff7300999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72f400640 1 -- 172.21.15.97:0/3249147488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009400 con 0x7ff73015b400 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff727e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7300ccd00 0x7ff7300999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.430+0000 7ff72ea00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300cb810 0x7ff7301528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.431+0000 7ff72f400640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 0x7ff73009bfd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c7e0 tx=0x7ff72800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.431+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800a640 con 0x7ff73015b400 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.431+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72800a7c0 con 0x7ff73015b400 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.431+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73015cec0 con 0x7ff73015b400 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.431+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800ab70 con 0x7ff73015b400 2024-01-31T05:04:21.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.431+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73009aaa0 con 0x7ff73015b400 2024-01-31T05:04:21.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.432+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7300677c0 con 0x7ff73015b400 2024-01-31T05:04:21.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.433+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff72800acd0 con 0x7ff73015b400 2024-01-31T05:04:21.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.434+0000 7ff727400640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff70806aed0 0x7ff70806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.434+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff728088fe0 con 0x7ff73015b400 2024-01-31T05:04:21.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.434+0000 7ff72ea00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff70806aed0 0x7ff70806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.435+0000 7ff72ea00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff70806aed0 0x7ff70806d390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff720036000 tx=0x7ff72003b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:21.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.438+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff728053d80 con 0x7ff73015b400 2024-01-31T05:04:21.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.633+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff73006d9b0 con 0x7ff73015b400 2024-01-31T05:04:21.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.635+0000 7ff727400640 1 -- 172.21.15.97:0/3249147488 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7ff728018020 con 0x7ff73015b400 2024-01-31T05:04:21.635 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:21.635 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:21.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.638+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff70806aed0 msgr2=0x7ff70806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.638+0000 7ff735e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff70806aed0 0x7ff70806d390 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff720036000 tx=0x7ff72003b040 comp rx=0 tx=0).stop 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.638+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 msgr2=0x7ff73009bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.638+0000 7ff735e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 0x7ff73009bfd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff72800c7e0 tx=0x7ff72800cca0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 shutdown_connections 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff70806aed0 0x7ff70806d390 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff73015b400 0x7ff73009bfd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7300ccd00 0x7ff7300999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 --2- 172.21.15.97:0/3249147488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7300cb810 0x7ff7301528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 >> 172.21.15.97:0/3249147488 conn(0x7ff7300b0a10 msgr2=0x7ff73015f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 shutdown_connections 2024-01-31T05:04:21.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.639+0000 7ff735e00640 1 -- 172.21.15.97:0/3249147488 wait complete. 2024-01-31T05:04:21.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:03:36.506793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:03:37.685194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:04:21.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:04:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.868+0000 7f0b03e00640 1 Processor -- start 2024-01-31T05:04:21.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.868+0000 7f0b03e00640 1 -- start start 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0b0406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b03e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b040b2320 0x7f0b0406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b03e00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b04070fa0 con 0x7f0b04161770 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b03e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b04070d90 con 0x7f0b040b2320 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b03e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b04071280 con 0x7f0b040ae480 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b02a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0b04161770 0x7f0b040b1c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39694/0 (socket says 172.21.15.97:39694) 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b02a00640 1 -- 172.21.15.97:0/3321545753 learned_addr learned my addr 172.21.15.97:0/3321545753 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891659646 0 0) 0x7f0b04070fa0 con 0x7f0b04161770 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b03400640 1 --2- 172.21.15.97:0/3321545753 >> v2:172.21.15.188:3300/0 conn(0x7f0b040b2320 0x7f0b0406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae4003520 con 0x7f0b04161770 2024-01-31T05:04:21.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.870+0000 7f0afbe00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0b0406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2784756777 0 0) 0x7f0ae4003520 con 0x7f0b04161770 2024-01-31T05:04:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 msgr2=0x7f0b0406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0b0406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 >> v2:172.21.15.188:3300/0 conn(0x7f0b040b2320 msgr2=0x7f0b0406cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 --2- 172.21.15.97:0/3321545753 >> v2:172.21.15.188:3300/0 conn(0x7f0b040b2320 0x7f0b0406cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04071560 con 0x7f0b04161770 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f0af4001670 con 0x7f0b04161770 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 >> v1:172.21.15.97:6789/0 conn(0x7f0b04161770 legacy=0x7f0b040b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0ae4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0ae40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ae4004d00 0x7f0ae4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b04070d90 con 0x7f0ae40039e0 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b04071280 con 0x7f0ae4004d00 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae400bae0 con 0x7f0b040ae480 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.871+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0ae4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0afbe00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0ae40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0b02a00640 1 -- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 msgr2=0x7f0ae40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0ae40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0b02a00640 1 -- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ae4004d00 msgr2=0x7f0ae4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ae4004d00 0x7f0ae4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0b02a00640 1 -- 172.21.15.97:0/3321545753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4003520 con 0x7f0ae40039e0 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.872+0000 7f0afbe00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b040ae480 0x7f0ae40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.873+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0ae4003e00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0af400c0e0 tx=0x7f0ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2823a48ed7832929 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.873+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4005ad0 con 0x7f0ae40039e0 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.873+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af4005c50 con 0x7f0ae40039e0 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.873+0000 7f0b02000640 1 -- 172.21.15.97:0/3321545753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af402c810 con 0x7f0ae40039e0 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03e00640 1 -- 172.21.15.97:0/3321545753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 msgr2=0x7f0ae4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3321545753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0ae4003e00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0af400c0e0 tx=0x7f0ae40057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03400640 1 -- 172.21.15.97:0/3321545753 reap_dead start 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03e00640 1 -- 172.21.15.97:0/3321545753 shutdown_connections 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03e00640 1 -- 172.21.15.97:0/3321545753 >> 172.21.15.97:0/3321545753 conn(0x7f0b0406ad40 msgr2=0x7f0b040b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03e00640 1 -- 172.21.15.97:0/3321545753 shutdown_connections 2024-01-31T05:04:21.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.874+0000 7f0b03e00640 1 -- 172.21.15.97:0/3321545753 wait complete. 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.876+0000 7f0b03e00640 1 Processor -- start 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.876+0000 7f0b03e00640 1 -- start start 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.876+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0b0409b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b041646c0 0x7f0b041f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b03e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b041f7570 0x7f0b041f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b03e00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0409c3c0 con 0x7f0ae40039e0 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b03e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041f7f60 con 0x7f0b041646c0 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b03e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b04164b40 con 0x7f0b041f7570 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0b0409b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0b0409b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39828/0 (socket says 172.21.15.97:39828) 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 -- 172.21.15.97:0/3370369741 learned_addr learned my addr 172.21.15.97:0/3370369741 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:21.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 -- 172.21.15.97:0/3370369741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b041f7570 msgr2=0x7f0b041f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b041f7570 0x7f0b041f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 -- 172.21.15.97:0/3370369741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b041646c0 msgr2=0x7f0b041f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b041646c0 0x7f0b041f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.877+0000 7f0b02a00640 1 -- 172.21.15.97:0/3370369741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0409bb20 con 0x7f0ae40039e0 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.878+0000 7f0b02a00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0b0409b420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0af402c480 tx=0x7f0af4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.878+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af402cc20 con 0x7f0ae40039e0 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.878+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af402cd80 con 0x7f0ae40039e0 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.878+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0409be00 con 0x7f0ae40039e0 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.878+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40072b0 con 0x7f0ae40039e0 2024-01-31T05:04:21.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.878+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b04161cf0 con 0x7f0ae40039e0 2024-01-31T05:04:21.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.880+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b04161b50 con 0x7f0ae40039e0 2024-01-31T05:04:21.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.880+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f0af4032070 con 0x7f0ae40039e0 2024-01-31T05:04:21.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.881+0000 7f0afaa00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0adc06aed0 0x7f0adc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:21.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.881+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0af403c040 con 0x7f0ae40039e0 2024-01-31T05:04:21.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.882+0000 7f0afbe00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0adc06aed0 0x7f0adc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:21.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.882+0000 7f0afbe00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0adc06aed0 0x7f0adc06d390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0b0409c520 tx=0x7f0aec009350 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:21.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:21.887+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f0b04161b50 con 0x7f0ae40039e0 2024-01-31T05:04:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.082+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b04161b50 con 0x7f0ae40039e0 2024-01-31T05:04:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.083+0000 7f0afaa00640 1 -- 172.21.15.97:0/3370369741 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b04161b50 con 0x7f0ae40039e0 2024-01-31T05:04:22.083 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:04:22.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0adc06aed0 msgr2=0x7f0adc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0adc06aed0 0x7f0adc06d390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0b0409c520 tx=0x7f0aec009350 comp rx=0 tx=0).stop 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 msgr2=0x7f0b0409b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0b0409b420 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0af402c480 tx=0x7f0af4001930 comp rx=0 tx=0).stop 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 shutdown_connections 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b041f7570 0x7f0b041f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b041646c0 0x7f0b041f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f0adc06aed0 0x7f0adc06d390 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 --2- 172.21.15.97:0/3370369741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ae40039e0 0x7f0b0409b420 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 >> 172.21.15.97:0/3370369741 conn(0x7f0b0406ad40 msgr2=0x7f0b0415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 shutdown_connections 2024-01-31T05:04:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.086+0000 7f0b03e00640 1 -- 172.21.15.97:0/3370369741 wait complete. 2024-01-31T05:04:22.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.317+0000 7f602d600640 1 Processor -- start 2024-01-31T05:04:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.317+0000 7f602d600640 1 -- start start 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f602d600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60280d0040 0x7f60280b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f602d600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60280d2190 con 0x7f60280d39f0 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f602d600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60280b2a30 con 0x7f602815d450 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f602d600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60280b3060 con 0x7f60280d0040 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f6027400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f60280d0040 0x7f60280b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f602cc00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f60280d39f0 0x7f60280b39c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39704/0 (socket says 172.21.15.97:39704) 2024-01-31T05:04:22.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.319+0000 7f602cc00640 1 -- 172.21.15.97:0/1878515321 learned_addr learned my addr 172.21.15.97:0/1878515321 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.320+0000 7f6027400640 1 -- 172.21.15.97:0/1878515321 >> v1:172.21.15.188:6789/0 conn(0x7f602815d450 legacy=0x7f60280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.320+0000 7f6027400640 1 -- 172.21.15.97:0/1878515321 >> v1:172.21.15.97:6789/0 conn(0x7f60280d39f0 legacy=0x7f60280b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.320+0000 7f6027400640 1 -- 172.21.15.97:0/1878515321 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280d1c30 con 0x7f60280d0040 2024-01-31T05:04:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.320+0000 7f6026a00640 1 -- 172.21.15.97:0/1878515321 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615466632 0 0) 0x7f60280d2190 con 0x7f60280d39f0 2024-01-31T05:04:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.321+0000 7f6027400640 1 --2- 172.21.15.97:0/1878515321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d0040 0x7f60280b3480 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6018009d90 tx=0x7f601802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4569da7f3ba5754 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.321+0000 7f6026a00640 1 -- 172.21.15.97:0/1878515321 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601802cd00 con 0x7f60280d0040 2024-01-31T05:04:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.321+0000 7f6026a00640 1 -- 172.21.15.97:0/1878515321 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601802ce60 con 0x7f60280d0040 2024-01-31T05:04:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.321+0000 7f6026a00640 1 -- 172.21.15.97:0/1878515321 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018034660 con 0x7f60280d0040 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 -- 172.21.15.97:0/1878515321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d0040 msgr2=0x7f60280b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 --2- 172.21.15.97:0/1878515321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d0040 0x7f60280b3480 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6018009d90 tx=0x7f601802bf50 comp rx=0 tx=0).stop 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 -- 172.21.15.97:0/1878515321 shutdown_connections 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 --2- 172.21.15.97:0/1878515321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d0040 0x7f60280b3480 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 -- 172.21.15.97:0/1878515321 >> 172.21.15.97:0/1878515321 conn(0x7f602809c400 msgr2=0x7f60280d30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 -- 172.21.15.97:0/1878515321 shutdown_connections 2024-01-31T05:04:22.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.322+0000 7f602d600640 1 -- 172.21.15.97:0/1878515321 wait complete. 2024-01-31T05:04:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.324+0000 7f602d600640 1 Processor -- start 2024-01-31T05:04:22.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.324+0000 7f602d600640 1 -- start start 2024-01-31T05:04:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f602d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60280d0040 0x7f6028155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f602d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 0x7f60280cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f602d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602815d450 0x7f60280cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f602d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6028155b40 con 0x7f60280d39f0 2024-01-31T05:04:22.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f602d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280cc270 con 0x7f60280d0040 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f602d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280cbce0 con 0x7f602815d450 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f6027400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 0x7f60280cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f6027400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 0x7f60280cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39840/0 (socket says 172.21.15.97:39840) 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f6027400640 1 -- 172.21.15.97:0/1705180960 learned_addr learned my addr 172.21.15.97:0/1705180960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.325+0000 7f6027e00640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60280d0040 0x7f6028155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027400640 1 -- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602815d450 msgr2=0x7f60280cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027400640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602815d450 0x7f60280cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027400640 1 -- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60280d0040 msgr2=0x7f6028155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027400640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60280d0040 0x7f6028155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027400640 1 -- 172.21.15.97:0/1705180960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018009a70 con 0x7f60280d39f0 2024-01-31T05:04:22.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027e00640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60280d0040 0x7f6028155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:22.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f6027400640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 0x7f60280cb7a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6018037040 tx=0x7f6018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601803d600 con 0x7f60280d39f0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601803d780 con 0x7f60280d39f0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601803db30 con 0x7f60280d39f0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60280cbf90 con 0x7f60280d39f0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.326+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60280ca5b0 con 0x7f60280d39f0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.328+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028067dc0 con 0x7f60280d39f0 2024-01-31T05:04:22.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.328+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f601803b040 con 0x7f60280d39f0 2024-01-31T05:04:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.330+0000 7f601fe00640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f600006af00 0x7f600006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.330+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f60180ab470 con 0x7f60280d39f0 2024-01-31T05:04:22.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.330+0000 7f6027e00640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f600006af00 0x7f600006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.331+0000 7f6027e00640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f600006af00 0x7f600006d3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6010000f60 tx=0x7f6010009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:22.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.334+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f6018076200 con 0x7f60280d39f0 2024-01-31T05:04:22.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.527+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60280ceec0 con 0x7f60280d39f0 2024-01-31T05:04:22.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.528+0000 7f601fe00640 1 -- 172.21.15.97:0/1705180960 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6018079820 con 0x7f60280d39f0 2024-01-31T05:04:22.527 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:04:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.530+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f600006af00 msgr2=0x7f600006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.530+0000 7f602d600640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f600006af00 0x7f600006d3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6010000f60 tx=0x7f6010009040 comp rx=0 tx=0).stop 2024-01-31T05:04:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.530+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 msgr2=0x7f60280cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.530+0000 7f602d600640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 0x7f60280cb7a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6018037040 tx=0x7f6018004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:22.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 shutdown_connections 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f602815d450 0x7f60280cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f600006af00 0x7f600006d3c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60280d39f0 0x7f60280cb7a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 --2- 172.21.15.97:0/1705180960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60280d0040 0x7f6028155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 >> 172.21.15.97:0/1705180960 conn(0x7f602809c400 msgr2=0x7f602806bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 shutdown_connections 2024-01-31T05:04:22.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.531+0000 7f602d600640 1 -- 172.21.15.97:0/1705180960 wait complete. 2024-01-31T05:04:22.550 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:04:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.766+0000 7f5787400640 1 Processor -- start 2024-01-31T05:04:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.766+0000 7f5787400640 1 -- start start 2024-01-31T05:04:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.767+0000 7f5787400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57801663a0 0x7f5780168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f5787400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57800ceb40 con 0x7f578015d690 2024-01-31T05:04:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f5787400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578015db40 con 0x7f578006d580 2024-01-31T05:04:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f5787400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578015e7c0 con 0x7f57801663a0 2024-01-31T05:04:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f5777c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f578015d690 0x7f57800ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39708/0 (socket says 172.21.15.97:39708) 2024-01-31T05:04:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f5777c00640 1 -- 172.21.15.97:0/3438072139 learned_addr learned my addr 172.21.15.97:0/3438072139 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f5785400640 1 --2- 172.21.15.97:0/3438072139 >> v2:172.21.15.188:3300/0 conn(0x7f57801663a0 0x7f5780168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566520510 0 0) 0x7f57800ceb40 con 0x7f578015d690 2024-01-31T05:04:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.768+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5760003520 con 0x7f578015d690 2024-01-31T05:04:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640348072 0 0) 0x7f5760003520 con 0x7f578015d690 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 >> v2:172.21.15.188:3300/0 conn(0x7f57801663a0 msgr2=0x7f5780168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 --2- 172.21.15.97:0/3438072139 >> v2:172.21.15.188:3300/0 conn(0x7f57801663a0 0x7f5780168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 >> v1:172.21.15.188:6789/0 conn(0x7f578006d580 legacy=0x7f578006d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578015de20 con 0x7f578015d690 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f5768001670 con 0x7f578015d690 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 >> v1:172.21.15.97:6789/0 conn(0x7f578015d690 legacy=0x7f57800ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5760003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57600047c0 0x7f5760004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5760005180 0x7f576000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578015e7c0 con 0x7f5760003970 2024-01-31T05:04:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576000c020 con 0x7f5760005180 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576000c1e0 con 0x7f57600047c0 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.769+0000 7f5784a00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5760003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5785400640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5760005180 0x7f576000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5784a00640 1 -- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57600047c0 msgr2=0x7f5760004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5784a00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57600047c0 0x7f5760004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5784a00640 1 -- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5760005180 msgr2=0x7f576000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5784a00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5760005180 0x7f576000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5784a00640 1 -- 172.21.15.97:0/3438072139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760003520 con 0x7f5760003970 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5785400640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5760005180 0x7f576000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.770+0000 7f5785400640 1 -- 172.21.15.97:0/3438072139 reap_dead start 2024-01-31T05:04:22.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.771+0000 7f5784a00640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5760003d90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5770001770 tx=0x7f57600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c0ee3a21ad89ed server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.771+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700052d0 con 0x7f5760003970 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.771+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577002e050 con 0x7f5760003970 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.771+0000 7f577fe00640 1 -- 172.21.15.97:0/3438072139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770003640 con 0x7f5760003970 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.771+0000 7f5787400640 1 -- 172.21.15.97:0/3438072139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 msgr2=0x7f5760003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.771+0000 7f5787400640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5760003d90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5770001770 tx=0x7f57600057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.772+0000 7f5787400640 1 -- 172.21.15.97:0/3438072139 shutdown_connections 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.772+0000 7f5787400640 1 --2- 172.21.15.97:0/3438072139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5760003d90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.772+0000 7f5787400640 1 -- 172.21.15.97:0/3438072139 >> 172.21.15.97:0/3438072139 conn(0x7f57800b0a10 msgr2=0x7f57800b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.772+0000 7f5787400640 1 -- 172.21.15.97:0/3438072139 shutdown_connections 2024-01-31T05:04:22.774 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.772+0000 7f5787400640 1 -- 172.21.15.97:0/3438072139 wait complete. 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.773+0000 7f5787400640 1 Processor -- start 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.774+0000 7f5787400640 1 -- start start 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.774+0000 7f5787400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5780152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.774+0000 7f5787400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f578014d640 0x7f5780152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5787400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57800abf90 0x7f57800ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5787400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57800ad960 con 0x7f5760003970 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5787400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57800acce0 con 0x7f57800abf90 2024-01-31T05:04:22.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5787400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57800acfc0 con 0x7f578014d640 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5780152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5780152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39854/0 (socket says 172.21.15.97:39854) 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 -- 172.21.15.97:0/588452365 learned_addr learned my addr 172.21.15.97:0/588452365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 -- 172.21.15.97:0/588452365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f578014d640 msgr2=0x7f5780152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f578014d640 0x7f5780152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 -- 172.21.15.97:0/588452365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57800abf90 msgr2=0x7f57800ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57800abf90 0x7f57800ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 -- 172.21.15.97:0/588452365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57800ad2a0 con 0x7f5760003970 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.775+0000 7f5784a00640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5780152680 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5770034040 tx=0x7f5770001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:22.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.776+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770007a80 con 0x7f5760003970 2024-01-31T05:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.776+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770006e20 con 0x7f5760003970 2024-01-31T05:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.776+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770007470 con 0x7f5760003970 2024-01-31T05:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.776+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57800ad580 con 0x7f5760003970 2024-01-31T05:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.776+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5780154bb0 con 0x7f5760003970 2024-01-31T05:04:22.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.777+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57800ad710 con 0x7f5760003970 2024-01-31T05:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.778+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f57700075d0 con 0x7f5760003970 2024-01-31T05:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.779+0000 7f577d600640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f575806af00 0x7f575806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.780+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f57700ab3b0 con 0x7f5760003970 2024-01-31T05:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.780+0000 7f5777c00640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f575806af00 0x7f575806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:22.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.780+0000 7f5777c00640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f575806af00 0x7f575806d3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5768000fd0 tx=0x7f57680014b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:22.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.784+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f5770076140 con 0x7f5760003970 2024-01-31T05:04:22.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.979+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5780067830 con 0x7f5760003970 2024-01-31T05:04:22.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.980+0000 7f577d600640 1 -- 172.21.15.97:0/588452365 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5770079760 con 0x7f5760003970 2024-01-31T05:04:22.980 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:04:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.982+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f575806af00 msgr2=0x7f575806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.982+0000 7f5787400640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f575806af00 0x7f575806d3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5768000fd0 tx=0x7f57680014b0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.982+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 msgr2=0x7f5780152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.982+0000 7f5787400640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5780152680 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5770034040 tx=0x7f5770001170 comp rx=0 tx=0).stop 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 shutdown_connections 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57800abf90 0x7f57800ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f578014d640 0x7f5780152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f575806af00 0x7f575806d3c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 --2- 172.21.15.97:0/588452365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5760003970 0x7f5780152680 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 >> 172.21.15.97:0/588452365 conn(0x7f57800b0a10 msgr2=0x7f57800cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 shutdown_connections 2024-01-31T05:04:22.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:22.983+0000 7f5787400640 1 -- 172.21.15.97:0/588452365 wait complete. 2024-01-31T05:04:23.002 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-31T05:04:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.216+0000 7f1333400640 1 Processor -- start 2024-01-31T05:04:23.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.216+0000 7f1333400640 1 -- start start 2024-01-31T05:04:23.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1333400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1333400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f132c15a730 0x7f132c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1333400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f132c06dfc0 0x7f132c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1333400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132c0b2c90 con 0x7f132c06dfc0 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1333400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f132c15abc0 con 0x7f132c15a730 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1333400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c15aea0 con 0x7f132c15a350 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1331400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f132c06dfc0 0x7f132c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1331400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f132c06dfc0 0x7f132c0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39862/0 (socket says 172.21.15.97:39862) 2024-01-31T05:04:23.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.218+0000 7f1331400640 1 -- 172.21.15.97:0/663793507 learned_addr learned my addr 172.21.15.97:0/663793507 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f132be00640 1 --2- 172.21.15.97:0/663793507 >> v2:172.21.15.188:3300/0 conn(0x7f132c15a730 0x7f132c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f1330a00640 1 --2- 172.21.15.97:0/663793507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f1331400640 1 -- 172.21.15.97:0/663793507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 msgr2=0x7f132c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f1331400640 1 --2- 172.21.15.97:0/663793507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f1331400640 1 -- 172.21.15.97:0/663793507 >> v2:172.21.15.188:3300/0 conn(0x7f132c15a730 msgr2=0x7f132c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:23.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f1331400640 1 --2- 172.21.15.97:0/663793507 >> v2:172.21.15.188:3300/0 conn(0x7f132c15a730 0x7f132c06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.219+0000 7f1331400640 1 -- 172.21.15.97:0/663793507 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c15b180 con 0x7f132c06dfc0 2024-01-31T05:04:23.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.220+0000 7f1331400640 1 --2- 172.21.15.97:0/663793507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c06dfc0 0x7f132c0cde10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f132c072e90 tx=0x7f132402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48bec7cca7a201d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.220+0000 7f132b400640 1 -- 172.21.15.97:0/663793507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132402ca80 con 0x7f132c06dfc0 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.220+0000 7f132b400640 1 -- 172.21.15.97:0/663793507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132402cbe0 con 0x7f132c06dfc0 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.220+0000 7f132b400640 1 -- 172.21.15.97:0/663793507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240343f0 con 0x7f132c06dfc0 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 -- 172.21.15.97:0/663793507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c06dfc0 msgr2=0x7f132c0cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 --2- 172.21.15.97:0/663793507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c06dfc0 0x7f132c0cde10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f132c072e90 tx=0x7f132402bee0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 -- 172.21.15.97:0/663793507 shutdown_connections 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 --2- 172.21.15.97:0/663793507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c06dfc0 0x7f132c0cde10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 --2- 172.21.15.97:0/663793507 >> v2:172.21.15.188:3300/0 conn(0x7f132c15a730 0x7f132c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 --2- 172.21.15.97:0/663793507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 -- 172.21.15.97:0/663793507 >> 172.21.15.97:0/663793507 conn(0x7f132c0b0a10 msgr2=0x7f132c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 -- 172.21.15.97:0/663793507 shutdown_connections 2024-01-31T05:04:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.221+0000 7f1333400640 1 -- 172.21.15.97:0/663793507 wait complete. 2024-01-31T05:04:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.223+0000 7f1333400640 1 Processor -- start 2024-01-31T05:04:23.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.223+0000 7f1333400640 1 -- start start 2024-01-31T05:04:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1333400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f132c06dfc0 0x7f132c15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1333400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1333400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 0x7f132c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1333400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c0cb100 con 0x7f132c14bea0 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1333400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c0cae20 con 0x7f132c06dfc0 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1333400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c15ba60 con 0x7f132c15a350 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1331400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 0x7f132c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1331400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 0x7f132c0cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39878/0 (socket says 172.21.15.97:39878) 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.224+0000 7f1331400640 1 -- 172.21.15.97:0/1991271715 learned_addr learned my addr 172.21.15.97:0/1991271715 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f132be00640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1330a00640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f132c06dfc0 0x7f132c15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1331400640 1 -- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 msgr2=0x7f132c0ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1331400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1331400640 1 -- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f132c06dfc0 msgr2=0x7f132c15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1331400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f132c06dfc0 0x7f132c15c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1331400640 1 -- 172.21.15.97:0/1991271715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009b30 con 0x7f132c14bea0 2024-01-31T05:04:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1331400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 0x7f132c0cf2b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1324036040 tx=0x7f1324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324009510 con 0x7f132c14bea0 2024-01-31T05:04:23.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132403e450 con 0x7f132c14bea0 2024-01-31T05:04:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132403e800 con 0x7f132c14bea0 2024-01-31T05:04:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.225+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132c15bd40 con 0x7f132c14bea0 2024-01-31T05:04:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.226+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c0ca010 con 0x7f132c14bea0 2024-01-31T05:04:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.227+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132c15c200 con 0x7f132c14bea0 2024-01-31T05:04:23.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.228+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f132402cd50 con 0x7f132c14bea0 2024-01-31T05:04:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.229+0000 7f1328c00640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f130406aed0 0x7f130406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.229+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f13240ac320 con 0x7f132c14bea0 2024-01-31T05:04:23.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.229+0000 7f1330a00640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f130406aed0 0x7f130406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:23.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.229+0000 7f1330a00640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f130406aed0 0x7f130406d390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ba10 tx=0x7f131c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:23.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.233+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f1324077010 con 0x7f132c14bea0 2024-01-31T05:04:23.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.437+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f132c067830 con 0x7f132c14bea0 2024-01-31T05:04:23.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.438+0000 7f1328c00640 1 -- 172.21.15.97:0/1991271715 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f132407a630 con 0x7f132c14bea0 2024-01-31T05:04:23.437 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f130406aed0 msgr2=0x7f130406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f130406aed0 0x7f130406d390 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f131c00ba10 tx=0x7f131c008040 comp rx=0 tx=0).stop 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 msgr2=0x7f132c0cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 0x7f132c0cf2b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1324036040 tx=0x7f1324004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 shutdown_connections 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f130406aed0 0x7f130406d390 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.440 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.440+0000 7f1333400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f132c14bea0 0x7f132c0cf2b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.441+0000 7f1333400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132c15a350 0x7f132c0ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.441+0000 7f1333400640 1 --2- 172.21.15.97:0/1991271715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f132c06dfc0 0x7f132c15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.441+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 >> 172.21.15.97:0/1991271715 conn(0x7f132c0b0a10 msgr2=0x7f132c073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.441+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 shutdown_connections 2024-01-31T05:04:23.441 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:23.441+0000 7f1333400640 1 -- 172.21.15.97:0/1991271715 wait complete. 2024-01-31T05:04:23.458 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi097.front.sepia.ceph.com... 2024-01-31T05:04:23.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:23.458 DEBUG:teuthology.orchestra.run.smithi097:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-31T05:04:23.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi097.stderr:ceph-fuse[60347]: fuse finished with error 0 and tester_r 0 2024-01-31T05:04:23.546 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:23.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:23.546 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-31T05:04:23.715 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:04:23.716 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:23.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:04:23.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:23.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:23.772 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-31T05:04:23.823 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-01-31T05:04:23.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:23.825 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:04:23.825 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:23.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:04:23.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:23.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:23.882 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-31T05:04:23.932 INFO:teuthology.orchestra.run.smithi097.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-31T05:04:23.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:23.934 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:04:23.934 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:23.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:04:23.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:23.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:23.991 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-31T05:04:24.043 INFO:teuthology.orchestra.run.smithi188.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-31T05:04:24.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:24.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:04:24.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.258+0000 7f7d9ae00640 1 Processor -- start 2024-01-31T05:04:24.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.258+0000 7f7d9ae00640 1 -- start start 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.259+0000 7f7d9ae00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d9415d890 0x7f7d940c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.259+0000 7f7d9ae00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d940c8290 con 0x7f7d9406c180 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.259+0000 7f7d9ae00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d940cca20 con 0x7f7d9415e380 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.259+0000 7f7d9ae00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d940ccd00 con 0x7f7d9415d890 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d99a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7d9415d890 0x7f7d940c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d99000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7d9406c180 0x7f7d9415dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39722/0 (socket says 172.21.15.97:39722) 2024-01-31T05:04:24.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d99000640 1 -- 172.21.15.97:0/4184041258 learned_addr learned my addr 172.21.15.97:0/4184041258 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:24.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d99a00640 1 -- 172.21.15.97:0/4184041258 >> v1:172.21.15.188:6789/0 conn(0x7f7d9415e380 legacy=0x7f7d94166e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:24.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d99a00640 1 -- 172.21.15.97:0/4184041258 >> v1:172.21.15.97:6789/0 conn(0x7f7d9406c180 legacy=0x7f7d9415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d99a00640 1 -- 172.21.15.97:0/4184041258 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d940ca8e0 con 0x7f7d9415d890 2024-01-31T05:04:24.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.260+0000 7f7d93e00640 1 -- 172.21.15.97:0/4184041258 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245486714 0 0) 0x7f7d940c8290 con 0x7f7d9406c180 2024-01-31T05:04:24.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.261+0000 7f7d99a00640 1 --2- 172.21.15.97:0/4184041258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d940c9c70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097a0 tx=0x7f7d8402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c2e4e02375c7eb server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.261+0000 7f7d93e00640 1 -- 172.21.15.97:0/4184041258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8402f050 con 0x7f7d9415d890 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.261+0000 7f7d93e00640 1 -- 172.21.15.97:0/4184041258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84031700 con 0x7f7d9415d890 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.261+0000 7f7d93e00640 1 -- 172.21.15.97:0/4184041258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84031a90 con 0x7f7d9415d890 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 -- 172.21.15.97:0/4184041258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 msgr2=0x7f7d940c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/4184041258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d940c9c70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d840097a0 tx=0x7f7d8402b7a0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 -- 172.21.15.97:0/4184041258 shutdown_connections 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/4184041258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d940c9c70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 -- 172.21.15.97:0/4184041258 >> 172.21.15.97:0/4184041258 conn(0x7f7d940cec40 msgr2=0x7f7d940cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 -- 172.21.15.97:0/4184041258 shutdown_connections 2024-01-31T05:04:24.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.262+0000 7f7d9ae00640 1 -- 172.21.15.97:0/4184041258 wait complete. 2024-01-31T05:04:24.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.264+0000 7f7d9ae00640 1 Processor -- start 2024-01-31T05:04:24.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.264+0000 7f7d9ae00640 1 -- start start 2024-01-31T05:04:24.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9ae00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406c180 0x7f7d940b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9ae00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d9414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9ae00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9415e380 0x7f7d9406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9ae00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940721b0 con 0x7f7d9415d890 2024-01-31T05:04:24.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9ae00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940b19e0 con 0x7f7d9415e380 2024-01-31T05:04:24.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9ae00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940afe00 con 0x7f7d9406c180 2024-01-31T05:04:24.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d99000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d9414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d99000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d9414c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39900/0 (socket says 172.21.15.97:39900) 2024-01-31T05:04:24.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d99000640 1 -- 172.21.15.97:0/3305331435 learned_addr learned my addr 172.21.15.97:0/3305331435 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:24.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d9a400640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9415e380 0x7f7d9406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d99000640 1 -- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406c180 msgr2=0x7f7d940b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.265+0000 7f7d99000640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406c180 0x7f7d940b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d99000640 1 -- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9415e380 msgr2=0x7f7d9406c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d99000640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9415e380 0x7f7d9406c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d99000640 1 -- 172.21.15.97:0/3305331435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d84009400 con 0x7f7d9415d890 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d9a400640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9415e380 0x7f7d9406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d99000640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d9414c600 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00c4b0 tx=0x7f7d7c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c0074e0 con 0x7f7d9415d890 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7c007660 con 0x7f7d9415d890 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c004a20 con 0x7f7d9415d890 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d940713f0 con 0x7f7d9415d890 2024-01-31T05:04:24.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.266+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d94071c90 con 0x7f7d9415d890 2024-01-31T05:04:24.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.267+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d940715e0 con 0x7f7d9415d890 2024-01-31T05:04:24.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.268+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f7d7c004b80 con 0x7f7d9415d890 2024-01-31T05:04:24.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.269+0000 7f7d91600640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7d6c06aed0 0x7f7d6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.270+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7d7c088ce0 con 0x7f7d9415d890 2024-01-31T05:04:24.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.270+0000 7f7d99a00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7d6c06aed0 0x7f7d6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.270+0000 7f7d99a00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7d6c06aed0 0x7f7d6c06d390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d84035000 tx=0x7f7d84030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:24.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.274+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7d7c053a90 con 0x7f7d9415d890 2024-01-31T05:04:24.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.502+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d94071eb0 con 0x7f7d9415d890 2024-01-31T05:04:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.503+0000 7f7d91600640 1 -- 172.21.15.97:0/3305331435 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f7d7c018020 con 0x7f7d9415d890 2024-01-31T05:04:24.503 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:24.503 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:03:38.646144+0000","modified":"2024-01-31T05:03:41.747278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3986947126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3986947126},{"type":"v1","addr":"172.21.15.97:6837","nonce":3986947126}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-31T05:04:24.503 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7d6c06aed0 msgr2=0x7f7d6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7d6c06aed0 0x7f7d6c06d390 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d84035000 tx=0x7f7d84030000 comp rx=0 tx=0).stop 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 msgr2=0x7f7d9414c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d9414c600 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00c4b0 tx=0x7f7d7c00c970 comp rx=0 tx=0).stop 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 shutdown_connections 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d9415e380 0x7f7d9406c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f7d6c06aed0 0x7f7d6c06d390 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d9415d890 0x7f7d9414c600 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 --2- 172.21.15.97:0/3305331435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d9406c180 0x7f7d940b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 >> 172.21.15.97:0/3305331435 conn(0x7f7d940cec40 msgr2=0x7f7d9415dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 shutdown_connections 2024-01-31T05:04:24.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.506+0000 7f7d9ae00640 1 -- 172.21.15.97:0/3305331435 wait complete. 2024-01-31T05:04:24.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.732+0000 7f79b5800640 1 Processor -- start 2024-01-31T05:04:24.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.732+0000 7f79b5800640 1 -- start start 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.733+0000 7f79b5800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79b00b1d10 0x7f79b00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.733+0000 7f79b5800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b015ea70 con 0x7f79b015a600 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79b5800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00b2630 con 0x7f79b006e0a0 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79b5800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b006c8b0 con 0x7f79b00b1d10 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79a5e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f79b015a600 0x7f79b015dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39728/0 (socket says 172.21.15.97:39728) 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79a5e00640 1 -- 172.21.15.97:0/867072688 learned_addr learned my addr 172.21.15.97:0/867072688 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:24.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79aea00640 1 --2- 172.21.15.97:0/867072688 >> v2:172.21.15.97:3300/0 conn(0x7f79b00b1d10 0x7f79b00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79ae000640 1 -- 172.21.15.97:0/867072688 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227462674 0 0) 0x7f79b015ea70 con 0x7f79b015a600 2024-01-31T05:04:24.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79aea00640 1 -- 172.21.15.97:0/867072688 >> v1:172.21.15.188:6789/0 conn(0x7f79b006e0a0 legacy=0x7f79b015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:24.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79aea00640 1 -- 172.21.15.97:0/867072688 >> v1:172.21.15.97:6789/0 conn(0x7f79b015a600 legacy=0x7f79b015dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.734+0000 7f79aea00640 1 -- 172.21.15.97:0/867072688 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b015b120 con 0x7f79b00b1d10 2024-01-31T05:04:24.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.735+0000 7f79aea00640 1 --2- 172.21.15.97:0/867072688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b00b1d10 0x7f79b00b20f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097a0 tx=0x7f79a002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5ca64347df665d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:24.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79ae000640 1 -- 172.21.15.97:0/867072688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a002f8a0 con 0x7f79b00b1d10 2024-01-31T05:04:24.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79ae000640 1 -- 172.21.15.97:0/867072688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a002fa00 con 0x7f79b00b1d10 2024-01-31T05:04:24.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79ae000640 1 -- 172.21.15.97:0/867072688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a002fd90 con 0x7f79b00b1d10 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 -- 172.21.15.97:0/867072688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b00b1d10 msgr2=0x7f79b00b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 --2- 172.21.15.97:0/867072688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b00b1d10 0x7f79b00b20f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097a0 tx=0x7f79a002dae0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 -- 172.21.15.97:0/867072688 shutdown_connections 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 --2- 172.21.15.97:0/867072688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b00b1d10 0x7f79b00b20f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 -- 172.21.15.97:0/867072688 >> 172.21.15.97:0/867072688 conn(0x7f79b00b0a10 msgr2=0x7f79b00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 -- 172.21.15.97:0/867072688 shutdown_connections 2024-01-31T05:04:24.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.736+0000 7f79b5800640 1 -- 172.21.15.97:0/867072688 wait complete. 2024-01-31T05:04:24.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.738+0000 7f79b5800640 1 Processor -- start 2024-01-31T05:04:24.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.738+0000 7f79b5800640 1 -- start start 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79b5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79b006e0a0 0x7f79b0149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79b5800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79b00b1d10 0x7f79b015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79b5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 0x7f79b01e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79b5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015b9e0 con 0x7f79b015a600 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79b5800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015bcc0 con 0x7f79b00b1d10 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79b5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015bfa0 con 0x7f79b006e0a0 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 0x7f79b01e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79af400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 0x7f79b01e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39916/0 (socket says 172.21.15.97:39916) 2024-01-31T05:04:24.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.739+0000 7f79af400640 1 -- 172.21.15.97:0/3922161119 learned_addr learned my addr 172.21.15.97:0/3922161119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79a5e00640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79b00b1d10 0x7f79b015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79aea00640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79b006e0a0 0x7f79b0149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79af400640 1 -- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79b006e0a0 msgr2=0x7f79b0149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79af400640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79b006e0a0 0x7f79b0149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79af400640 1 -- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79b00b1d10 msgr2=0x7f79b015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79af400640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79b00b1d10 0x7f79b015d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79af400640 1 -- 172.21.15.97:0/3922161119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0009400 con 0x7f79b015a600 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79a5e00640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79b00b1d10 0x7f79b015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79af400640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 0x7f79b01e8e30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f79a800c6d0 tx=0x7f79a800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800a840 con 0x7f79b015a600 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800a9a0 con 0x7f79b015a600 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.740+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800ad90 con 0x7f79b015a600 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.741+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b00adb80 con 0x7f79b015a600 2024-01-31T05:04:24.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.741+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b015c560 con 0x7f79b015a600 2024-01-31T05:04:24.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.742+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b015c780 con 0x7f79b015a600 2024-01-31T05:04:24.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.743+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f79a8012630 con 0x7f79b015a600 2024-01-31T05:04:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.744+0000 7f79a7400640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f798806aed0 0x7f798806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.744+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f79a8089f30 con 0x7f79b015a600 2024-01-31T05:04:24.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.744+0000 7f79aea00640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f798806aed0 0x7f798806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:24.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.745+0000 7f79aea00640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f798806aed0 0x7f798806d390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f79a0038000 tx=0x7f79a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:24.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.748+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f79a8054c20 con 0x7f79b015a600 2024-01-31T05:04:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.942+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79b006e740 con 0x7f79b015a600 2024-01-31T05:04:24.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.945+0000 7f79a7400640 1 -- 172.21.15.97:0/3922161119 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f79a8016070 con 0x7f79b015a600 2024-01-31T05:04:24.945 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:24.945 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f798806aed0 msgr2=0x7f798806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f798806aed0 0x7f798806d390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f79a0038000 tx=0x7f79a0008030 comp rx=0 tx=0).stop 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 msgr2=0x7f79b01e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 0x7f79b01e8e30 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f79a800c6d0 tx=0x7f79a800cb90 comp rx=0 tx=0).stop 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 shutdown_connections 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f798806aed0 0x7f798806d390 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79b015a600 0x7f79b01e8e30 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79b00b1d10 0x7f79b015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 --2- 172.21.15.97:0/3922161119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79b006e0a0 0x7f79b0149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 >> 172.21.15.97:0/3922161119 conn(0x7f79b00b0a10 msgr2=0x7f79b00ac2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:24.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 shutdown_connections 2024-01-31T05:04:24.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:24.948+0000 7f79b5800640 1 -- 172.21.15.97:0/3922161119 wait complete. 2024-01-31T05:04:24.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-31T05:00:39.613314+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-31T05:03:36.506793+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-01-31T05:03:37.685194+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-31T05:04:24.967 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-31T05:04:25.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.179+0000 7f9f6ea00640 1 Processor -- start 2024-01-31T05:04:25.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.179+0000 7f9f6ea00640 1 -- start start 2024-01-31T05:04:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.181+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 0x7f9f680722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.181+0000 7f9f6ea00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6806cd20 con 0x7f9f680abcf0 2024-01-31T05:04:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.181+0000 7f9f6ea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f680ceb60 con 0x7f9f6815b470 2024-01-31T05:04:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.181+0000 7f9f6ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6815b850 con 0x7f9f680cbf20 2024-01-31T05:04:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.181+0000 7f9f6ca00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9f680abcf0 0x7f9f6806d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49310/0 (socket says 172.21.15.97:49310) 2024-01-31T05:04:25.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.181+0000 7f9f6ca00640 1 -- 172.21.15.97:0/1142951304 learned_addr learned my addr 172.21.15.97:0/1142951304 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f67400640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 0x7f9f680722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952648460 0 0) 0x7f9f6806cd20 con 0x7f9f680abcf0 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f48003520 con 0x7f9f680abcf0 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951632795 0 0) 0x7f9f48003520 con 0x7f9f680abcf0 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 msgr2=0x7f9f680722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 0x7f9f680722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 >> v1:172.21.15.188:6789/0 conn(0x7f9f6815b470 legacy=0x7f9f680cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6806c3f0 con 0x7f9f680abcf0 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f9f60001670 con 0x7f9f680abcf0 2024-01-31T05:04:25.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.182+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 >> v1:172.21.15.97:6789/0 conn(0x7f9f680abcf0 legacy=0x7f9f6806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f66a00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f480039a0 0x7f9f48003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f66a00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 0x7f9f480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f66a00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6815b850 con 0x7f9f48004d00 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4800b980 con 0x7f9f480039a0 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4800bb40 con 0x7f9f680cbf20 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f6ca00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f67400640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 0x7f9f480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f67e00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f480039a0 0x7f9f48003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f6ca00640 1 -- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 msgr2=0x7f9f480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f6ca00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f680cbf20 0x7f9f480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f6ca00640 1 -- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f480039a0 msgr2=0x7f9f48003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f6ca00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f480039a0 0x7f9f48003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.183+0000 7f9f6ca00640 1 -- 172.21.15.97:0/1142951304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48003520 con 0x7f9f48004d00 2024-01-31T05:04:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.184+0000 7f9f6ca00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f48005180 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9f680cb140 tx=0x7f9f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fc577107b3d5a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:25.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60005ad0 con 0x7f9f48004d00 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f60005c50 con 0x7f9f48004d00 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f66a00640 1 -- 172.21.15.97:0/1142951304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6002c890 con 0x7f9f48004d00 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1142951304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 msgr2=0x7f9f48005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1142951304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f48005180 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9f680cb140 tx=0x7f9f480057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ca00640 1 -- 172.21.15.97:0/1142951304 reap_dead start 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1142951304 shutdown_connections 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1142951304 >> 172.21.15.97:0/1142951304 conn(0x7f9f680b0a10 msgr2=0x7f9f680b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1142951304 shutdown_connections 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.185+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1142951304 wait complete. 2024-01-31T05:04:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.187+0000 7f9f6ea00640 1 Processor -- start 2024-01-31T05:04:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.187+0000 7f9f6ea00640 1 -- start start 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.188+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f681450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.188+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6809aa10 0x7f9f68144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.188+0000 7f9f6ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f68146850 0x7f9f680b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.188+0000 7f9f6ea00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f681e3c60 con 0x7f9f48004d00 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.188+0000 7f9f6ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f681457b0 con 0x7f9f68146850 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.188+0000 7f9f6ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68145a90 con 0x7f9f6809aa10 2024-01-31T05:04:25.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f681450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f681450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37854/0 (socket says 172.21.15.97:37854) 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 -- 172.21.15.97:0/1062635 learned_addr learned my addr 172.21.15.97:0/1062635 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67400640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6809aa10 0x7f9f68144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 -- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6809aa10 msgr2=0x7f9f68144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6809aa10 0x7f9f68144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 -- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f68146850 msgr2=0x7f9f680b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f6ca00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f68146850 0x7f9f680b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f68146850 0x7f9f680b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 -- 172.21.15.97:0/1062635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f68145d70 con 0x7f9f48004d00 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.189+0000 7f9f67e00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f681450b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800e580 tx=0x7f9f5800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.190+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580052e0 con 0x7f9f48004d00 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.190+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58005460 con 0x7f9f48004d00 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.190+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f68146050 con 0x7f9f48004d00 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.190+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58003880 con 0x7f9f48004d00 2024-01-31T05:04:25.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.190+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f68144d70 con 0x7f9f48004d00 2024-01-31T05:04:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.191+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f68146570 con 0x7f9f48004d00 2024-01-31T05:04:25.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.192+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f9f580039e0 con 0x7f9f48004d00 2024-01-31T05:04:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.193+0000 7f9f5fe00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f4006aed0 0x7f9f4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.193+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9f58089700 con 0x7f9f48004d00 2024-01-31T05:04:25.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.193+0000 7f9f67400640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f4006aed0 0x7f9f4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.194+0000 7f9f67400640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f4006aed0 0x7f9f4006d390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9f681f65c0 tx=0x7f9f5000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:25.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.197+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f9f68146570 con 0x7f9f48004d00 2024-01-31T05:04:25.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.387+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f68067140 con 0x7f9f48004d00 2024-01-31T05:04:25.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.388+0000 7f9f5fe00640 1 -- 172.21.15.97:0/1062635 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f58054480 con 0x7f9f48004d00 2024-01-31T05:04:25.388 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f4006aed0 msgr2=0x7f9f4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f4006aed0 0x7f9f4006d390 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9f681f65c0 tx=0x7f9f5000b040 comp rx=0 tx=0).stop 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 msgr2=0x7f9f681450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f681450b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800e580 tx=0x7f9f5800ea40 comp rx=0 tx=0).stop 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 shutdown_connections 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f68146850 0x7f9f680b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6809aa10 0x7f9f68144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f9f4006aed0 0x7f9f4006d390 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 --2- 172.21.15.97:0/1062635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48004d00 0x7f9f681450b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 >> 172.21.15.97:0/1062635 conn(0x7f9f680b0a10 msgr2=0x7f9f68158480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:25.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 shutdown_connections 2024-01-31T05:04:25.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.391+0000 7f9f6ea00640 1 -- 172.21.15.97:0/1062635 wait complete. 2024-01-31T05:04:25.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-31T05:04:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.622+0000 7f7720400640 1 Processor -- start 2024-01-31T05:04:25.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.622+0000 7f7720400640 1 -- start start 2024-01-31T05:04:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f7720400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7718161b10 0x7f77180b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f7720400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7718161fc0 con 0x7f77180cb7d0 2024-01-31T05:04:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f7720400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77181622a0 con 0x7f77180ce1e0 2024-01-31T05:04:25.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f7720400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7718162c40 con 0x7f7718161b10 2024-01-31T05:04:25.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f771e400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77180cb7d0 0x7f77180cbc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49322/0 (socket says 172.21.15.97:49322) 2024-01-31T05:04:25.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f771e400640 1 -- 172.21.15.97:0/1058595228 learned_addr learned my addr 172.21.15.97:0/1058595228 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425249196 0 0) 0x7f7718161fc0 con 0x7f77180cb7d0 2024-01-31T05:04:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f771d000640 1 --2- 172.21.15.97:0/1058595228 >> v2:172.21.15.188:3300/0 conn(0x7f7718161b10 0x7f77180b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.624+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76f8003520 con 0x7f77180cb7d0 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895375647 0 0) 0x7f76f8003520 con 0x7f77180cb7d0 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 >> v2:172.21.15.188:3300/0 conn(0x7f7718161b10 msgr2=0x7f77180b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 --2- 172.21.15.97:0/1058595228 >> v2:172.21.15.188:3300/0 conn(0x7f7718161b10 0x7f77180b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 >> v1:172.21.15.188:6789/0 conn(0x7f77180ce1e0 legacy=0x7f77180ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718162580 con 0x7f77180cb7d0 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f7710001670 con 0x7f77180cb7d0 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 >> v1:172.21.15.97:6789/0 conn(0x7f77180cb7d0 legacy=0x7f77180cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76f8003970 0x7f76f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f80047c0 0x7f76f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f80052a0 0x7f76f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718162c40 con 0x7f76f8003970 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f800c170 con 0x7f76f80047c0 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f800c330 con 0x7f76f80052a0 2024-01-31T05:04:25.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f771d000640 1 --2- 172.21.15.97:0/1058595228 >> v2:172.21.15.188:3300/0 conn(0x7f7718161b10 0x7f77180b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.625+0000 7f771da00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76f8003970 0x7f76f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771d000640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f80047c0 0x7f76f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771da00640 1 -- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f80052a0 msgr2=0x7f76f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771da00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f80052a0 0x7f76f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771da00640 1 -- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f80047c0 msgr2=0x7f76f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771da00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f80047c0 0x7f76f8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771da00640 1 -- 172.21.15.97:0/1058595228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8003520 con 0x7f76f8003970 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.626+0000 7f771e400640 1 -- 172.21.15.97:0/1058595228 reap_dead start 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.627+0000 7f771da00640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76f8003970 0x7f76f8003d90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7708007fd0 tx=0x7f76f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2288dee905a25113 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.627+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708003bc0 con 0x7f76f8003970 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.627+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7708003d20 con 0x7f76f8003970 2024-01-31T05:04:25.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.627+0000 7f7717e00640 1 -- 172.21.15.97:0/1058595228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708005310 con 0x7f76f8003970 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.627+0000 7f7720400640 1 -- 172.21.15.97:0/1058595228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76f8003970 msgr2=0x7f76f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.627+0000 7f7720400640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76f8003970 0x7f76f8003d90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7708007fd0 tx=0x7f76f80057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.628+0000 7f7720400640 1 -- 172.21.15.97:0/1058595228 shutdown_connections 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.628+0000 7f7720400640 1 --2- 172.21.15.97:0/1058595228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76f8003970 0x7f76f8003d90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.628+0000 7f7720400640 1 -- 172.21.15.97:0/1058595228 >> 172.21.15.97:0/1058595228 conn(0x7f77180b0a10 msgr2=0x7f77180b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.628+0000 7f7720400640 1 -- 172.21.15.97:0/1058595228 shutdown_connections 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.628+0000 7f7720400640 1 -- 172.21.15.97:0/1058595228 wait complete. 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.629+0000 7f7720400640 1 Processor -- start 2024-01-31T05:04:25.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 -- start start 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003970 0x7f771809c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77180ca5d0 0x7f77181ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 0x7f7718071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181ec5b0 con 0x7f77181ec130 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771809b650 con 0x7f77180ca5d0 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.630+0000 7f7720400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771809b8d0 con 0x7f76f8003970 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 0x7f7718071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 0x7f7718071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37862/0 (socket says 172.21.15.97:37862) 2024-01-31T05:04:25.631 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 -- 172.21.15.97:0/1607585114 learned_addr learned my addr 172.21.15.97:0/1607585114 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 -- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003970 msgr2=0x7f771809c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003970 0x7f771809c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771d000640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77180ca5d0 0x7f77181ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 -- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77180ca5d0 msgr2=0x7f77181ebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77180ca5d0 0x7f77181ebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 -- 172.21.15.97:0/1607585114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771809bbb0 con 0x7f77181ec130 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771d000640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77180ca5d0 0x7f77181ebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.631+0000 7f771e400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 0x7f7718071470 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f77100011e0 tx=0x7f771000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.632+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710007230 con 0x7f77181ec130 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.632+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710007390 con 0x7f77181ec130 2024-01-31T05:04:25.633 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.632+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000ce50 con 0x7f77181ec130 2024-01-31T05:04:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.632+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771809be90 con 0x7f77181ec130 2024-01-31T05:04:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.632+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180caa30 con 0x7f77181ec130 2024-01-31T05:04:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.633+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718067830 con 0x7f77181ec130 2024-01-31T05:04:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.634+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f77100047e0 con 0x7f77181ec130 2024-01-31T05:04:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.635+0000 7f7715600640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76f006aed0 0x7f76f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.636+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7710088b90 con 0x7f77181ec130 2024-01-31T05:04:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.636+0000 7f771da00640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76f006aed0 0x7f76f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:25.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.636+0000 7f771da00640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76f006aed0 0x7f76f006d390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7708034000 tx=0x7f7708030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:25.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.640+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f7710053880 con 0x7f77181ec130 2024-01-31T05:04:25.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.817+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77180cc6c0 con 0x7f77181ec130 2024-01-31T05:04:25.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.818+0000 7f7715600640 1 -- 172.21.15.97:0/1607585114 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f771001a020 con 0x7f77181ec130 2024-01-31T05:04:25.818 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-01-31T05:04:25.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.820+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76f006aed0 msgr2=0x7f76f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76f006aed0 0x7f76f006d390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7708034000 tx=0x7f7708030000 comp rx=0 tx=0).stop 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 msgr2=0x7f7718071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 0x7f7718071470 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f77100011e0 tx=0x7f771000cb10 comp rx=0 tx=0).stop 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 shutdown_connections 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f76f006aed0 0x7f76f006d390 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77181ec130 0x7f7718071470 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77180ca5d0 0x7f77181ebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 --2- 172.21.15.97:0/1607585114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003970 0x7f771809c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 >> 172.21.15.97:0/1607585114 conn(0x7f77180b0a10 msgr2=0x7f77180b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 shutdown_connections 2024-01-31T05:04:25.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:25.821+0000 7f7720400640 1 -- 172.21.15.97:0/1607585114 wait complete. 2024-01-31T05:04:25.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-31T05:04:26.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.046+0000 7fae7de00640 1 Processor -- start 2024-01-31T05:04:26.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.046+0000 7fae7de00640 1 -- start start 2024-01-31T05:04:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.047+0000 7fae7de00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae780722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.047+0000 7fae7de00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7806cd20 con 0x7fae780abcf0 2024-01-31T05:04:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.047+0000 7fae7de00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae780ceb60 con 0x7fae780cbf20 2024-01-31T05:04:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.047+0000 7fae7de00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815b850 con 0x7fae7815b470 2024-01-31T05:04:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.048+0000 7fae6fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fae780abcf0 0x7fae780cb810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49324/0 (socket says 172.21.15.97:49324) 2024-01-31T05:04:26.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.048+0000 7fae6fe00640 1 -- 172.21.15.97:0/2230757882 learned_addr learned my addr 172.21.15.97:0/2230757882 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.048+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200861334 0 0) 0x7fae7806cd20 con 0x7fae780abcf0 2024-01-31T05:04:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.048+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae58003520 con 0x7fae780abcf0 2024-01-31T05:04:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.048+0000 7fae76a00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae780722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4209980630 0 0) 0x7fae58003520 con 0x7fae780abcf0 2024-01-31T05:04:26.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 msgr2=0x7fae780722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae780722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 >> v1:172.21.15.188:6789/0 conn(0x7fae780cbf20 legacy=0x7fae7806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7806c3f0 con 0x7fae780abcf0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fae60001670 con 0x7fae780abcf0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 >> v1:172.21.15.97:6789/0 conn(0x7fae780abcf0 legacy=0x7fae780cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae580039a0 0x7fae58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815b850 con 0x7fae580047c0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5800bab0 con 0x7fae580039a0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5800bc70 con 0x7fae7815b470 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae6fe00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae76a00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae580039a0 0x7fae58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae77400640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.049+0000 7fae6fe00640 1 -- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 msgr2=0x7fae580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.050+0000 7fae6fe00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.050+0000 7fae6fe00640 1 -- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae580039a0 msgr2=0x7fae58003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.050+0000 7fae6fe00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae580039a0 0x7fae58003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.050+0000 7fae6fe00640 1 -- 172.21.15.97:0/2230757882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58003520 con 0x7fae580047c0 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.050+0000 7fae77400640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae7815b470 0x7fae580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae6fe00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae58004c40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c210 tx=0x7fae580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc94c869459f5ad server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60005ad0 con 0x7fae580047c0 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae60005c30 con 0x7fae580047c0 2024-01-31T05:04:26.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae76000640 1 -- 172.21.15.97:0/2230757882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6002c890 con 0x7fae580047c0 2024-01-31T05:04:26.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae7de00640 1 -- 172.21.15.97:0/2230757882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 msgr2=0x7fae58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae7de00640 1 --2- 172.21.15.97:0/2230757882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae58004c40 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c210 tx=0x7fae580057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae77400640 1 -- 172.21.15.97:0/2230757882 reap_dead start 2024-01-31T05:04:26.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.051+0000 7fae7de00640 1 -- 172.21.15.97:0/2230757882 shutdown_connections 2024-01-31T05:04:26.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.052+0000 7fae7de00640 1 -- 172.21.15.97:0/2230757882 >> 172.21.15.97:0/2230757882 conn(0x7fae780b0a10 msgr2=0x7fae780b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.052+0000 7fae7de00640 1 -- 172.21.15.97:0/2230757882 shutdown_connections 2024-01-31T05:04:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.052+0000 7fae7de00640 1 -- 172.21.15.97:0/2230757882 wait complete. 2024-01-31T05:04:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.053+0000 7fae7de00640 1 Processor -- start 2024-01-31T05:04:26.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.054+0000 7fae7de00640 1 -- start start 2024-01-31T05:04:26.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.054+0000 7fae7de00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae78142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.054+0000 7fae7de00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7815d7f0 0x7fae781401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae7de00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781e39b0 0x7fae7815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae7de00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae78141ee0 con 0x7fae580047c0 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae7de00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae78141540 con 0x7fae7815d7f0 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae7de00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae78141820 con 0x7fae781e39b0 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae78142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae78142300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37890/0 (socket says 172.21.15.97:37890) 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 -- 172.21.15.97:0/1838377800 learned_addr learned my addr 172.21.15.97:0/1838377800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 -- 172.21.15.97:0/1838377800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781e39b0 msgr2=0x7fae7815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781e39b0 0x7fae7815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 -- 172.21.15.97:0/1838377800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7815d7f0 msgr2=0x7fae781401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7815d7f0 0x7fae781401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 -- 172.21.15.97:0/1838377800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae78141b00 con 0x7fae580047c0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.055+0000 7fae76a00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae78142300 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fae6800f560 tx=0x7fae6800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.056+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68005da0 con 0x7fae580047c0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.056+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae68006050 con 0x7fae580047c0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.056+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680063e0 con 0x7fae580047c0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.056+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae78141180 con 0x7fae580047c0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.056+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae781e4010 con 0x7fae580047c0 2024-01-31T05:04:26.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.057+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78142150 con 0x7fae580047c0 2024-01-31T05:04:26.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.058+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fae68006540 con 0x7fae580047c0 2024-01-31T05:04:26.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.059+0000 7fae6f400640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae5006aed0 0x7fae5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.059+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fae68089e80 con 0x7fae580047c0 2024-01-31T05:04:26.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.059+0000 7fae6fe00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae5006aed0 0x7fae5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.060+0000 7fae6fe00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae5006aed0 0x7fae5006d390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fae60034000 tx=0x7fae60007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.063+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fae68054b70 con 0x7fae580047c0 2024-01-31T05:04:26.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.261+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae78141c90 con 0x7fae580047c0 2024-01-31T05:04:26.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.262+0000 7fae6f400640 1 -- 172.21.15.97:0/1838377800 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae68058190 con 0x7fae580047c0 2024-01-31T05:04:26.262 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-01-31T05:04:26.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.264+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae5006aed0 msgr2=0x7fae5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.264+0000 7fae7de00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae5006aed0 0x7fae5006d390 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fae60034000 tx=0x7fae60007000 comp rx=0 tx=0).stop 2024-01-31T05:04:26.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.264+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 msgr2=0x7fae78142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.264+0000 7fae7de00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae78142300 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fae6800f560 tx=0x7fae6800fa20 comp rx=0 tx=0).stop 2024-01-31T05:04:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 shutdown_connections 2024-01-31T05:04:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae781e39b0 0x7fae7815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae7815d7f0 0x7fae781401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fae5006aed0 0x7fae5006d390 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 --2- 172.21.15.97:0/1838377800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae580047c0 0x7fae78142300 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 >> 172.21.15.97:0/1838377800 conn(0x7fae780b0a10 msgr2=0x7fae7815b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 shutdown_connections 2024-01-31T05:04:26.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.265+0000 7fae7de00640 1 -- 172.21.15.97:0/1838377800 wait complete. 2024-01-31T05:04:26.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:04:26.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.492+0000 7ff75b200640 1 Processor -- start 2024-01-31T05:04:26.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.492+0000 7ff75b200640 1 -- start start 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff75b200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff75c0d0d60 0x7ff75c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff75b200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c15d8c0 con 0x7ff75c0d1c70 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff75b200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0b1920 con 0x7ff75c15d4e0 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff75b200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0b0b20 con 0x7ff75c0d0d60 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff75a800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff75c0d0d60 0x7ff75c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff753e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff75c0d1c70 0x7ff75c0b5e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49328/0 (socket says 172.21.15.97:49328) 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.494+0000 7ff753e00640 1 -- 172.21.15.97:0/1935874260 learned_addr learned my addr 172.21.15.97:0/1935874260 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:26.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.495+0000 7ff75a800640 1 -- 172.21.15.97:0/1935874260 >> v1:172.21.15.188:6789/0 conn(0x7ff75c15d4e0 legacy=0x7ff75c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.495+0000 7ff75a800640 1 -- 172.21.15.97:0/1935874260 >> v1:172.21.15.97:6789/0 conn(0x7ff75c0d1c70 legacy=0x7ff75c0b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.495+0000 7ff75a800640 1 -- 172.21.15.97:0/1935874260 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0cef50 con 0x7ff75c0d0d60 2024-01-31T05:04:26.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.496+0000 7ff75a800640 1 --2- 172.21.15.97:0/1935874260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c0d0d60 0x7ff75c0b05e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff754009d90 tx=0x7ff75402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fcca8da4ba3ea9 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.496+0000 7ff759400640 1 -- 172.21.15.97:0/1935874260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75402cd80 con 0x7ff75c0d0d60 2024-01-31T05:04:26.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.496+0000 7ff759400640 1 -- 172.21.15.97:0/1935874260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754034440 con 0x7ff75c0d0d60 2024-01-31T05:04:26.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.496+0000 7ff759400640 1 -- 172.21.15.97:0/1935874260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7540347d0 con 0x7ff75c0d0d60 2024-01-31T05:04:26.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 -- 172.21.15.97:0/1935874260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c0d0d60 msgr2=0x7ff75c0b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 --2- 172.21.15.97:0/1935874260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c0d0d60 0x7ff75c0b05e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff754009d90 tx=0x7ff75402bfd0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 -- 172.21.15.97:0/1935874260 shutdown_connections 2024-01-31T05:04:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 --2- 172.21.15.97:0/1935874260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c0d0d60 0x7ff75c0b05e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 -- 172.21.15.97:0/1935874260 >> 172.21.15.97:0/1935874260 conn(0x7ff75c0d3010 msgr2=0x7ff75c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 -- 172.21.15.97:0/1935874260 shutdown_connections 2024-01-31T05:04:26.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.497+0000 7ff75b200640 1 -- 172.21.15.97:0/1935874260 wait complete. 2024-01-31T05:04:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.499+0000 7ff75b200640 1 Processor -- start 2024-01-31T05:04:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.499+0000 7ff75b200640 1 -- start start 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff75c0d0d60 0x7ff75c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff75c0d1c70 0x7ff75c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 0x7ff75c0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75b200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c7800 con 0x7ff75c15d4e0 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c79f0 con 0x7ff75c0d1c70 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c7cd0 con 0x7ff75c0d0d60 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 0x7ff75c0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75a800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 0x7ff75c0c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37910/0 (socket says 172.21.15.97:37910) 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.500+0000 7ff75a800640 1 -- 172.21.15.97:0/1336361238 learned_addr learned my addr 172.21.15.97:0/1336361238 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff759e00640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff75c0d0d60 0x7ff75c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff753e00640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff75c0d1c70 0x7ff75c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff75a800640 1 -- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff75c0d0d60 msgr2=0x7ff75c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff75a800640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff75c0d0d60 0x7ff75c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff75a800640 1 -- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff75c0d1c70 msgr2=0x7ff75c0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff75a800640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff75c0d1c70 0x7ff75c0b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff75a800640 1 -- 172.21.15.97:0/1336361238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009a70 con 0x7ff75c15d4e0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff75a800640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 0x7ff75c0c72c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff754009d90 tx=0x7ff754004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754040650 con 0x7ff75c15d4e0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7540407b0 con 0x7ff75c15d4e0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.501+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754040b60 con 0x7ff75c15d4e0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.502+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c15ed40 con 0x7ff75c15d4e0 2024-01-31T05:04:26.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.502+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c15f300 con 0x7ff75c15d4e0 2024-01-31T05:04:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.503+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c0676b0 con 0x7ff75c15d4e0 2024-01-31T05:04:26.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.504+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff754040cc0 con 0x7ff75c15d4e0 2024-01-31T05:04:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.505+0000 7ff752000640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff73406aed0 0x7ff73406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.505+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff7540ac820 con 0x7ff75c15d4e0 2024-01-31T05:04:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.505+0000 7ff759e00640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff73406aed0 0x7ff73406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.506+0000 7ff759e00640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff73406aed0 0x7ff73406d390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff74c000f60 tx=0x7ff74c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.509+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff754077540 con 0x7ff75c15d4e0 2024-01-31T05:04:26.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.741+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff75c06ae60 con 0x7ff75c15d4e0 2024-01-31T05:04:26.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.742+0000 7ff752000640 1 -- 172.21.15.97:0/1336361238 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff75407ab60 con 0x7ff75c15d4e0 2024-01-31T05:04:26.742 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:26.742 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:03:38.646144+0000","modified":"2024-01-31T05:03:41.747278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3986947126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3986947126},{"type":"v1","addr":"172.21.15.97:6837","nonce":3986947126}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-31T05:04:26.742 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff73406aed0 msgr2=0x7ff73406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff73406aed0 0x7ff73406d390 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff74c000f60 tx=0x7ff74c009040 comp rx=0 tx=0).stop 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 msgr2=0x7ff75c0c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 0x7ff75c0c72c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff754009d90 tx=0x7ff754004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 shutdown_connections 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff73406aed0 0x7ff73406d390 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff75c15d4e0 0x7ff75c0c72c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff75c0d1c70 0x7ff75c0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 --2- 172.21.15.97:0/1336361238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff75c0d0d60 0x7ff75c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.745+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 >> 172.21.15.97:0/1336361238 conn(0x7ff75c0d3010 msgr2=0x7ff75c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:26.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.746+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 shutdown_connections 2024-01-31T05:04:26.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.746+0000 7ff75b200640 1 -- 172.21.15.97:0/1336361238 wait complete. 2024-01-31T05:04:26.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:26.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.972+0000 7f297b800640 1 Processor -- start 2024-01-31T05:04:26.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.972+0000 7f297b800640 1 -- start start 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.973+0000 7f297b800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f297415b400 0x7f297406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.973+0000 7f297b800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297406d1a0 con 0x7f29740cb810 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.973+0000 7f297b800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297406c600 con 0x7f297406db70 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.973+0000 7f297b800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29740cc960 con 0x7f297415b400 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2978e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f29740cb810 0x7f29740cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49332/0 (socket says 172.21.15.97:49332) 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2978e00640 1 -- 172.21.15.97:0/3641681156 learned_addr learned my addr 172.21.15.97:0/3641681156 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2973e00640 1 --2- 172.21.15.97:0/3641681156 >> v2:172.21.15.97:3300/0 conn(0x7f297415b400 0x7f297406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2973400640 1 -- 172.21.15.97:0/3641681156 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723541995 0 0) 0x7f297406d1a0 con 0x7f29740cb810 2024-01-31T05:04:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2973400640 1 -- 172.21.15.97:0/3641681156 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2954003520 con 0x7f29740cb810 2024-01-31T05:04:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2973e00640 1 -- 172.21.15.97:0/3641681156 >> v1:172.21.15.188:6789/0 conn(0x7f297406db70 legacy=0x7f29740ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2973e00640 1 -- 172.21.15.97:0/3641681156 >> v1:172.21.15.97:6789/0 conn(0x7f29740cb810 legacy=0x7f29740cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.974+0000 7f2973e00640 1 -- 172.21.15.97:0/3641681156 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297406c870 con 0x7f297415b400 2024-01-31T05:04:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.975+0000 7f2973e00640 1 --2- 172.21.15.97:0/3641681156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297415b400 0x7f297406d630 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f295c0097a0 tx=0x7f295c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20771ab8d51ac76e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f2973400640 1 -- 172.21.15.97:0/3641681156 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c031830 con 0x7f297415b400 2024-01-31T05:04:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f2973400640 1 -- 172.21.15.97:0/3641681156 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c031990 con 0x7f297415b400 2024-01-31T05:04:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f2973400640 1 -- 172.21.15.97:0/3641681156 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c031d20 con 0x7f297415b400 2024-01-31T05:04:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 -- 172.21.15.97:0/3641681156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297415b400 msgr2=0x7f297406d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 --2- 172.21.15.97:0/3641681156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297415b400 0x7f297406d630 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f295c0097a0 tx=0x7f295c02b8d0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 -- 172.21.15.97:0/3641681156 shutdown_connections 2024-01-31T05:04:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 --2- 172.21.15.97:0/3641681156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297415b400 0x7f297406d630 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 -- 172.21.15.97:0/3641681156 >> 172.21.15.97:0/3641681156 conn(0x7f29740b0a10 msgr2=0x7f29740b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 -- 172.21.15.97:0/3641681156 shutdown_connections 2024-01-31T05:04:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.976+0000 7f297b800640 1 -- 172.21.15.97:0/3641681156 wait complete. 2024-01-31T05:04:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.978+0000 7f297b800640 1 Processor -- start 2024-01-31T05:04:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.978+0000 7f297b800640 1 -- start start 2024-01-31T05:04:26.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f297b800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f297406db70 0x7f2974152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.980 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f297b800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 0x7f297415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f297b800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f297415b400 0x7f29740b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f297b800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974152cd0 con 0x7f29740cb810 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f297b800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297415caf0 con 0x7f297415b400 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f297b800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297415cdd0 con 0x7f297406db70 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f2973e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 0x7f297415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f2973e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 0x7f297415c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37922/0 (socket says 172.21.15.97:37922) 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.979+0000 7f2973e00640 1 -- 172.21.15.97:0/1406403887 learned_addr learned my addr 172.21.15.97:0/1406403887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2978e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f297406db70 0x7f2974152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2979800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f297415b400 0x7f29740b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2973e00640 1 -- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f297406db70 msgr2=0x7f2974152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2973e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f297406db70 0x7f2974152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2973e00640 1 -- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f297415b400 msgr2=0x7f29740b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:26.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2973e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f297415b400 0x7f29740b2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2973e00640 1 -- 172.21.15.97:0/1406403887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c009400 con 0x7f29740cb810 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2978e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f297406db70 0x7f2974152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.980+0000 7f2973e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 0x7f297415c1f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f295c000c00 tx=0x7f295c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.981+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c03a8f0 con 0x7f29740cb810 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.981+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c03aa50 con 0x7f29740cb810 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.981+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c03ae00 con 0x7f29740cb810 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.981+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297415d0b0 con 0x7f29740cb810 2024-01-31T05:04:26.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.981+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297415d670 con 0x7f29740cb810 2024-01-31T05:04:26.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.982+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29740677c0 con 0x7f29740cb810 2024-01-31T05:04:26.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.983+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f295c04d450 con 0x7f29740cb810 2024-01-31T05:04:26.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.984+0000 7f2970c00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f294c06aed0 0x7f294c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:26.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.984+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f295c0ab870 con 0x7f29740cb810 2024-01-31T05:04:26.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.984+0000 7f2978e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f294c06aed0 0x7f294c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:26.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.985+0000 7f2978e00640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f294c06aed0 0x7f294c06d390 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2964005880 tx=0x7f2964009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:26.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:26.989+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f295c076560 con 0x7f29740cb810 2024-01-31T05:04:27.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.182+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29740adb70 con 0x7f29740cb810 2024-01-31T05:04:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.184+0000 7f2970c00640 1 -- 172.21.15.97:0/1406403887 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f295c035090 con 0x7f29740cb810 2024-01-31T05:04:27.184 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:27.185 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f294c06aed0 msgr2=0x7f294c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f294c06aed0 0x7f294c06d390 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2964005880 tx=0x7f2964009040 comp rx=0 tx=0).stop 2024-01-31T05:04:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 msgr2=0x7f297415c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 0x7f297415c1f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f295c000c00 tx=0x7f295c0046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 shutdown_connections 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f297415b400 0x7f29740b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f294c06aed0 0x7f294c06d390 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29740cb810 0x7f297415c1f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 --2- 172.21.15.97:0/1406403887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f297406db70 0x7f2974152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 >> 172.21.15.97:0/1406403887 conn(0x7f29740b0a10 msgr2=0x7f29740abfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 shutdown_connections 2024-01-31T05:04:27.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.187+0000 7f297b800640 1 -- 172.21.15.97:0/1406403887 wait complete. 2024-01-31T05:04:27.205 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-31T05:04:27.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:04:27.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.416+0000 7fcd2ca00640 1 Processor -- start 2024-01-31T05:04:27.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.417+0000 7fcd2ca00640 1 -- start start 2024-01-31T05:04:27.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.418+0000 7fcd2ca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcd240cfe60 0x7fcd240c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.418+0000 7fcd2ca00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd240ccef0 con 0x7fcd240cdeb0 2024-01-31T05:04:27.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.418+0000 7fcd2ca00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd240d1d30 con 0x7fcd2415b540 2024-01-31T05:04:27.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.418+0000 7fcd2ca00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd240cc8a0 con 0x7fcd240cfe60 2024-01-31T05:04:27.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.418+0000 7fcd29600640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcd240cdeb0 0x7fcd240c4d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49344/0 (socket says 172.21.15.97:49344) 2024-01-31T05:04:27.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.418+0000 7fcd29600640 1 -- 172.21.15.97:0/1623618951 learned_addr learned my addr 172.21.15.97:0/1623618951 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:27.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd2aa00640 1 --2- 172.21.15.97:0/1623618951 >> v2:172.21.15.188:3300/0 conn(0x7fcd240cfe60 0x7fcd240c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832291701 0 0) 0x7fcd240ccef0 con 0x7fcd240cdeb0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd04003520 con 0x7fcd240cdeb0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 183582351 0 0) 0x7fcd04003520 con 0x7fcd240cdeb0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 >> v2:172.21.15.188:3300/0 conn(0x7fcd240cfe60 msgr2=0x7fcd240c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 --2- 172.21.15.97:0/1623618951 >> v2:172.21.15.188:3300/0 conn(0x7fcd240cfe60 0x7fcd240c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 >> v1:172.21.15.188:6789/0 conn(0x7fcd2415b540 legacy=0x7fcd240cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd240ccb80 con 0x7fcd240cdeb0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.419+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fcd140019f0 con 0x7fcd240cdeb0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 >> v1:172.21.15.97:6789/0 conn(0x7fcd240cdeb0 legacy=0x7fcd240c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd040047c0 0x7fcd04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd04005180 0x7fcd0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240cc8a0 con 0x7fcd040039c0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0400c080 con 0x7fcd040047c0 2024-01-31T05:04:27.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0400c240 con 0x7fcd04005180 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2a000640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd29600640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd040047c0 0x7fcd04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2a000640 1 -- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd04005180 msgr2=0x7fcd0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2aa00640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd04005180 0x7fcd0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2a000640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd04005180 0x7fcd0400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2a000640 1 -- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd040047c0 msgr2=0x7fcd04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2a000640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd040047c0 0x7fcd04004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.420+0000 7fcd2a000640 1 -- 172.21.15.97:0/1623618951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04003520 con 0x7fcd040039c0 2024-01-31T05:04:27.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.421+0000 7fcd2aa00640 1 -- 172.21.15.97:0/1623618951 reap_dead start 2024-01-31T05:04:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.421+0000 7fcd2a000640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd04003de0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c001750 tx=0x7fcd040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7401a32e5f3089d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:27.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0052d0 con 0x7fcd040039c0 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c02e050 con 0x7fcd040039c0 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd28c00640 1 -- 172.21.15.97:0/1623618951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c003830 con 0x7fcd040039c0 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd2ca00640 1 -- 172.21.15.97:0/1623618951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 msgr2=0x7fcd04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd04003de0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c001750 tx=0x7fcd040057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd2ca00640 1 -- 172.21.15.97:0/1623618951 shutdown_connections 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/1623618951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd04003de0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.422+0000 7fcd2ca00640 1 -- 172.21.15.97:0/1623618951 >> 172.21.15.97:0/1623618951 conn(0x7fcd240d1070 msgr2=0x7fcd240d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.423+0000 7fcd2ca00640 1 -- 172.21.15.97:0/1623618951 shutdown_connections 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.423+0000 7fcd2ca00640 1 -- 172.21.15.97:0/1623618951 wait complete. 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.424+0000 7fcd2ca00640 1 Processor -- start 2024-01-31T05:04:27.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.424+0000 7fcd2ca00640 1 -- start start 2024-01-31T05:04:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2ca00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd24158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.425 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd241e5050 0x7fcd241e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2ca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240b5c90 0x7fcd240b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2ca00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd240b6a00 con 0x7fcd040039c0 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2ca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd24158d50 con 0x7fcd240b5c90 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd241e5a10 con 0x7fcd241e5050 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd24158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2a000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd24158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37930/0 (socket says 172.21.15.97:37930) 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.425+0000 7fcd2a000640 1 -- 172.21.15.97:0/174174290 learned_addr learned my addr 172.21.15.97:0/174174290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd2a000640 1 -- 172.21.15.97:0/174174290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd241e5050 msgr2=0x7fcd241e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd2a000640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd241e5050 0x7fcd241e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd2a000640 1 -- 172.21.15.97:0/174174290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240b5c90 msgr2=0x7fcd240b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd2a000640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240b5c90 0x7fcd240b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd2a000640 1 -- 172.21.15.97:0/174174290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2415bc20 con 0x7fcd040039c0 2024-01-31T05:04:27.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd2a000640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd24158810 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c034040 tx=0x7fcd0c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.426+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0076c0 con 0x7fcd040039c0 2024-01-31T05:04:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.427+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c007a30 con 0x7fcd040039c0 2024-01-31T05:04:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.427+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c006be0 con 0x7fcd040039c0 2024-01-31T05:04:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.427+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd240b3db0 con 0x7fcd040039c0 2024-01-31T05:04:27.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.427+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd240b3760 con 0x7fcd040039c0 2024-01-31T05:04:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.428+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd240b2960 con 0x7fcd040039c0 2024-01-31T05:04:27.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.429+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fcd0c006d40 con 0x7fcd040039c0 2024-01-31T05:04:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.430+0000 7fcd22000640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fccfc06af00 0x7fccfc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.430+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd0c0ab460 con 0x7fcd040039c0 2024-01-31T05:04:27.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.430+0000 7fcd29600640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fccfc06af00 0x7fccfc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.431+0000 7fcd29600640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fccfc06af00 0x7fccfc06d3c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcd14002230 tx=0x7fcd14001250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:27.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.435+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fcd0c076310 con 0x7fcd040039c0 2024-01-31T05:04:27.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.667+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd241e42b0 con 0x7fcd040039c0 2024-01-31T05:04:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.668+0000 7fcd22000640 1 -- 172.21.15.97:0/174174290 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcd0c07ee60 con 0x7fcd040039c0 2024-01-31T05:04:27.668 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:27.668 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:03:38.646144+0000","modified":"2024-01-31T05:03:41.747278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3986947126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3986947126},{"type":"v1","addr":"172.21.15.97:6837","nonce":3986947126}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-31T05:04:27.668 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-01-31T05:04:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.671+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fccfc06af00 msgr2=0x7fccfc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.671+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fccfc06af00 0x7fccfc06d3c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcd14002230 tx=0x7fcd14001250 comp rx=0 tx=0).stop 2024-01-31T05:04:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.671+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 msgr2=0x7fcd24158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.671+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd24158810 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c034040 tx=0x7fcd0c001120 comp rx=0 tx=0).stop 2024-01-31T05:04:27.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 shutdown_connections 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240b5c90 0x7fcd240b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd241e5050 0x7fcd241e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fccfc06af00 0x7fccfc06d3c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 --2- 172.21.15.97:0/174174290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd040039c0 0x7fcd24158810 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 >> 172.21.15.97:0/174174290 conn(0x7fcd240d1070 msgr2=0x7fcd2415a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 shutdown_connections 2024-01-31T05:04:27.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.672+0000 7fcd2ca00640 1 -- 172.21.15.97:0/174174290 wait complete. 2024-01-31T05:04:27.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:04:27.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.898+0000 7f2628a00640 1 Processor -- start 2024-01-31T05:04:27.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.898+0000 7f2628a00640 1 -- start start 2024-01-31T05:04:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2628a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f262409a4b0 0x7f26240cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2628a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f262415edd0 0x7f262415f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2628a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26240cff00 con 0x7f262415b270 2024-01-31T05:04:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2628a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26240cca00 con 0x7f262409a4b0 2024-01-31T05:04:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2628a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262415f780 con 0x7f262415edd0 2024-01-31T05:04:27.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f262409a4b0 0x7f26240cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f262409a4b0 0x7f26240cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37936/0 (socket says 172.21.15.97:37936) 2024-01-31T05:04:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 -- 172.21.15.97:0/2178829469 learned_addr learned my addr 172.21.15.97:0/2178829469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 -- 172.21.15.97:0/2178829469 >> v2:172.21.15.188:3300/0 conn(0x7f262415edd0 msgr2=0x7f262415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 --2- 172.21.15.97:0/2178829469 >> v2:172.21.15.188:3300/0 conn(0x7f262415edd0 0x7f262415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 -- 172.21.15.97:0/2178829469 >> v1:172.21.15.188:6789/0 conn(0x7f262415b270 legacy=0x7f26240cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:27.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.900+0000 7f2621e00640 1 -- 172.21.15.97:0/2178829469 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240ceac0 con 0x7f262409a4b0 2024-01-31T05:04:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.901+0000 7f2621e00640 1 --2- 172.21.15.97:0/2178829469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240cc4c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f260c0098a0 tx=0x7f260c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516ad6e934dcb476 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2621400640 1 -- 172.21.15.97:0/2178829469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c031640 con 0x7f262409a4b0 2024-01-31T05:04:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2621400640 1 -- 172.21.15.97:0/2178829469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c0317a0 con 0x7f262409a4b0 2024-01-31T05:04:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2621400640 1 -- 172.21.15.97:0/2178829469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c031b30 con 0x7f262409a4b0 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 -- 172.21.15.97:0/2178829469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 msgr2=0x7f26240cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 --2- 172.21.15.97:0/2178829469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240cc4c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f260c0098a0 tx=0x7f260c02b8f0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 -- 172.21.15.97:0/2178829469 shutdown_connections 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 --2- 172.21.15.97:0/2178829469 >> v2:172.21.15.188:3300/0 conn(0x7f262415edd0 0x7f262415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 --2- 172.21.15.97:0/2178829469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240cc4c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 -- 172.21.15.97:0/2178829469 >> 172.21.15.97:0/2178829469 conn(0x7f26240999c0 msgr2=0x7f2624099da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.902+0000 7f2628a00640 1 -- 172.21.15.97:0/2178829469 shutdown_connections 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.903+0000 7f2628a00640 1 -- 172.21.15.97:0/2178829469 wait complete. 2024-01-31T05:04:27.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.904+0000 7f2628a00640 1 Processor -- start 2024-01-31T05:04:27.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.904+0000 7f2628a00640 1 -- start start 2024-01-31T05:04:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2628a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2628a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262415b270 0x7f262415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2628a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262415edd0 0x7f26240c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2628a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c79b0 con 0x7f262409a4b0 2024-01-31T05:04:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2628a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c9e60 con 0x7f262415edd0 2024-01-31T05:04:27.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2628a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c9500 con 0x7f262415b270 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2622800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2622800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37942/0 (socket says 172.21.15.97:37942) 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.905+0000 7f2622800640 1 -- 172.21.15.97:0/806863173 learned_addr learned my addr 172.21.15.97:0/806863173 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2621e00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262415b270 0x7f262415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2622800640 1 -- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262415b270 msgr2=0x7f262415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2622800640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262415b270 0x7f262415a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2622800640 1 -- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262415edd0 msgr2=0x7f26240c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2623200640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262415edd0 0x7f26240c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2622800640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262415edd0 0x7f26240c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2622800640 1 -- 172.21.15.97:0/806863173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0094d0 con 0x7f262409a4b0 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2623200640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262415edd0 0x7f26240c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f2622800640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240c4a20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c7a0 tx=0x7f261c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:27.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c003d80 con 0x7f262409a4b0 2024-01-31T05:04:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c005070 con 0x7f262409a4b0 2024-01-31T05:04:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.906+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c005420 con 0x7f262409a4b0 2024-01-31T05:04:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.907+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26240c76d0 con 0x7f262409a4b0 2024-01-31T05:04:27.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.907+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240c9a80 con 0x7f262409a4b0 2024-01-31T05:04:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.908+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624067dc0 con 0x7f262409a4b0 2024-01-31T05:04:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.909+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f261c005580 con 0x7f262409a4b0 2024-01-31T05:04:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.909+0000 7f261aa00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25fc06aed0 0x7f25fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.909+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f261c088be0 con 0x7f262409a4b0 2024-01-31T05:04:27.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.910+0000 7f2621e00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25fc06aed0 0x7f25fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:27.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.910+0000 7f2621e00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25fc06aed0 0x7f25fc06d390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f260c036000 tx=0x7f260c03c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:27.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:27.914+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f261c053970 con 0x7f262409a4b0 2024-01-31T05:04:28.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.140+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26240b6e70 con 0x7f262409a4b0 2024-01-31T05:04:28.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.142+0000 7f261aa00640 1 -- 172.21.15.97:0/806863173 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f261c056f90 con 0x7f262409a4b0 2024-01-31T05:04:28.141 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:28.141 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-31T05:03:38.646144+0000","modified":"2024-01-31T05:03:41.747278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3986947126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3986947126},{"type":"v1","addr":"172.21.15.97:6837","nonce":3986947126}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-31T05:04:28.142 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2024-01-31T05:04:28.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25fc06aed0 msgr2=0x7f25fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25fc06aed0 0x7f25fc06d390 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f260c036000 tx=0x7f260c03c040 comp rx=0 tx=0).stop 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 msgr2=0x7f26240c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240c4a20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c7a0 tx=0x7f261c00cc60 comp rx=0 tx=0).stop 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 shutdown_connections 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262415edd0 0x7f26240c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262415b270 0x7f262415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f25fc06aed0 0x7f25fc06d390 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 --2- 172.21.15.97:0/806863173 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f262409a4b0 0x7f26240c4a20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 >> 172.21.15.97:0/806863173 conn(0x7f26240999c0 msgr2=0x7f26240cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 shutdown_connections 2024-01-31T05:04:28.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.144+0000 7f2628a00640 1 -- 172.21.15.97:0/806863173 wait complete. 2024-01-31T05:04:28.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:28.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.358+0000 7f791aa00640 1 Processor -- start 2024-01-31T05:04:28.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.358+0000 7f791aa00640 1 -- start start 2024-01-31T05:04:28.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.360+0000 7f791aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 0x7f791406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.360+0000 7f791aa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791415aef0 con 0x7f791415a600 2024-01-31T05:04:28.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.360+0000 7f791aa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791406c8b0 con 0x7f79140b1d10 2024-01-31T05:04:28.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.360+0000 7f791aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791415b120 con 0x7f791415aa60 2024-01-31T05:04:28.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.360+0000 7f790be00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f791415a600 0x7f791406d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49360/0 (socket says 172.21.15.97:49360) 2024-01-31T05:04:28.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.360+0000 7f790be00640 1 -- 172.21.15.97:0/895120890 learned_addr learned my addr 172.21.15.97:0/895120890 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7918a00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 0x7f791406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090380011 0 0) 0x7f791415aef0 con 0x7f791415a600 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f4005730 con 0x7f791415a600 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 686894346 0 0) 0x7f78f4005730 con 0x7f791415a600 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 msgr2=0x7f791406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 0x7f791406d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 >> v1:172.21.15.188:6789/0 conn(0x7f79140b1d10 legacy=0x7f79140b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791415b400 con 0x7f791415a600 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f78fc001670 con 0x7f791415a600 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 >> v1:172.21.15.97:6789/0 conn(0x7f791415a600 legacy=0x7f791406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f4005bb0 0x7f78f4005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f40069d0 0x7f78f4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 0x7f78f4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791415b120 con 0x7f78f4005bb0 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f400dc70 con 0x7f78f40069d0 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.361+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f400de30 con 0x7f791415aa60 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7913e00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f4005bb0 0x7f78f4005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f790be00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f40069d0 0x7f78f4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7918a00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 0x7f78f4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7913e00640 1 -- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 msgr2=0x7f78f4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7913e00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f791415aa60 0x7f78f4007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7913e00640 1 -- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f40069d0 msgr2=0x7f78f4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7913e00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78f40069d0 0x7f78f4006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.362+0000 7f7913e00640 1 -- 172.21.15.97:0/895120890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4005730 con 0x7f78f4005bb0 2024-01-31T05:04:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.363+0000 7f7913e00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f4005bb0 0x7f78f4005fd0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7904001720 tx=0x7f78f4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58b3bcab9daf117 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:28.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.363+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040052d0 con 0x7f78f4005bb0 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.363+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790402e050 con 0x7f78f4005bb0 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.363+0000 7f7913400640 1 -- 172.21.15.97:0/895120890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040035f0 con 0x7f78f4005bb0 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f791aa00640 1 -- 172.21.15.97:0/895120890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f4005bb0 msgr2=0x7f78f4005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f791aa00640 1 --2- 172.21.15.97:0/895120890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f4005bb0 0x7f78f4005fd0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f7904001720 tx=0x7f78f4007a00 comp rx=0 tx=0).stop 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f7918a00640 1 -- 172.21.15.97:0/895120890 reap_dead start 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f791aa00640 1 -- 172.21.15.97:0/895120890 shutdown_connections 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f791aa00640 1 -- 172.21.15.97:0/895120890 >> 172.21.15.97:0/895120890 conn(0x7f79140b0a10 msgr2=0x7f79140b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f791aa00640 1 -- 172.21.15.97:0/895120890 shutdown_connections 2024-01-31T05:04:28.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.364+0000 7f791aa00640 1 -- 172.21.15.97:0/895120890 wait complete. 2024-01-31T05:04:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.366+0000 7f791aa00640 1 Processor -- start 2024-01-31T05:04:28.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.366+0000 7f791aa00640 1 -- start start 2024-01-31T05:04:28.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f791aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f4005bb0 0x7f791415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f791aa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 0x7f791415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f791aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7914152d80 0x7f7914154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f791aa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791415d750 con 0x7f7914153610 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f791aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914153e20 con 0x7f7914152d80 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f791aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791415c170 con 0x7f78f4005bb0 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f790be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 0x7f791415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f790be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 0x7f791415d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37956/0 (socket says 172.21.15.97:37956) 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f790be00640 1 -- 172.21.15.97:0/4205466548 learned_addr learned my addr 172.21.15.97:0/4205466548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.367+0000 7f7913e00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f4005bb0 0x7f791415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f790be00640 1 -- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f4005bb0 msgr2=0x7f791415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f790be00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f4005bb0 0x7f791415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f790be00640 1 -- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7914152d80 msgr2=0x7f7914154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f790be00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7914152d80 0x7f7914154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f790be00640 1 -- 172.21.15.97:0/4205466548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140adb80 con 0x7f7914153610 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f790be00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 0x7f791415d210 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00e730 tx=0x7f78fc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00c640 con 0x7f7914153610 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc00c7c0 con 0x7f7914153610 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00cb70 con 0x7f7914153610 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791415c450 con 0x7f7914153610 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.368+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791415ca10 con 0x7f7914153610 2024-01-31T05:04:28.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.369+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79140677c0 con 0x7f7914153610 2024-01-31T05:04:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.370+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f78fc00ccd0 con 0x7f7914153610 2024-01-31T05:04:28.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.371+0000 7f7910c00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f78ec06aed0 0x7f78ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.371+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f78fc089600 con 0x7f7914153610 2024-01-31T05:04:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.371+0000 7f7913e00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f78ec06aed0 0x7f78ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.372+0000 7f7913e00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f78ec06aed0 0x7f78ec06d390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7904033000 tx=0x7f7904009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:28.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.375+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f78fc0542f0 con 0x7f7914153610 2024-01-31T05:04:28.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.569+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f791406e740 con 0x7f7914153610 2024-01-31T05:04:28.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.572+0000 7f7910c00640 1 -- 172.21.15.97:0/4205466548 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f78fc057910 con 0x7f7914153610 2024-01-31T05:04:28.572 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:28.572 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:03:40.216415+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-31T05:03:36.506793+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-31T05:03:37.685194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f78ec06aed0 msgr2=0x7f78ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f78ec06aed0 0x7f78ec06d390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7904033000 tx=0x7f7904009030 comp rx=0 tx=0).stop 2024-01-31T05:04:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 msgr2=0x7f791415d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 0x7f791415d210 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00e730 tx=0x7f78fc00ebf0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 shutdown_connections 2024-01-31T05:04:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7914152d80 0x7f7914154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f78ec06aed0 0x7f78ec06d390 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7914153610 0x7f791415d210 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 --2- 172.21.15.97:0/4205466548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78f4005bb0 0x7f791415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 >> 172.21.15.97:0/4205466548 conn(0x7f79140b0a10 msgr2=0x7f79140ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.575+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 shutdown_connections 2024-01-31T05:04:28.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.576+0000 7f791aa00640 1 -- 172.21.15.97:0/4205466548 wait complete. 2024-01-31T05:04:28.594 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-31T05:04:28.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.802+0000 7fc99f800640 1 Processor -- start 2024-01-31T05:04:28.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.802+0000 7fc99f800640 1 -- start start 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.803+0000 7fc99f800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815a350 0x7fc9980abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.803+0000 7fc99f800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc99815a730 0x7fc99806da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc99f800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99815abc0 con 0x7fc99806dfc0 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc99f800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9980b2c90 con 0x7fc99815a730 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc99f800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99815aea0 con 0x7fc99815a350 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc99815a730 0x7fc99806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc99815a730 0x7fc99806da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37960/0 (socket says 172.21.15.97:37960) 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 -- 172.21.15.97:0/824553446 learned_addr learned my addr 172.21.15.97:0/824553446 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc99ce00640 1 --2- 172.21.15.97:0/824553446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815a350 0x7fc9980abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 -- 172.21.15.97:0/824553446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815a350 msgr2=0x7fc9980abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:28.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 --2- 172.21.15.97:0/824553446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815a350 0x7fc9980abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 -- 172.21.15.97:0/824553446 >> v1:172.21.15.188:6789/0 conn(0x7fc99806dfc0 legacy=0x7fc9980cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:28.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.804+0000 7fc997e00640 1 -- 172.21.15.97:0/824553446 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99815b180 con 0x7fc99815a730 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.805+0000 7fc997e00640 1 --2- 172.21.15.97:0/824553446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815a730 0x7fc99806da80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc980009870 tx=0x7fc98002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60649cf55994c62 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc997400640 1 -- 172.21.15.97:0/824553446 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980031640 con 0x7fc99815a730 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc997400640 1 -- 172.21.15.97:0/824553446 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9800317a0 con 0x7fc99815a730 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc997400640 1 -- 172.21.15.97:0/824553446 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980031b30 con 0x7fc99815a730 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 -- 172.21.15.97:0/824553446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815a730 msgr2=0x7fc99806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 --2- 172.21.15.97:0/824553446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815a730 0x7fc99806da80 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc980009870 tx=0x7fc98002b8f0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 -- 172.21.15.97:0/824553446 shutdown_connections 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 --2- 172.21.15.97:0/824553446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc99815a730 0x7fc99806da80 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 --2- 172.21.15.97:0/824553446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99815a350 0x7fc9980abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 -- 172.21.15.97:0/824553446 >> 172.21.15.97:0/824553446 conn(0x7fc9980b0a10 msgr2=0x7fc9980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 -- 172.21.15.97:0/824553446 shutdown_connections 2024-01-31T05:04:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.806+0000 7fc99f800640 1 -- 172.21.15.97:0/824553446 wait complete. 2024-01-31T05:04:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.808+0000 7fc99f800640 1 Processor -- start 2024-01-31T05:04:28.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.808+0000 7fc99f800640 1 -- start start 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99f800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99806dfc0 0x7fc9981579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99f800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99815a350 0x7fc9981f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99f800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 0x7fc9981f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99f800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980ca8a0 con 0x7fc9980ccd70 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99f800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980cd270 con 0x7fc99815a350 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99f800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99815d610 con 0x7fc99806dfc0 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 0x7fc9981f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99d800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 0x7fc9981f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37964/0 (socket says 172.21.15.97:37964) 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.809+0000 7fc99d800640 1 -- 172.21.15.97:0/926439548 learned_addr learned my addr 172.21.15.97:0/926439548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:28.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99ce00640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99806dfc0 0x7fc9981579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99d800640 1 -- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99806dfc0 msgr2=0x7fc9981579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99d800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99806dfc0 0x7fc9981579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99d800640 1 -- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99815a350 msgr2=0x7fc9981f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc997e00640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99815a350 0x7fc9981f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99d800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99815a350 0x7fc9981f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99d800640 1 -- 172.21.15.97:0/926439548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800094d0 con 0x7fc9980ccd70 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99ce00640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99806dfc0 0x7fc9981579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99d800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 0x7fc9981f68e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c780 tx=0x7fc99000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc990004b60 con 0x7fc9980ccd70 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99000ce90 con 0x7fc9980ccd70 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000a850 con 0x7fc9980ccd70 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.810+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99815d8f0 con 0x7fc9980ccd70 2024-01-31T05:04:28.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.811+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9980cafd0 con 0x7fc9980ccd70 2024-01-31T05:04:28.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.812+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9980cacf0 con 0x7fc9980ccd70 2024-01-31T05:04:28.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.813+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fc99000a9b0 con 0x7fc9980ccd70 2024-01-31T05:04:28.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.813+0000 7fc994c00640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006af00 0x7fc97006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:28.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.813+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc990089820 con 0x7fc9980ccd70 2024-01-31T05:04:28.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.814+0000 7fc99ce00640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006af00 0x7fc97006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:28.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.814+0000 7fc99ce00640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006af00 0x7fc97006d3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc988009820 tx=0x7fc988008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:28.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:28.817+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fc9980cacf0 con 0x7fc9980ccd70 2024-01-31T05:04:29.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.037+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc9980cacf0 con 0x7fc9980ccd70 2024-01-31T05:04:29.037 INFO:tasks.ceph.mon.a.smithi097.stderr:2024-01-31T05:04:29.038+0000 7f85e2800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-31T05:04:29.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.042+0000 7fc994c00640 1 -- 172.21.15.97:0/926439548 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc9980cacf0 con 0x7fc9980ccd70 2024-01-31T05:04:29.042 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-31T05:04:29.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.047+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006af00 msgr2=0x7fc97006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:29.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.047+0000 7fc99f800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006af00 0x7fc97006d3c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc988009820 tx=0x7fc988008040 comp rx=0 tx=0).stop 2024-01-31T05:04:29.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.047+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 msgr2=0x7fc9981f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:29.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.047+0000 7fc99f800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 0x7fc9981f68e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc99000c780 tx=0x7fc99000cc40 comp rx=0 tx=0).stop 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 shutdown_connections 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fc97006af00 0x7fc97006d3c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9980ccd70 0x7fc9981f68e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99815a350 0x7fc9981f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 --2- 172.21.15.97:0/926439548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc99806dfc0 0x7fc9981579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 >> 172.21.15.97:0/926439548 conn(0x7fc9980b0a10 msgr2=0x7fc9980cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 shutdown_connections 2024-01-31T05:04:29.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.048+0000 7fc99f800640 1 -- 172.21.15.97:0/926439548 wait complete. 2024-01-31T05:04:29.055 INFO:tasks.ceph.mds.c.smithi097.stderr: -3161> 2024-01-31T05:03:45.183+0000 7ff578200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-31T05:04:29.073 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-31T05:04:29.108 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2024-01-31T05:04:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.250+0000 7ff00e400640 1 Processor -- start 2024-01-31T05:04:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.251+0000 7ff00e400640 1 -- start start 2024-01-31T05:04:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff00e400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0080ce520 0x7ff0080abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff00e400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00815abc0 con 0x7ff00815a350 2024-01-31T05:04:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff00e400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00815aea0 con 0x7ff00815a730 2024-01-31T05:04:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff00e400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080b2c90 con 0x7ff0080ce520 2024-01-31T05:04:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff007400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff00815a350 0x7ff00806da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49362/0 (socket says 172.21.15.97:49362) 2024-01-31T05:04:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff007400640 1 -- 172.21.15.97:0/4118280311 learned_addr learned my addr 172.21.15.97:0/4118280311 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff007e00640 1 --2- 172.21.15.97:0/4118280311 >> v2:172.21.15.97:3300/0 conn(0x7ff0080ce520 0x7ff0080abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.252+0000 7ff006a00640 1 -- 172.21.15.97:0/4118280311 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703845384 0 0) 0x7ff00815abc0 con 0x7ff00815a350 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.253+0000 7ff006a00640 1 -- 172.21.15.97:0/4118280311 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe8005730 con 0x7ff00815a350 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.253+0000 7ff007e00640 1 -- 172.21.15.97:0/4118280311 >> v1:172.21.15.188:6789/0 conn(0x7ff00815a730 legacy=0x7ff0080cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.253+0000 7ff007e00640 1 -- 172.21.15.97:0/4118280311 >> v1:172.21.15.97:6789/0 conn(0x7ff00815a350 legacy=0x7ff00806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.253+0000 7ff007e00640 1 -- 172.21.15.97:0/4118280311 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00815b180 con 0x7ff0080ce520 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.254+0000 7ff007e00640 1 --2- 172.21.15.97:0/4118280311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080abcf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff000009d90 tx=0x7ff00002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d592c8ced9336466 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.254+0000 7ff006a00640 1 -- 172.21.15.97:0/4118280311 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00002cbd0 con 0x7ff0080ce520 2024-01-31T05:04:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.254+0000 7ff006a00640 1 -- 172.21.15.97:0/4118280311 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00002cd30 con 0x7ff0080ce520 2024-01-31T05:04:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.254+0000 7ff006a00640 1 -- 172.21.15.97:0/4118280311 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00002cea0 con 0x7ff0080ce520 2024-01-31T05:04:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.254+0000 7ff00e400640 1 -- 172.21.15.97:0/4118280311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 msgr2=0x7ff0080abcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:29.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.255+0000 7ff00e400640 1 --2- 172.21.15.97:0/4118280311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080abcf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff000009d90 tx=0x7ff00002be20 comp rx=0 tx=0).stop 2024-01-31T05:04:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.255+0000 7ff00e400640 1 -- 172.21.15.97:0/4118280311 shutdown_connections 2024-01-31T05:04:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.255+0000 7ff00e400640 1 --2- 172.21.15.97:0/4118280311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080abcf0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.255+0000 7ff00e400640 1 -- 172.21.15.97:0/4118280311 >> 172.21.15.97:0/4118280311 conn(0x7ff0080b0a10 msgr2=0x7ff0080b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.255+0000 7ff00e400640 1 -- 172.21.15.97:0/4118280311 shutdown_connections 2024-01-31T05:04:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.255+0000 7ff00e400640 1 -- 172.21.15.97:0/4118280311 wait complete. 2024-01-31T05:04:29.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.256+0000 7ff00e400640 1 Processor -- start 2024-01-31T05:04:29.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.256+0000 7ff00e400640 1 -- start start 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff00e400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff00e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff00815a350 0x7ff00809ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff00e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0080999c0 0x7ff0080cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff00e400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081473e0 con 0x7ff0080ce520 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff00e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081476c0 con 0x7ff00815a350 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff00e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081479a0 con 0x7ff0080999c0 2024-01-31T05:04:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.257+0000 7ff007400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37988/0 (socket says 172.21.15.97:37988) 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 -- 172.21.15.97:0/462466565 learned_addr learned my addr 172.21.15.97:0/462466565 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007e00640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0080999c0 0x7ff0080cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7fefffe00640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff00815a350 0x7ff00809ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 -- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0080999c0 msgr2=0x7ff0080cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0080999c0 0x7ff0080cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 -- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff00815a350 msgr2=0x7ff00809ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff00815a350 0x7ff00809ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 -- 172.21.15.97:0/462466565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000009a70 con 0x7ff0080ce520 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7ff007400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080af120 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7feff800c700 tx=0x7feff800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.258+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8007270 con 0x7ff0080ce520 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.259+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff80073f0 con 0x7ff0080ce520 2024-01-31T05:04:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.259+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8001d40 con 0x7ff0080ce520 2024-01-31T05:04:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.259+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff008147c80 con 0x7ff0080ce520 2024-01-31T05:04:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.259+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0080af660 con 0x7ff0080ce520 2024-01-31T05:04:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.260+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0081481a0 con 0x7ff0080ce520 2024-01-31T05:04:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.261+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7feff80057a0 con 0x7ff0080ce520 2024-01-31T05:04:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.262+0000 7fefff400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fefe006aed0 0x7fefe006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.262+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6115+0+0 (secure 0 0 0) 0x7feff8088e40 con 0x7ff0080ce520 2024-01-31T05:04:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.262+0000 7fefffe00640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fefe006aed0 0x7fefe006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.262+0000 7fefffe00640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fefe006aed0 0x7fefe006d390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff008071f40 tx=0x7feff0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.266+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff0081481a0 con 0x7ff0080ce520 2024-01-31T05:04:29.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:29.492+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff0081481a0 con 0x7ff0080ce520 2024-01-31T05:04:30.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.045+0000 7fefff400640 1 -- 172.21.15.97:0/462466565 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff0081481a0 con 0x7ff0080ce520 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fefe006aed0 msgr2=0x7fefe006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fefe006aed0 0x7fefe006d390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff008071f40 tx=0x7feff0008040 comp rx=0 tx=0).stop 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 msgr2=0x7ff0080af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080af120 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7feff800c700 tx=0x7feff800cbc0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 shutdown_connections 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0080999c0 0x7ff0080cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff00815a350 0x7ff00809ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fefe006aed0 0x7fefe006d390 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 --2- 172.21.15.97:0/462466565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0080ce520 0x7ff0080af120 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 >> 172.21.15.97:0/462466565 conn(0x7ff0080b0a10 msgr2=0x7ff0080b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.049+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 shutdown_connections 2024-01-31T05:04:30.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.050+0000 7ff00e400640 1 -- 172.21.15.97:0/462466565 wait complete. 2024-01-31T05:04:30.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-31T05:04:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.319+0000 7f3ca6a00640 1 Processor -- start 2024-01-31T05:04:30.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.319+0000 7f3ca6a00640 1 -- start start 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.320+0000 7f3ca6a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ca00ad780 0x7f3ca00ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.320+0000 7f3ca6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca0070910 0x7f3ca0070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.320+0000 7f3ca6a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca006e4b0 con 0x7f3ca00712b0 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3ca6a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca00adb60 con 0x7f3ca00ad780 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3ca6a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0072720 con 0x7f3ca0070910 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ca00ad780 0x7f3ca00ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3ca00ad780 0x7f3ca00ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37996/0 (socket says 172.21.15.97:37996) 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 -- 172.21.15.97:0/3057258532 learned_addr learned my addr 172.21.15.97:0/3057258532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 -- 172.21.15.97:0/3057258532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca0070910 msgr2=0x7f3ca0070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 --2- 172.21.15.97:0/3057258532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca0070910 0x7f3ca0070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 -- 172.21.15.97:0/3057258532 >> v1:172.21.15.188:6789/0 conn(0x7f3ca00712b0 legacy=0x7f3ca015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.321+0000 7f3c9fe00640 1 -- 172.21.15.97:0/3057258532 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca006dea0 con 0x7f3ca00ad780 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.322+0000 7f3c9fe00640 1 --2- 172.21.15.97:0/3057258532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca00ad780 0x7f3ca00ae220 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009870 tx=0x7f3c9002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3686440e7900d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.322+0000 7f3c9ea00640 1 -- 172.21.15.97:0/3057258532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c900317c0 con 0x7f3ca00ad780 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.322+0000 7f3c9ea00640 1 -- 172.21.15.97:0/3057258532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90031920 con 0x7f3ca00ad780 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.322+0000 7f3c9ea00640 1 -- 172.21.15.97:0/3057258532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90031cb0 con 0x7f3ca00ad780 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 -- 172.21.15.97:0/3057258532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca00ad780 msgr2=0x7f3ca00ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/3057258532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca00ad780 0x7f3ca00ae220 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009870 tx=0x7f3c9002b860 comp rx=0 tx=0).stop 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 -- 172.21.15.97:0/3057258532 shutdown_connections 2024-01-31T05:04:30.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/3057258532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca0070910 0x7f3ca0070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/3057258532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca00ad780 0x7f3ca00ae220 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 -- 172.21.15.97:0/3057258532 >> 172.21.15.97:0/3057258532 conn(0x7f3ca00cede0 msgr2=0x7f3ca00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:30.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 -- 172.21.15.97:0/3057258532 shutdown_connections 2024-01-31T05:04:30.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.323+0000 7f3ca6a00640 1 -- 172.21.15.97:0/3057258532 wait complete. 2024-01-31T05:04:30.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.325+0000 7f3ca6a00640 1 Processor -- start 2024-01-31T05:04:30.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.325+0000 7f3ca6a00640 1 -- start start 2024-01-31T05:04:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.326+0000 7f3ca6a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 0x7f3ca00b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.326+0000 7f3ca6a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00712b0 0x7f3ca00c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:30.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.326+0000 7f3ca6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca00ad780 0x7f3ca009bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.326+0000 7f3ca6a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00b4800 con 0x7f3ca0070910 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.326+0000 7f3ca6a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca01dc880 con 0x7f3ca00712b0 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.326+0000 7f3ca6a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca009b6d0 con 0x7f3ca00ad780 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 0x7f3ca00b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 0x7f3ca00b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38012/0 (socket says 172.21.15.97:38012) 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 -- 172.21.15.97:0/1752640972 learned_addr learned my addr 172.21.15.97:0/1752640972 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 -- 172.21.15.97:0/1752640972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca00ad780 msgr2=0x7f3ca009bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca00ad780 0x7f3ca009bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 -- 172.21.15.97:0/1752640972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00712b0 msgr2=0x7f3ca00c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00712b0 0x7f3ca00c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 -- 172.21.15.97:0/1752640972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900094d0 con 0x7f3ca0070910 2024-01-31T05:04:30.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c9fe00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 0x7f3ca00b3db0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009870 tx=0x7f3c900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.327+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90032860 con 0x7f3ca0070910 2024-01-31T05:04:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.328+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c900329c0 con 0x7f3ca0070910 2024-01-31T05:04:30.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.328+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90032d50 con 0x7f3ca0070910 2024-01-31T05:04:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.328+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca009ab00 con 0x7f3ca0070910 2024-01-31T05:04:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.328+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca009b080 con 0x7f3ca0070910 2024-01-31T05:04:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.329+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca0067830 con 0x7f3ca0070910 2024-01-31T05:04:30.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.330+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3c9004e450 con 0x7f3ca0070910 2024-01-31T05:04:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.330+0000 7f3c97e00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3c7806aed0 0x7f3c7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.330+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f3c900abc10 con 0x7f3ca0070910 2024-01-31T05:04:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.331+0000 7f3c9f400640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3c7806aed0 0x7f3c7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:30.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.331+0000 7f3c9f400640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3c7806aed0 0x7f3c7806d390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00b4ae0 tx=0x7f3c88006c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:30.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.335+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3c900768a0 con 0x7f3ca0070910 2024-01-31T05:04:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:30.511+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ca0070660 con 0x7f3ca0070910 2024-01-31T05:04:31.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.056+0000 7f3c97e00640 1 -- 172.21.15.97:0/1752640972 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3c90079ec0 con 0x7f3ca0070910 2024-01-31T05:04:31.056 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-01-31T05:04:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3c7806aed0 msgr2=0x7f3c7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3c7806aed0 0x7f3c7806d390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00b4ae0 tx=0x7f3c88006c90 comp rx=0 tx=0).stop 2024-01-31T05:04:31.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 msgr2=0x7f3ca00b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 0x7f3ca00b3db0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009870 tx=0x7f3c900046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 shutdown_connections 2024-01-31T05:04:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca00ad780 0x7f3ca009bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00712b0 0x7f3ca00c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f3c7806aed0 0x7f3c7806d390 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 --2- 172.21.15.97:0/1752640972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ca0070910 0x7f3ca00b3db0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.059+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 >> 172.21.15.97:0/1752640972 conn(0x7f3ca00cede0 msgr2=0x7f3ca00c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.060+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 shutdown_connections 2024-01-31T05:04:31.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.060+0000 7f3ca6a00640 1 -- 172.21.15.97:0/1752640972 wait complete. 2024-01-31T05:04:31.081 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-31T05:04:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.292+0000 7f7967c00640 1 Processor -- start 2024-01-31T05:04:31.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.292+0000 7f7967c00640 1 -- start start 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f7967c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f79600c9430 0x7f79600ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f7967c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79600cab10 con 0x7f796006e1e0 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f7967c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79600c99f0 con 0x7f796015a230 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f7967c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796006c720 con 0x7f79600c9430 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f795fe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f796006e1e0 0x7f7960071450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49368/0 (socket says 172.21.15.97:49368) 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f795fe00640 1 -- 172.21.15.97:0/1417622856 learned_addr learned my addr 172.21.15.97:0/1417622856 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954700733 0 0) 0x7f79600cab10 con 0x7f796006e1e0 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7940003520 con 0x7f796006e1e0 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f7965c00640 1 --2- 172.21.15.97:0/1417622856 >> v2:172.21.15.188:3300/0 conn(0x7f79600c9430 0x7f79600ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.294+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2763929674 0 0) 0x7f7940003520 con 0x7f796006e1e0 2024-01-31T05:04:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 >> v2:172.21.15.188:3300/0 conn(0x7f79600c9430 msgr2=0x7f79600ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 --2- 172.21.15.97:0/1417622856 >> v2:172.21.15.188:3300/0 conn(0x7f79600c9430 0x7f79600ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 >> v1:172.21.15.188:6789/0 conn(0x7f796015a230 legacy=0x7f796006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600ae050 con 0x7f796006e1e0 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f7948001670 con 0x7f796006e1e0 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 >> v1:172.21.15.97:6789/0 conn(0x7f796006e1e0 legacy=0x7f7960071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7940003970 0x7f7940003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79400047c0 0x7f7940004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79400052a0 0x7f794000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796006c720 con 0x7f7940003970 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794000e360 con 0x7f79400052a0 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794000e520 con 0x7f79400047c0 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965200640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7940003970 0x7f7940003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f795fe00640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79400047c0 0x7f7940004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965200640 1 -- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79400047c0 msgr2=0x7f7940004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965200640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79400047c0 0x7f7940004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965200640 1 -- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79400052a0 msgr2=0x7f794000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965200640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79400052a0 0x7f794000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965200640 1 -- 172.21.15.97:0/1417622856 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940003520 con 0x7f7940003970 2024-01-31T05:04:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.295+0000 7f7965c00640 1 -- 172.21.15.97:0/1417622856 reap_dead start 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.296+0000 7f7965200640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7940003970 0x7f7940003d70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7950003d80 tx=0x7f79400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf6e4082c1111f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7950032600 con 0x7f7940003970 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795002e050 con 0x7f7940003970 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f795f400640 1 -- 172.21.15.97:0/1417622856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79500329e0 con 0x7f7940003970 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 -- 172.21.15.97:0/1417622856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7940003970 msgr2=0x7f7940003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7940003970 0x7f7940003d70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7950003d80 tx=0x7f79400057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 -- 172.21.15.97:0/1417622856 shutdown_connections 2024-01-31T05:04:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 --2- 172.21.15.97:0/1417622856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7940003970 0x7f7940003d70 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 -- 172.21.15.97:0/1417622856 >> 172.21.15.97:0/1417622856 conn(0x7f79600cede0 msgr2=0x7f79600cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 -- 172.21.15.97:0/1417622856 shutdown_connections 2024-01-31T05:04:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.297+0000 7f7967c00640 1 -- 172.21.15.97:0/1417622856 wait complete. 2024-01-31T05:04:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.299+0000 7f7967c00640 1 Processor -- start 2024-01-31T05:04:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.299+0000 7f7967c00640 1 -- start start 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7967c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7940003970 0x7f79601efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7967c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79601f1e30 0x7f79601ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7967c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 0x7f7960157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7967c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796014bc00 con 0x7f79600c8590 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7967c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79601529c0 con 0x7f7940003970 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7967c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79601eff40 con 0x7f79601f1e30 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7965c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 0x7f7960157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7965c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 0x7f7960157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38022/0 (socket says 172.21.15.97:38022) 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.300+0000 7f7965c00640 1 -- 172.21.15.97:0/2749742608 learned_addr learned my addr 172.21.15.97:0/2749742608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965200640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7940003970 0x7f79601efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f795fe00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79601f1e30 0x7f79601ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965c00640 1 -- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79601f1e30 msgr2=0x7f79601ef2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79601f1e30 0x7f79601ef2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965c00640 1 -- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7940003970 msgr2=0x7f79601efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7940003970 0x7f79601efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965c00640 1 -- 172.21.15.97:0/2749742608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79601ef7f0 con 0x7f79600c8590 2024-01-31T05:04:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f795fe00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79601f1e30 0x7f79601ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.301+0000 7f7965c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 0x7f7960157150 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f79600b1850 tx=0x7f7958007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.302+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800ec50 con 0x7f79600c8590 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.302+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795800edb0 con 0x7f79600c8590 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.302+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958014660 con 0x7f79600c8590 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.302+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79601f23a0 con 0x7f79600c8590 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.302+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79601f2960 con 0x7f79600c8590 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.303+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796014bd70 con 0x7f79600c8590 2024-01-31T05:04:31.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.304+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f79580147c0 con 0x7f79600c8590 2024-01-31T05:04:31.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.305+0000 7f795cc00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f793806aed0 0x7f793806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:31.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.305+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5677+0+0 (secure 0 0 0) 0x7f7958089a20 con 0x7f79600c8590 2024-01-31T05:04:31.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.305+0000 7f7965200640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f793806aed0 0x7f793806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:31.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.306+0000 7f7965200640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f793806aed0 0x7f793806d390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7950034000 tx=0x7f7950009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:31.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.309+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f796014bd70 con 0x7f79600c8590 2024-01-31T05:04:31.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:31.500+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f796014bd70 con 0x7f79600c8590 2024-01-31T05:04:32.059 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.058+0000 7f795cc00640 1 -- 172.21.15.97:0/2749742608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f796014bd70 con 0x7f79600c8590 2024-01-31T05:04:32.059 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-01-31T05:04:32.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.063+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f793806aed0 msgr2=0x7f793806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.063+0000 7f7967c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f793806aed0 0x7f793806d390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7950034000 tx=0x7f7950009030 comp rx=0 tx=0).stop 2024-01-31T05:04:32.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.063+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 msgr2=0x7f7960157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.063+0000 7f7967c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 0x7f7960157150 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f79600b1850 tx=0x7f7958007f40 comp rx=0 tx=0).stop 2024-01-31T05:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.064+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 shutdown_connections 2024-01-31T05:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.064+0000 7f7967c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f793806aed0 0x7f793806d390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.064+0000 7f7967c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79600c8590 0x7f7960157150 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.064+0000 7f7967c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79601f1e30 0x7f79601ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.064+0000 7f7967c00640 1 --2- 172.21.15.97:0/2749742608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7940003970 0x7f79601efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.064+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 >> 172.21.15.97:0/2749742608 conn(0x7f79600cede0 msgr2=0x7f79600ad810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.065+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 shutdown_connections 2024-01-31T05:04:32.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.065+0000 7f7967c00640 1 -- 172.21.15.97:0/2749742608 wait complete. 2024-01-31T05:04:32.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-31T05:04:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.288+0000 7fe86d200640 1 Processor -- start 2024-01-31T05:04:32.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.288+0000 7fe86d200640 1 -- start start 2024-01-31T05:04:32.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe86d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe86815d590 0x7fe86815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe86d200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8680cff20 0x7fe868161790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe86d200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8680cded0 con 0x7fe8680cfae0 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe86d200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86815d970 con 0x7fe8680cff20 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe86d200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680cf210 con 0x7fe86815d590 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe866800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8680cff20 0x7fe868161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe866800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8680cff20 0x7fe868161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38028/0 (socket says 172.21.15.97:38028) 2024-01-31T05:04:32.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.290+0000 7fe866800640 1 -- 172.21.15.97:0/992099687 learned_addr learned my addr 172.21.15.97:0/992099687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.291+0000 7fe866800640 1 -- 172.21.15.97:0/992099687 >> v2:172.21.15.188:3300/0 conn(0x7fe86815d590 msgr2=0x7fe86815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.291+0000 7fe865e00640 1 --2- 172.21.15.97:0/992099687 >> v2:172.21.15.188:3300/0 conn(0x7fe86815d590 0x7fe86815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.291+0000 7fe866800640 1 --2- 172.21.15.97:0/992099687 >> v2:172.21.15.188:3300/0 conn(0x7fe86815d590 0x7fe86815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.291+0000 7fe866800640 1 -- 172.21.15.97:0/992099687 >> v1:172.21.15.188:6789/0 conn(0x7fe8680cfae0 legacy=0x7fe868161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.291+0000 7fe866800640 1 -- 172.21.15.97:0/992099687 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680ce170 con 0x7fe8680cff20 2024-01-31T05:04:32.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.291+0000 7fe865e00640 1 --2- 172.21.15.97:0/992099687 >> v2:172.21.15.188:3300/0 conn(0x7fe86815d590 0x7fe86815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.292+0000 7fe866800640 1 --2- 172.21.15.97:0/992099687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8680cff20 0x7fe868161790 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe86808b830 tx=0x7fe86002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5941401b30abfc9a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.292+0000 7fe865400640 1 -- 172.21.15.97:0/992099687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86002cb20 con 0x7fe8680cff20 2024-01-31T05:04:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.292+0000 7fe865400640 1 -- 172.21.15.97:0/992099687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86002cc80 con 0x7fe8680cff20 2024-01-31T05:04:32.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.292+0000 7fe865400640 1 -- 172.21.15.97:0/992099687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600343f0 con 0x7fe8680cff20 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 -- 172.21.15.97:0/992099687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8680cff20 msgr2=0x7fe868161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 --2- 172.21.15.97:0/992099687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8680cff20 0x7fe868161790 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe86808b830 tx=0x7fe86002bf80 comp rx=0 tx=0).stop 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 -- 172.21.15.97:0/992099687 shutdown_connections 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 --2- 172.21.15.97:0/992099687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8680cff20 0x7fe868161790 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 --2- 172.21.15.97:0/992099687 >> v2:172.21.15.188:3300/0 conn(0x7fe86815d590 0x7fe86815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 -- 172.21.15.97:0/992099687 >> 172.21.15.97:0/992099687 conn(0x7fe86809c560 msgr2=0x7fe8680d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 -- 172.21.15.97:0/992099687 shutdown_connections 2024-01-31T05:04:32.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.293+0000 7fe86d200640 1 -- 172.21.15.97:0/992099687 wait complete. 2024-01-31T05:04:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.295+0000 7fe86d200640 1 Processor -- start 2024-01-31T05:04:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.296+0000 7fe86d200640 1 -- start start 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe86d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8680cfae0 0x7fe868154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe86d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8680cff20 0x7fe8680c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe86d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 0x7fe8680b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe86d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680c7d70 con 0x7fe86815d590 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe86d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680b6f60 con 0x7fe8680cfae0 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe86d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680c7a90 con 0x7fe8680cff20 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe866800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 0x7fe8680b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe866800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 0x7fe8680b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38044/0 (socket says 172.21.15.97:38044) 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe866800640 1 -- 172.21.15.97:0/2435280768 learned_addr learned my addr 172.21.15.97:0/2435280768 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:32.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe865e00640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8680cfae0 0x7fe868154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe85fe00640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8680cff20 0x7fe8680c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.297+0000 7fe866800640 1 -- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8680cff20 msgr2=0x7fe8680c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe866800640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8680cff20 0x7fe8680c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe866800640 1 -- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8680cfae0 msgr2=0x7fe868154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe866800640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8680cfae0 0x7fe868154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe866800640 1 -- 172.21.15.97:0/2435280768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe860009b30 con 0x7fe86815d590 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe866800640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 0x7fe8680b07a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe860036040 tx=0x7fe860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860007e70 con 0x7fe86815d590 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860009510 con 0x7fe86815d590 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8680b7330 con 0x7fe86815d590 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86003e690 con 0x7fe86815d590 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.298+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8680b78f0 con 0x7fe86815d590 2024-01-31T05:04:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.300+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8680b7b10 con 0x7fe86815d590 2024-01-31T05:04:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.301+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fe86003e7f0 con 0x7fe86815d590 2024-01-31T05:04:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.302+0000 7fe85e000640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe84006aed0 0x7fe84006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.302+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe8600ac0f0 con 0x7fe86815d590 2024-01-31T05:04:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.302+0000 7fe865e00640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe84006aed0 0x7fe84006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.303+0000 7fe865e00640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe84006aed0 0x7fe84006d390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe85800a950 tx=0x7fe858008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.307+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fe860077170 con 0x7fe86815d590 2024-01-31T05:04:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.501+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe868067830 con 0x7fe86815d590 2024-01-31T05:04:32.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.509+0000 7fe85e000640 1 -- 172.21.15.97:0/2435280768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe86007a790 con 0x7fe86815d590 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.511+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe84006aed0 msgr2=0x7fe84006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.511+0000 7fe86d200640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe84006aed0 0x7fe84006d390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe85800a950 tx=0x7fe858008040 comp rx=0 tx=0).stop 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.511+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 msgr2=0x7fe8680b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.511+0000 7fe86d200640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 0x7fe8680b07a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe860036040 tx=0x7fe860004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 shutdown_connections 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fe84006aed0 0x7fe84006d390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe86815d590 0x7fe8680b07a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8680cff20 0x7fe8680c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 --2- 172.21.15.97:0/2435280768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8680cfae0 0x7fe868154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 >> 172.21.15.97:0/2435280768 conn(0x7fe86809c560 msgr2=0x7fe8680b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 shutdown_connections 2024-01-31T05:04:32.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.512+0000 7fe86d200640 1 -- 172.21.15.97:0/2435280768 wait complete. 2024-01-31T05:04:32.532 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-01-31T05:04:32.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.738+0000 7fa5d2c00640 1 Processor -- start 2024-01-31T05:04:32.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.738+0000 7fa5d2c00640 1 -- start start 2024-01-31T05:04:32.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.740+0000 7fa5d2c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5cc0b1d10 0x7fa5cc0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.740+0000 7fa5d2c00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc15ea70 con 0x7fa5cc15a600 2024-01-31T05:04:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.740+0000 7fa5d2c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc0b2630 con 0x7fa5cc06e0a0 2024-01-31T05:04:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.740+0000 7fa5d2c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc06c8b0 con 0x7fa5cc0b1d10 2024-01-31T05:04:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.740+0000 7fa5cb400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5cc15a600 0x7fa5cc15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:49382/0 (socket says 172.21.15.97:49382) 2024-01-31T05:04:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.740+0000 7fa5cb400640 1 -- 172.21.15.97:0/3560478693 learned_addr learned my addr 172.21.15.97:0/3560478693 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787806382 0 0) 0x7fa5cc15ea70 con 0x7fa5cc15a600 2024-01-31T05:04:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ac003520 con 0x7fa5cc15a600 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5cbe00640 1 --2- 172.21.15.97:0/3560478693 >> v2:172.21.15.188:3300/0 conn(0x7fa5cc0b1d10 0x7fa5cc0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665533374 0 0) 0x7fa5ac003520 con 0x7fa5cc15a600 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 >> v2:172.21.15.188:3300/0 conn(0x7fa5cc0b1d10 msgr2=0x7fa5cc0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 --2- 172.21.15.97:0/3560478693 >> v2:172.21.15.188:3300/0 conn(0x7fa5cc0b1d10 0x7fa5cc0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 >> v1:172.21.15.188:6789/0 conn(0x7fa5cc06e0a0 legacy=0x7fa5cc15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc15b120 con 0x7fa5cc15a600 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa5bc001670 con 0x7fa5cc15a600 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 >> v1:172.21.15.97:6789/0 conn(0x7fa5cc15a600 legacy=0x7fa5cc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5ac003970 0x7fa5ac003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0047c0 0x7fa5ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5ac005180 0x7fa5ac00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc06c8b0 con 0x7fa5ac005180 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac00e2c0 con 0x7fa5ac0047c0 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.741+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac00e480 con 0x7fa5ac003970 2024-01-31T05:04:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5d0c00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5ac005180 0x7fa5ac00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5cb400640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0047c0 0x7fa5ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5cbe00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5ac003970 0x7fa5ac003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5d0c00640 1 -- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5ac003970 msgr2=0x7fa5ac003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5d0c00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5ac003970 0x7fa5ac003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5d0c00640 1 -- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0047c0 msgr2=0x7fa5ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5d0c00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0047c0 0x7fa5ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.742+0000 7fa5d0c00640 1 -- 172.21.15.97:0/3560478693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac003520 con 0x7fa5ac005180 2024-01-31T05:04:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.743+0000 7fa5d0c00640 1 -- 172.21.15.97:0/3560478693 reap_dead start 2024-01-31T05:04:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.743+0000 7fa5d0c00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5ac005180 0x7fa5ac00db90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc06e4b0 tx=0x7fa5ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fe1534f44fae5d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.743+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4006b50 con 0x7fa5ac005180 2024-01-31T05:04:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.743+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4006cd0 con 0x7fa5ac005180 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.743+0000 7fa5caa00640 1 -- 172.21.15.97:0/3560478693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007260 con 0x7fa5ac005180 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 -- 172.21.15.97:0/3560478693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5ac005180 msgr2=0x7fa5ac00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5ac005180 0x7fa5ac00db90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc06e4b0 tx=0x7fa5ac0057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 -- 172.21.15.97:0/3560478693 shutdown_connections 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/3560478693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5ac005180 0x7fa5ac00db90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 -- 172.21.15.97:0/3560478693 >> 172.21.15.97:0/3560478693 conn(0x7fa5cc0b0a10 msgr2=0x7fa5cc0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 -- 172.21.15.97:0/3560478693 shutdown_connections 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.744+0000 7fa5d2c00640 1 -- 172.21.15.97:0/3560478693 wait complete. 2024-01-31T05:04:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.746+0000 7fa5d2c00640 1 Processor -- start 2024-01-31T05:04:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.746+0000 7fa5d2c00640 1 -- start start 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5d2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac005180 0x7fa5cc156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5d2c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 0x7fa5cc15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5d2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5cc159260 0x7fa5cc1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5d2c00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc15e510 con 0x7fa5cc15db50 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5d2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc099a60 con 0x7fa5ac005180 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5d2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc099cd0 con 0x7fa5cc159260 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 0x7fa5cc15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5cb400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 0x7fa5cc15dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38062/0 (socket says 172.21.15.97:38062) 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.747+0000 7fa5cb400640 1 -- 172.21.15.97:0/1510923421 learned_addr learned my addr 172.21.15.97:0/1510923421 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cbe00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac005180 0x7fa5cc156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5d0c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5cc159260 0x7fa5cc1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cb400640 1 -- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5cc159260 msgr2=0x7fa5cc1596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cb400640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5cc159260 0x7fa5cc1596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cb400640 1 -- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac005180 msgr2=0x7fa5cc156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cb400640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac005180 0x7fa5cc156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cb400640 1 -- 172.21.15.97:0/1510923421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0adb80 con 0x7fa5cc15db50 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cbe00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac005180 0x7fa5cc156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5cb400640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 0x7fa5cc15dfd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e7e0 tx=0x7fa5bc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00c8d0 con 0x7fa5cc15db50 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00ca30 con 0x7fa5cc15db50 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc099fb0 con 0x7fa5cc15db50 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.748+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00cde0 con 0x7fa5cc15db50 2024-01-31T05:04:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.749+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc159f90 con 0x7fa5cc15db50 2024-01-31T05:04:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.750+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc09a4d0 con 0x7fa5cc15db50 2024-01-31T05:04:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.750+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa5bc01d460 con 0x7fa5cc15db50 2024-01-31T05:04:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.751+0000 7fa5c3e00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5a406af00 0x7fa5a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.751+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa5bc0894f0 con 0x7fa5cc15db50 2024-01-31T05:04:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.752+0000 7fa5cbe00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5a406af00 0x7fa5a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.752+0000 7fa5cbe00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5a406af00 0x7fa5a406d3c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4006fd0 tx=0x7fa5b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:32.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:32.756+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa5cc09a4d0 con 0x7fa5cc15db50 2024-01-31T05:04:33.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.013+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fa5cc09a4d0 con 0x7fa5cc15db50 2024-01-31T05:04:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.074+0000 7fa5c3e00640 1 -- 172.21.15.97:0/1510923421 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v217) v1 ==== 134+0+0 (secure 0 0 0) 0x7fa5bc0544e0 con 0x7fa5cc15db50 2024-01-31T05:04:33.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5a406af00 msgr2=0x7fa5a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:33.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5a406af00 0x7fa5a406d3c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4006fd0 tx=0x7fa5b4008040 comp rx=0 tx=0).stop 2024-01-31T05:04:33.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 msgr2=0x7fa5cc15dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:33.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 0x7fa5cc15dfd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e7e0 tx=0x7fa5bc00eca0 comp rx=0 tx=0).stop 2024-01-31T05:04:33.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 shutdown_connections 2024-01-31T05:04:33.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5cc159260 0x7fa5cc1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa5a406af00 0x7fa5a406d3c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5cc15db50 0x7fa5cc15dfd0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 --2- 172.21.15.97:0/1510923421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac005180 0x7fa5cc156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 >> 172.21.15.97:0/1510923421 conn(0x7fa5cc0b0a10 msgr2=0x7fa5cc0ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 shutdown_connections 2024-01-31T05:04:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:33.077+0000 7fa5d2c00640 1 -- 172.21.15.97:0/1510923421 wait complete. 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner: 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-01-31T05:04:33.101 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/cephfs/mount.py", line 775, in run_python 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: p.wait() 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi097 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:Ran 2 tests in 166.142s 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner: 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-31T05:04:33.102 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/cephfs/mount.py", line 775, in run_python 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: p.wait() 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi097 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-01-31T05:04:33.103 INFO:tasks.cephfs_test_runner: 2024-01-31T05:04:33.103 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-31T05:04:33.339 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d9a11201f3204b21b798c4942adc682f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-01-31T05:04:33.341 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-31T05:04:33.350 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-01-31T05:04:33.369 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-01-31T05:04:33.369 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:04:33.369 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:33.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:04:33.428 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-01-31T05:04:33.429 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:33.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053858,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249404k,nr_inodes=812351,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498812k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-31T05:04:33.478 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:04:33.478 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:33.533 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:04:33.533 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-01-31T05:04:33.533 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-31T05:04:33.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053848,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498796k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-31T05:04:33.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:33.586 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-01-31T05:04:33.642 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-31T05:04:33.642 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-31T05:04:33.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:33.643 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-31T05:04:33.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:33.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-31T05:04:33.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:33.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-01-31T05:04:33.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:33.997 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-01-31T05:04:34.056 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-31T05:04:34.056 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-31T05:04:34.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:34.057 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-31T05:04:34.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:34.154 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-31T05:04:34.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-31T05:04:34.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-01-31T05:04:34.369 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-31T05:04:34.379 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-31T05:04:34.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:04:34.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.600+0000 7f387be00640 1 Processor -- start 2024-01-31T05:04:34.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.600+0000 7f387be00640 1 -- start start 2024-01-31T05:04:34.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.602+0000 7f387be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f387c0c9f40 0x7f387c0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:34.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.602+0000 7f387be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:34.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.602+0000 7f387be00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c15a5c0 con 0x7f387c0c9420 2024-01-31T05:04:34.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.602+0000 7f387be00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c0ac400 con 0x7f387c0c9f40 2024-01-31T05:04:34.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.602+0000 7f387be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0cd8d0 con 0x7f387c071e90 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387aa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f387c0c9f40 0x7f387c0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:59198/0 (socket says 172.21.15.97:59198) 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f3871e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f387c0c9420 0x7f387c0abcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:53080/0 (socket says 172.21.15.97:53080) 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387b400640 1 -- 172.21.15.97:0/444973394 learned_addr learned my addr 172.21.15.97:0/444973394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387aa00640 1 -- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 msgr2=0x7f387c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:34.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387aa00640 1 --2- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387aa00640 1 -- 172.21.15.97:0/444973394 >> v1:172.21.15.188:6789/0 conn(0x7f387c0c9420 legacy=0x7f387c0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:34.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.603+0000 7f387aa00640 1 -- 172.21.15.97:0/444973394 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c072320 con 0x7f387c0c9f40 2024-01-31T05:04:34.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.605+0000 7f387aa00640 1 --2- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c0ca320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f386c00bfa0 tx=0x7f386c02e0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befd4ff955289bb8 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:34.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.605+0000 7f387a000640 1 -- 172.21.15.97:0/444973394 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c02ea40 con 0x7f387c0c9f40 2024-01-31T05:04:34.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.605+0000 7f387a000640 1 -- 172.21.15.97:0/444973394 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c02eba0 con 0x7f387c0c9f40 2024-01-31T05:04:34.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.605+0000 7f387a000640 1 -- 172.21.15.97:0/444973394 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c009540 con 0x7f387c0c9f40 2024-01-31T05:04:34.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 -- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 msgr2=0x7f387c0ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:34.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 --2- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c0ca320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f386c00bfa0 tx=0x7f386c02e0a0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 -- 172.21.15.97:0/444973394 shutdown_connections 2024-01-31T05:04:34.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 --2- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 --2- 172.21.15.97:0/444973394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c0ca320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 -- 172.21.15.97:0/444973394 >> 172.21.15.97:0/444973394 conn(0x7f387c0cec40 msgr2=0x7f387c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 -- 172.21.15.97:0/444973394 shutdown_connections 2024-01-31T05:04:34.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.606+0000 7f387be00640 1 -- 172.21.15.97:0/444973394 wait complete. 2024-01-31T05:04:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.608+0000 7f387be00640 1 Processor -- start 2024-01-31T05:04:34.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.608+0000 7f387be00640 1 -- start start 2024-01-31T05:04:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f387be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f387be00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 0x7f387c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f387be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c144580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:34.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f387be00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1f2d50 con 0x7f387c0c9420 2024-01-31T05:04:34.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f387be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0b1ed0 con 0x7f387c0c9f40 2024-01-31T05:04:34.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f387be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0b34c0 con 0x7f387c071e90 2024-01-31T05:04:34.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f3871e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 0x7f387c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:34.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f3871e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 0x7f387c0b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38076/0 (socket says 172.21.15.97:38076) 2024-01-31T05:04:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.609+0000 7f3871e00640 1 -- 172.21.15.97:0/2545915407 learned_addr learned my addr 172.21.15.97:0/2545915407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f387b400640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f387aa00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:34.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3871e00640 1 -- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 msgr2=0x7f387c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3871e00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3871e00640 1 -- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 msgr2=0x7f387c144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3871e00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3871e00640 1 -- 172.21.15.97:0/2545915407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c00bc80 con 0x7f387c0c9420 2024-01-31T05:04:34.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3871e00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 0x7f387c0b37e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f386400c400 tx=0x7f386400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:34.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640074e0 con 0x7f387c0c9420 2024-01-31T05:04:34.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864007640 con 0x7f387c0c9420 2024-01-31T05:04:34.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864004a00 con 0x7f387c0c9420 2024-01-31T05:04:34.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.610+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c1f67d0 con 0x7f387c0c9420 2024-01-31T05:04:34.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.611+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c1f6d90 con 0x7f387c0c9420 2024-01-31T05:04:34.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.612+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c1f2ea0 con 0x7f387c0c9420 2024-01-31T05:04:34.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.613+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f3864004b60 con 0x7f387c0c9420 2024-01-31T05:04:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.613+0000 7f3873400640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f385406aed0 0x7f385406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.614+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3864088e00 con 0x7f387c0c9420 2024-01-31T05:04:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.614+0000 7f387aa00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f385406aed0 0x7f385406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.614+0000 7f387aa00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f385406aed0 0x7f385406d390 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f386c037000 tx=0x7f386c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:34.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.618+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f387c1f2ea0 con 0x7f387c0c9420 2024-01-31T05:04:34.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.806+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f387c0cd560 con 0x7f385406aed0 2024-01-31T05:04:34.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.810+0000 7f3873400640 1 -- 172.21.15.97:0/2545915407 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48458 (secure 0 0 0) 0x7f386c0364f0 con 0x7f385406aed0 2024-01-31T05:04:34.810 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:34.810 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:04:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f385406aed0 msgr2=0x7f385406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f385406aed0 0x7f385406d390 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f386c037000 tx=0x7f386c032000 comp rx=0 tx=0).stop 2024-01-31T05:04:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 msgr2=0x7f387c0b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 0x7f387c0b37e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f386400c400 tx=0x7f386400c8c0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 shutdown_connections 2024-01-31T05:04:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f387c0c9f40 0x7f387c144580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f385406aed0 0x7f385406d390 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f387c0c9420 0x7f387c0b37e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 --2- 172.21.15.97:0/2545915407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f387c071e90 0x7f387c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 >> 172.21.15.97:0/2545915407 conn(0x7f387c0cec40 msgr2=0x7f387c06ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 shutdown_connections 2024-01-31T05:04:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:34.813+0000 7f387be00640 1 -- 172.21.15.97:0/2545915407 wait complete. 2024-01-31T05:04:34.832 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-01-31T05:04:33.626204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222520,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752200,"statfs":{"total":773094113280,"available":772866252800,"internally_reserved":0,"allocated":6524928,"data_stored":3544161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16571,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001540"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":232,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491830+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:40.491830+0000","last_peered":"2024-01-31T05:03:40.491830+0000","last_clean":"2024-01-31T05:03:40.491830+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:40.491830+0000","last_undegraded":"2024-01-31T05:03:40.491830+0000","last_fullsized":"2024-01-31T05:03:40.491830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":295378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":1.514,"15min":1.514},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.0469999999999999,"15min":1.012},"min":{"1min":0.69499999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.423}]},{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0960000000000001,"15min":1.0880000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0259999999999998,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.397},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6890000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.3120000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8100000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.252},{"interface":"front","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2869999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8720000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.97999999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.78000000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7270000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.131},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.2250000000000001},"min":{"1min":0.749,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.792,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3540000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9670000000000001,"5min":2.25,"15min":2.25},"last":1.47},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3660000000000001},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8049999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3540000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.476,"15min":1.476},"min":{"1min":1.0169999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.905,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.92900000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.784,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.554}]},{"osd":6,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4570000000000001,"15min":1.4810000000000001},"min":{"1min":1.044,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.617,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5409999999999999,"15min":1.5569999999999999},"min":{"1min":1.0209999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8660000000000001,"5min":2.629,"15min":2.629},"last":1.619}]}]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":295902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0329999999999999,"15min":1.03},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.403,"5min":1.476,"15min":1.476},"last":1.298},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":0.99299999999999999},"min":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.2490000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.502,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.349},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.129,"15min":1.1339999999999999},"min":{"1min":0.80800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6659999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.91000000000000003}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1930000000000001,"15min":1.1830000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5800000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.1499999999999999,"15min":1.1739999999999999},"min":{"1min":0.60599999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5900000000000001,"5min":1.883,"15min":1.883},"last":1.0189999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.345,"15min":1.383},"min":{"1min":0.84099999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.536,"5min":1.73,"15min":1.73},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.413,"15min":1.425},"min":{"1min":1.0549999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6259999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.5860000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3460000000000001,"15min":1.403},"min":{"1min":0.80800000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.478,"5min":2.032,"15min":2.032},"last":1.21},{"interface":"front","average":{"1min":1.395,"5min":1.5289999999999999,"15min":1.571},"min":{"1min":1.0680000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.631}]},{"osd":6,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.522},"min":{"1min":0.98099999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7190000000000001,"5min":2.335,"15min":2.335},"last":1.147},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.5189999999999999,"15min":1.583},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.3879999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.5389999999999999,"15min":1.5960000000000001},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.696,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.597,"15min":1.645},"min":{"1min":1.0960000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.7729999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.698}]}]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":295376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.0169999999999999,"15min":1.127},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3009999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.995,"15min":1.071},"min":{"1min":0.53900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.615,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0649999999999999,"15min":1.159},"min":{"1min":0.57599999999999996,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.77,"5min":5.617,"15min":5.617},"last":0.376},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.1579999999999999,"15min":1.274},"min":{"1min":0.628,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3500000000000001,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.41399999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.345},"min":{"1min":0.61299999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.395,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":1.125,"5min":1.3859999999999999,"15min":1.5389999999999999},"min":{"1min":0.53200000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5680000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.497}]},{"osd":4,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5089999999999999,"15min":1.6850000000000001},"min":{"1min":0.89700000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.921,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.351,"5min":1.5429999999999999,"15min":1.675},"min":{"1min":1.135,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.869,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.2}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.5609999999999999,"15min":1.6970000000000001},"min":{"1min":0.98299999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9850000000000001,"5min":6.048,"15min":6.048},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.637,"15min":1.7949999999999999},"min":{"1min":0.94699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8109999999999999,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.601,"15min":1.7190000000000001},"min":{"1min":0.995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8759999999999999,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.746},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.5580000000000001,"15min":1.7150000000000001},"min":{"1min":0.88,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6459999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.84399999999999997}]},{"osd":7,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.573,"15min":1.7130000000000001},"min":{"1min":0.86899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0499999999999998,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.7310000000000001,"15min":1.911},"min":{"1min":1.0509999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.9370000000000001,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.417}]}]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":684032,"data_stored":304818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1080000000000001,"5min":1.361,"15min":1.361},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4390000000000001,"5min":1.73,"15min":1.73},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.536,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.748},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3580000000000001,"5min":1.671,"15min":1.671},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.544,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0289999999999999,"15min":1.071},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":0.60999999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.72,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.204},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.272,"15min":1.3140000000000001},"min":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.218,"15min":1.236},"min":{"1min":0.746,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5549999999999999,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.272,"15min":1.306},"min":{"1min":0.65300000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.522,"5min":2.52,"15min":2.52},"last":1.0429999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.89800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.663,"5min":2.992,"15min":2.992},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.3520000000000001,"15min":1.3939999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.631,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.341}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.3859999999999999},"min":{"1min":0.76700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5349999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.131},{"interface":"front","average":{"1min":1.218,"5min":1.3700000000000001,"15min":1.4059999999999999},"min":{"1min":0.76800000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.619,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.0109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":879835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98299999999999998,"15min":1.0680000000000001},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.409,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.028},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.86199999999999999,"15min":0.90800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4890000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.94899999999999995,"15min":1.0169999999999999},"min":{"1min":0.52700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.3440000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.163},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.321,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.1220000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.99099999999999999,"15min":1.038},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.605,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.111,"15min":1.204},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4830000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3859999999999999,"15min":1.486},"min":{"1min":0.78600000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5469999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3009999999999999,"15min":1.365},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5469999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.2170000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3540000000000001,"15min":1.4299999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.611,"5min":2.722,"15min":2.722},"last":1.333},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.4390000000000001},"min":{"1min":0.84099999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5189999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.2769999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.427,"15min":1.5209999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.625,"5min":2.577,"15min":2.577},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4410000000000001},"min":{"1min":0.85199999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6930000000000001,"5min":2.794,"15min":2.794},"last":1.0880000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4390000000000001,"15min":1.5269999999999999},"min":{"1min":0.95599999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.9870000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.5089999999999999,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8460000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.3140000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":655360,"data_stored":288907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5760000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.054},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.113,"15min":1.125},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1850000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.75600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.111},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1439999999999999,"15min":1.155},"min":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":1.976,"15min":1.976},"last":1.141}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.179,"15min":1.1739999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4530000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.84699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.6930000000000001,"5min":1.873,"15min":1.873},"last":1.409}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3660000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6639999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.92700000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.264,"15min":1.256},"min":{"1min":0.871,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.448,"15min":1.472},"min":{"1min":0.755,"5min":0.628,"15min":0.628},"max":{"1min":1.8100000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4350000000000001,"15min":1.4510000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.573,"15min":1.597},"min":{"1min":0.878,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.966,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4379999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":1.0860000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1309999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.651,"15min":1.677},"min":{"1min":1.157,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.04,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.548}]}]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":879287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.04,"15min":1.022},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.117},"min":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.018}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1379999999999999,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.732,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.194,"15min":1.2170000000000001},"min":{"1min":0.77600000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.77,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.2190000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.724,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.208},{"interface":"front","average":{"1min":1.222,"5min":1.2769999999999999,"15min":1.3029999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6870000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.3680000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.28,"15min":1.298},"min":{"1min":0.79800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6859999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.149},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.369,"15min":1.393},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.673,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.873}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.3700000000000001},"min":{"1min":0.94399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9530000000000001,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.94199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8280000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.2949999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4470000000000001,"15min":1.417},"min":{"1min":1.0960000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0510000000000002,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.4930000000000001,"15min":1.4950000000000001},"min":{"1min":1.167,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.0009999999999999,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4850000000000001,"15min":1.4930000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9339999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.548},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.617,"15min":1.619},"min":{"1min":1.268,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2040000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.5149999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":675840,"data_stored":304658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.84299999999999997,"15min":0.83699999999999997},"min":{"1min":0.371,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":3.77,"15min":3.77},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.99299999999999999,"15min":1.0329999999999999},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4299999999999999,"5min":3.282,"15min":3.282},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":1.0009999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.357,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82999999999999996,"15min":0.85799999999999998},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.296,"5min":1.774,"15min":1.774},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.028,"15min":1.054},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4430000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0820000000000001},"min":{"1min":0.63200000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5589999999999999,"5min":3.121,"15min":3.121},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2170000000000001},"min":{"1min":0.81000000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5489999999999999,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.3899999999999999,"15min":1.4239999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6970000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.4379999999999999,"15min":1.464},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7709999999999999,"5min":5.75,"15min":5.75},"last":1.228},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.4099999999999999,"15min":1.456},"min":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5089999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.323,"15min":1.319},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":7.415,"15min":7.415},"last":0.872},{"interface":"front","average":{"1min":1.302,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.7130000000000001,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.0449999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.3500000000000001,"15min":1.3799999999999999},"min":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.552,"5min":3.633,"15min":3.633},"last":1.149},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.504,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.899,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:04:34.834 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:04:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.038+0000 7f3826600640 1 Processor -- start 2024-01-31T05:04:35.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.038+0000 7f3826600640 1 -- start start 2024-01-31T05:04:35.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3826600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38200cb810 0x7f38200cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3826600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f382015b400 0x7f382006d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3826600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382006db70 0x7f38200ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3826600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38200cc960 con 0x7f382015b400 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3826600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382006d1a0 con 0x7f38200cb810 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3826600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382006c600 con 0x7f382006db70 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3816800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f382015b400 0x7f382006d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3816800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f382015b400 0x7f382006d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59680/0 (socket says 172.21.15.97:59680) 2024-01-31T05:04:35.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.040+0000 7f3816800640 1 -- 172.21.15.97:0/1660590989 learned_addr learned my addr 172.21.15.97:0/1660590989 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f381fe00640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382006db70 0x7f38200ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f381f400640 1 --2- 172.21.15.97:0/1660590989 >> v2:172.21.15.188:3300/0 conn(0x7f38200cb810 0x7f38200cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f3816800640 1 -- 172.21.15.97:0/1660590989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382006db70 msgr2=0x7f38200ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f3816800640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382006db70 0x7f38200ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f3816800640 1 -- 172.21.15.97:0/1660590989 >> v2:172.21.15.188:3300/0 conn(0x7f38200cb810 msgr2=0x7f38200cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f3816800640 1 --2- 172.21.15.97:0/1660590989 >> v2:172.21.15.188:3300/0 conn(0x7f38200cb810 0x7f38200cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f3816800640 1 -- 172.21.15.97:0/1660590989 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200cc130 con 0x7f382015b400 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f381fe00640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382006db70 0x7f38200ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.041+0000 7f381f400640 1 --2- 172.21.15.97:0/1660590989 >> v2:172.21.15.188:3300/0 conn(0x7f38200cb810 0x7f38200cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.042+0000 7f3816800640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382015b400 0x7f382006d630 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3820067410 tx=0x7f380802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cf174a52cb5533 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.042+0000 7f381ea00640 1 -- 172.21.15.97:0/1660590989 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808031870 con 0x7f382015b400 2024-01-31T05:04:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.042+0000 7f381ea00640 1 -- 172.21.15.97:0/1660590989 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38080319d0 con 0x7f382015b400 2024-01-31T05:04:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.042+0000 7f381ea00640 1 -- 172.21.15.97:0/1660590989 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808031d60 con 0x7f382015b400 2024-01-31T05:04:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 -- 172.21.15.97:0/1660590989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382015b400 msgr2=0x7f382006d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382015b400 0x7f382006d630 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3820067410 tx=0x7f380802b910 comp rx=0 tx=0).stop 2024-01-31T05:04:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 -- 172.21.15.97:0/1660590989 shutdown_connections 2024-01-31T05:04:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382006db70 0x7f38200ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 --2- 172.21.15.97:0/1660590989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382015b400 0x7f382006d630 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 --2- 172.21.15.97:0/1660590989 >> v2:172.21.15.188:3300/0 conn(0x7f38200cb810 0x7f38200cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 -- 172.21.15.97:0/1660590989 >> 172.21.15.97:0/1660590989 conn(0x7f38200b0a10 msgr2=0x7f38200b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 -- 172.21.15.97:0/1660590989 shutdown_connections 2024-01-31T05:04:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.043+0000 7f3826600640 1 -- 172.21.15.97:0/1660590989 wait complete. 2024-01-31T05:04:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.045+0000 7f3826600640 1 Processor -- start 2024-01-31T05:04:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.045+0000 7f3826600640 1 -- start start 2024-01-31T05:04:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f3826600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 0x7f38200c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f3826600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cb810 0x7f38200ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f3826600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b400 0x7f3820157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f3826600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382015cd30 con 0x7f382006db70 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f3826600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38200c8f50 con 0x7f38200cb810 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f3826600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382015c970 con 0x7f382015b400 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f381f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 0x7f38200c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f381f400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 0x7f38200c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59682/0 (socket says 172.21.15.97:59682) 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.046+0000 7f381f400640 1 -- 172.21.15.97:0/3268273738 learned_addr learned my addr 172.21.15.97:0/3268273738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f3816800640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cb810 0x7f38200ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381fe00640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b400 0x7f3820157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381f400640 1 -- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b400 msgr2=0x7f3820157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381f400640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b400 0x7f3820157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381f400640 1 -- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cb810 msgr2=0x7f38200ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381f400640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cb810 0x7f38200ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381f400640 1 -- 172.21.15.97:0/3268273738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080094d0 con 0x7f382006db70 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f381f400640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 0x7f38200c8090 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f381000c7f0 tx=0x7f381000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.047+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810007a30 con 0x7f382006db70 2024-01-31T05:04:35.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.048+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3810007b90 con 0x7f382006db70 2024-01-31T05:04:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.048+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000ce40 con 0x7f382006db70 2024-01-31T05:04:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.048+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38200c8990 con 0x7f382006db70 2024-01-31T05:04:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.048+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38200c85d0 con 0x7f382006db70 2024-01-31T05:04:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.049+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38200677c0 con 0x7f382006db70 2024-01-31T05:04:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.050+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f38100113f0 con 0x7f382006db70 2024-01-31T05:04:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.051+0000 7f3817e00640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f37f806aed0 0x7f37f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.051+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3810089540 con 0x7f382006db70 2024-01-31T05:04:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.051+0000 7f3816800640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f37f806aed0 0x7f37f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.052 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.052+0000 7f3816800640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f37f806aed0 0x7f37f806d390 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3808036000 tx=0x7f380803c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.055+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f3810054660 con 0x7f382006db70 2024-01-31T05:04:35.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.245+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f382006ce30 con 0x7f37f806aed0 2024-01-31T05:04:35.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.248+0000 7f3817e00640 1 -- 172.21.15.97:0/3268273738 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48458 (secure 0 0 0) 0x7f3808031810 con 0x7f37f806aed0 2024-01-31T05:04:35.249 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:35.249 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:04:35.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f37f806aed0 msgr2=0x7f37f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f37f806aed0 0x7f37f806d390 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3808036000 tx=0x7f380803c040 comp rx=0 tx=0).stop 2024-01-31T05:04:35.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 msgr2=0x7f38200c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 0x7f38200c8090 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f381000c7f0 tx=0x7f381000cbf0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 shutdown_connections 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382015b400 0x7f3820157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38200cb810 0x7f38200ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f37f806aed0 0x7f37f806d390 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 --2- 172.21.15.97:0/3268273738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f382006db70 0x7f38200c8090 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.251+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 >> 172.21.15.97:0/3268273738 conn(0x7f38200b0a10 msgr2=0x7f38200ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:35.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.252+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 shutdown_connections 2024-01-31T05:04:35.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.252+0000 7f3826600640 1 -- 172.21.15.97:0/3268273738 wait complete. 2024-01-31T05:04:35.271 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-01-31T05:04:33.626204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222520,"kb_used_data":6372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752200,"statfs":{"total":773094113280,"available":772866252800,"internally_reserved":0,"allocated":6524928,"data_stored":3544161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16571,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001540"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":232,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-01-31T05:03:40.491830+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:03:40.491830+0000","last_peered":"2024-01-31T05:03:40.491830+0000","last_clean":"2024-01-31T05:03:40.491830+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:03:40.491830+0000","last_undegraded":"2024-01-31T05:03:40.491830+0000","last_fullsized":"2024-01-31T05:03:40.491830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":295378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":1.514,"15min":1.514},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.155,"5min":1.0469999999999999,"15min":1.012},"min":{"1min":0.69499999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.423}]},{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0960000000000001,"15min":1.0880000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0259999999999998,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.397},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6890000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.3120000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8100000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.252},{"interface":"front","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2869999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8720000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.97999999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.78000000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7270000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.131},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.2250000000000001},"min":{"1min":0.749,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.792,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3540000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9670000000000001,"5min":2.25,"15min":2.25},"last":1.47},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3660000000000001},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8049999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3540000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.476,"15min":1.476},"min":{"1min":1.0169999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.905,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.92900000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.784,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.554}]},{"osd":6,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4570000000000001,"15min":1.4810000000000001},"min":{"1min":1.044,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.617,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5409999999999999,"15min":1.5569999999999999},"min":{"1min":1.0209999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8660000000000001,"5min":2.629,"15min":2.629},"last":1.619}]}]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":671744,"data_stored":295902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0329999999999999,"15min":1.03},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.403,"5min":1.476,"15min":1.476},"last":1.298},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":0.99299999999999999},"min":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.2490000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.502,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.349},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.129,"15min":1.1339999999999999},"min":{"1min":0.80800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6659999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.91000000000000003}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1930000000000001,"15min":1.1830000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5800000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.1499999999999999,"15min":1.1739999999999999},"min":{"1min":0.60599999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5900000000000001,"5min":1.883,"15min":1.883},"last":1.0189999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.345,"15min":1.383},"min":{"1min":0.84099999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.536,"5min":1.73,"15min":1.73},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.413,"15min":1.425},"min":{"1min":1.0549999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6259999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.5860000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3460000000000001,"15min":1.403},"min":{"1min":0.80800000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.478,"5min":2.032,"15min":2.032},"last":1.21},{"interface":"front","average":{"1min":1.395,"5min":1.5289999999999999,"15min":1.571},"min":{"1min":1.0680000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.631}]},{"osd":6,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.522},"min":{"1min":0.98099999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7190000000000001,"5min":2.335,"15min":2.335},"last":1.147},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.5189999999999999,"15min":1.583},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.3879999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.5389999999999999,"15min":1.5960000000000001},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.696,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.597,"15min":1.645},"min":{"1min":1.0960000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.7729999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.698}]}]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":295376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.0169999999999999,"15min":1.127},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3009999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.995,"15min":1.071},"min":{"1min":0.53900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.615,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0649999999999999,"15min":1.159},"min":{"1min":0.57599999999999996,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.77,"5min":5.617,"15min":5.617},"last":0.376},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.1579999999999999,"15min":1.274},"min":{"1min":0.628,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3500000000000001,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.41399999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.345},"min":{"1min":0.61299999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.395,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":1.125,"5min":1.3859999999999999,"15min":1.5389999999999999},"min":{"1min":0.53200000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5680000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":0.497}]},{"osd":4,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5089999999999999,"15min":1.6850000000000001},"min":{"1min":0.89700000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.921,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.351,"5min":1.5429999999999999,"15min":1.675},"min":{"1min":1.135,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.869,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.2}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.5609999999999999,"15min":1.6970000000000001},"min":{"1min":0.98299999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9850000000000001,"5min":6.048,"15min":6.048},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.637,"15min":1.7949999999999999},"min":{"1min":0.94699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8109999999999999,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":0.90000000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.601,"15min":1.7190000000000001},"min":{"1min":0.995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8759999999999999,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":0.746},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.5580000000000001,"15min":1.7150000000000001},"min":{"1min":0.88,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6459999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":0.84399999999999997}]},{"osd":7,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.573,"15min":1.7130000000000001},"min":{"1min":0.86899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0499999999999998,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.7310000000000001,"15min":1.911},"min":{"1min":1.0509999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.9370000000000001,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.417}]}]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":684032,"data_stored":304818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1080000000000001,"5min":1.361,"15min":1.361},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4390000000000001,"5min":1.73,"15min":1.73},"last":0.71199999999999997}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.536,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.748},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3580000000000001,"5min":1.671,"15min":1.671},"last":0.93600000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.544,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0289999999999999,"15min":1.071},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":0.60999999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.72,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.204},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.272,"15min":1.3140000000000001},"min":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.218,"15min":1.236},"min":{"1min":0.746,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5549999999999999,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.272,"15min":1.306},"min":{"1min":0.65300000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.522,"5min":2.52,"15min":2.52},"last":1.0429999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.89800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.663,"5min":2.992,"15min":2.992},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.3520000000000001,"15min":1.3939999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.631,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.341}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.3859999999999999},"min":{"1min":0.76700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5349999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.131},{"interface":"front","average":{"1min":1.218,"5min":1.3700000000000001,"15min":1.4059999999999999},"min":{"1min":0.76800000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.619,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.0109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":879835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98299999999999998,"15min":1.0680000000000001},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.409,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.028},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.86199999999999999,"15min":0.90800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4890000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.94899999999999995,"15min":1.0169999999999999},"min":{"1min":0.52700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.3440000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.163},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.321,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.1220000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.99099999999999999,"15min":1.038},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.605,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.872},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.111,"15min":1.204},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4830000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.1559999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3859999999999999,"15min":1.486},"min":{"1min":0.78600000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5469999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3009999999999999,"15min":1.365},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5469999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.2170000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3540000000000001,"15min":1.4299999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.611,"5min":2.722,"15min":2.722},"last":1.333},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.4390000000000001},"min":{"1min":0.84099999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5189999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.2769999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.427,"15min":1.5209999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.625,"5min":2.577,"15min":2.577},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4410000000000001},"min":{"1min":0.85199999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6930000000000001,"5min":2.794,"15min":2.794},"last":1.0880000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4390000000000001,"15min":1.5269999999999999},"min":{"1min":0.95599999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.9870000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.5089999999999999,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8460000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.3140000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":655360,"data_stored":288907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5760000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.054},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.113,"15min":1.125},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.1850000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.75600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.111},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1439999999999999,"15min":1.155},"min":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":1.976,"15min":1.976},"last":1.141}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.179,"15min":1.1739999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4530000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.84699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.6930000000000001,"5min":1.873,"15min":1.873},"last":1.409}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3660000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6639999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.92700000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.345}]},{"osd":4,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.264,"15min":1.256},"min":{"1min":0.871,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.371},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.448,"15min":1.472},"min":{"1min":0.755,"5min":0.628,"15min":0.628},"max":{"1min":1.8100000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.222}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4350000000000001,"15min":1.4510000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.573,"15min":1.597},"min":{"1min":0.878,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.966,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4379999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":1.0860000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1309999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.651,"15min":1.677},"min":{"1min":1.157,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.04,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.548}]}]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28240,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343600,"statfs":{"total":96636764160,"available":96607846400,"internally_reserved":0,"allocated":1249280,"data_stored":879287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.04,"15min":1.022},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.117},"min":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.018}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1379999999999999,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.732,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.194,"15min":1.2170000000000001},"min":{"1min":0.77600000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.77,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.2190000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.724,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.208},{"interface":"front","average":{"1min":1.222,"5min":1.2769999999999999,"15min":1.3029999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6870000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.3680000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.28,"15min":1.298},"min":{"1min":0.79800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6859999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.149},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.369,"15min":1.393},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.673,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.873}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.3700000000000001},"min":{"1min":0.94399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9530000000000001,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.94199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8280000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.2949999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4470000000000001,"15min":1.417},"min":{"1min":1.0960000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0510000000000002,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.4930000000000001,"15min":1.4950000000000001},"min":{"1min":1.167,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.0009999999999999,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4850000000000001,"15min":1.4930000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9339999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.548},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.617,"15min":1.619},"min":{"1min":1.268,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2040000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.5149999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":675840,"data_stored":304658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.84299999999999997,"15min":0.83699999999999997},"min":{"1min":0.371,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":3.77,"15min":3.77},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.99299999999999999,"15min":1.0329999999999999},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4299999999999999,"5min":3.282,"15min":3.282},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":1.0009999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.357,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82999999999999996,"15min":0.85799999999999998},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.296,"5min":1.774,"15min":1.774},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.028,"15min":1.054},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4430000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0820000000000001},"min":{"1min":0.63200000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5589999999999999,"5min":3.121,"15min":3.121},"last":0.65000000000000002}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2170000000000001},"min":{"1min":0.81000000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5489999999999999,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.3899999999999999,"15min":1.4239999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6970000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.4379999999999999,"15min":1.464},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7709999999999999,"5min":5.75,"15min":5.75},"last":1.228},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.4099999999999999,"15min":1.456},"min":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5089999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.323,"15min":1.319},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":7.415,"15min":7.415},"last":0.872},{"interface":"front","average":{"1min":1.302,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.7130000000000001,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.0449999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.3500000000000001,"15min":1.3799999999999999},"min":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.552,"5min":3.633,"15min":3.633},"last":1.149},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.504,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.899,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":0.93700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:04:35.272 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-31T05:04:35.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:04:35.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.478+0000 7f0634600640 1 Processor -- start 2024-01-31T05:04:35.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.479+0000 7f0634600640 1 -- start start 2024-01-31T05:04:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.480+0000 7f0634600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f062c0cb810 0x7f062c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.480+0000 7f0634600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c06d1a0 con 0x7f062c0ccd00 2024-01-31T05:04:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.480+0000 7f0634600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c06c600 con 0x7f062c15b400 2024-01-31T05:04:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.480+0000 7f0634600640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c0cc960 con 0x7f062c0cb810 2024-01-31T05:04:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f0631c00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f062c0cb810 0x7f062c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f0632600640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f062c0ccd00 0x7f062c0cd170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47512/0 (socket says 172.21.15.97:47512) 2024-01-31T05:04:35.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f0632600640 1 -- 172.21.15.97:0/4073560040 learned_addr learned my addr 172.21.15.97:0/4073560040 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f0631c00640 1 -- 172.21.15.97:0/4073560040 >> v1:172.21.15.188:6789/0 conn(0x7f062c15b400 legacy=0x7f062c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f0631c00640 1 -- 172.21.15.97:0/4073560040 >> v1:172.21.15.97:6789/0 conn(0x7f062c0ccd00 legacy=0x7f062c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f0631c00640 1 -- 172.21.15.97:0/4073560040 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0cc130 con 0x7f062c0cb810 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.481+0000 7f062be00640 1 -- 172.21.15.97:0/4073560040 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369079852 0 0) 0x7f062c06d1a0 con 0x7f062c0ccd00 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.482+0000 7f0631c00640 1 --2- 172.21.15.97:0/4073560040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c0cbbf0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6e2eb563c2cdfe server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.482+0000 7f062be00640 1 -- 172.21.15.97:0/4073560040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c02cc60 con 0x7f062c0cb810 2024-01-31T05:04:35.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.482+0000 7f062be00640 1 -- 172.21.15.97:0/4073560040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c02cdc0 con 0x7f062c0cb810 2024-01-31T05:04:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.482+0000 7f062be00640 1 -- 172.21.15.97:0/4073560040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c034660 con 0x7f062c0cb810 2024-01-31T05:04:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 -- 172.21.15.97:0/4073560040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 msgr2=0x7f062c0cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 --2- 172.21.15.97:0/4073560040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c0cbbf0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c02beb0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 -- 172.21.15.97:0/4073560040 shutdown_connections 2024-01-31T05:04:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 --2- 172.21.15.97:0/4073560040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c0cbbf0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 -- 172.21.15.97:0/4073560040 >> 172.21.15.97:0/4073560040 conn(0x7f062c0b0a10 msgr2=0x7f062c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 -- 172.21.15.97:0/4073560040 shutdown_connections 2024-01-31T05:04:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.483+0000 7f0634600640 1 -- 172.21.15.97:0/4073560040 wait complete. 2024-01-31T05:04:35.484 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.485+0000 7f0634600640 1 Processor -- start 2024-01-31T05:04:35.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.485+0000 7f0634600640 1 -- start start 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0634600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0634600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f062c0ccd00 0x7f062c14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0634600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f062c15b400 0x7f062c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0634600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c1f71c0 con 0x7f062c0cb810 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0634600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c1f65b0 con 0x7f062c0ccd00 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0634600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c1f6890 con 0x7f062c15b400 2024-01-31T05:04:35.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0631c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0631c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c15dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59704/0 (socket says 172.21.15.97:59704) 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.486+0000 7f0631c00640 1 -- 172.21.15.97:0/3087541567 learned_addr learned my addr 172.21.15.97:0/3087541567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0632600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f062c15b400 0x7f062c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631c00640 1 -- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f062c15b400 msgr2=0x7f062c0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631c00640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f062c15b400 0x7f062c0c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631c00640 1 -- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f062c0ccd00 msgr2=0x7f062c14f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631200640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f062c0ccd00 0x7f062c14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631c00640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f062c0ccd00 0x7f062c14f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631c00640 1 -- 172.21.15.97:0/3087541567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c009a70 con 0x7f062c0cb810 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631200640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f062c0ccd00 0x7f062c14f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0631c00640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c15dab0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c03e610 con 0x7f062c0cb810 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c03e770 con 0x7f062c0cb810 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c1f6b70 con 0x7f062c0cb810 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.487+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c03eb20 con 0x7f062c0cb810 2024-01-31T05:04:35.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.488+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c14fba0 con 0x7f062c0cb810 2024-01-31T05:04:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.489+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c0677c0 con 0x7f062c0cb810 2024-01-31T05:04:35.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.490+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f061c03b050 con 0x7f062c0cb810 2024-01-31T05:04:35.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.491+0000 7f0629600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f060406aed0 0x7f060406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.491+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f061c0abe90 con 0x7f062c0cb810 2024-01-31T05:04:35.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.492+0000 7f0631200640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f060406aed0 0x7f060406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.492+0000 7f0631200640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f060406aed0 0x7f060406d390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f062c15b7e0 tx=0x7f0614008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.496+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f061c0770d0 con 0x7f062c0cb810 2024-01-31T05:04:35.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.687+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f062c06ce30 con 0x7f060406aed0 2024-01-31T05:04:35.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.693+0000 7f0629600640 1 -- 172.21.15.97:0/3087541567 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48528 (secure 0 0 0) 0x7f061400ad80 con 0x7f060406aed0 2024-01-31T05:04:35.693 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:35.694 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:04:35.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f060406aed0 msgr2=0x7f060406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f060406aed0 0x7f060406d390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f062c15b7e0 tx=0x7f0614008040 comp rx=0 tx=0).stop 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 msgr2=0x7f062c15dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c15dab0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f061c009d90 tx=0x7f061c004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 shutdown_connections 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f062c15b400 0x7f062c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f062c0ccd00 0x7f062c14f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f060406aed0 0x7f060406d390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 --2- 172.21.15.97:0/3087541567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f062c0cb810 0x7f062c15dab0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 >> 172.21.15.97:0/3087541567 conn(0x7f062c0b0a10 msgr2=0x7f062c0cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:35.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.696+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 shutdown_connections 2024-01-31T05:04:35.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.697+0000 7f0634600640 1 -- 172.21.15.97:0/3087541567 wait complete. 2024-01-31T05:04:35.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-01-31T05:04:35.626673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222776,"kb_used_data":6600,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751944,"statfs":{"total":773094113280,"available":772865990656,"internally_reserved":0,"allocated":6758400,"data_stored":3640688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16571,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.580683"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":238,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-01-31T05:04:32.065425+0000","last_change":"2024-01-31T05:00:41.348391+0000","last_active":"2024-01-31T05:04:32.065425+0000","last_peered":"2024-01-31T05:04:32.065425+0000","last_clean":"2024-01-31T05:04:32.065425+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:04:32.065425+0000","last_undegraded":"2024-01-31T05:04:32.065425+0000","last_fullsized":"2024-01-31T05:04:32.065425+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-31T05:00:40.321395+0000","last_clean_scrub_stamp":"2024-01-31T05:00:40.321395+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T07:34:07.326077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":1.514,"15min":1.514},"last":1.363},{"interface":"front","average":{"1min":1.155,"5min":1.0469999999999999,"15min":1.012},"min":{"1min":0.69499999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.103}]},{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0960000000000001,"15min":1.0880000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0259999999999998,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6890000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.2030000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8100000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.161},{"interface":"front","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2869999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8720000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.89000000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.78000000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7270000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.2250000000000001},"min":{"1min":0.749,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.792,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.002}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3540000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9670000000000001,"5min":2.25,"15min":2.25},"last":1.27},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3660000000000001},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8049999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.732}]},{"osd":5,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.476,"15min":1.476},"min":{"1min":1.0169999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.905,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.92900000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.784,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.8979999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4570000000000001,"15min":1.4810000000000001},"min":{"1min":1.044,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.617,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.631},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5409999999999999,"15min":1.5569999999999999},"min":{"1min":1.0209999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8660000000000001,"5min":2.629,"15min":2.629},"last":1.996}]}]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0329999999999999,"15min":1.03},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.403,"5min":1.476,"15min":1.476},"last":1.155},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":0.99299999999999999},"min":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.502,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.129,"15min":1.1339999999999999},"min":{"1min":0.80800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6659999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.7190000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1930000000000001,"15min":1.1830000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5800000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.04,"5min":1.1499999999999999,"15min":1.1739999999999999},"min":{"1min":0.60599999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5900000000000001,"5min":1.883,"15min":1.883},"last":1.6599999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.345,"15min":1.383},"min":{"1min":0.84099999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.536,"5min":1.73,"15min":1.73},"last":1.369},{"interface":"front","average":{"1min":1.379,"5min":1.413,"15min":1.425},"min":{"1min":1.0549999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6259999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.804}]},{"osd":5,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3460000000000001,"15min":1.403},"min":{"1min":0.80800000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.478,"5min":2.032,"15min":2.032},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.5289999999999999,"15min":1.571},"min":{"1min":1.0680000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.4350000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.522},"min":{"1min":0.98099999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7190000000000001,"5min":2.335,"15min":2.335},"last":1.623},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.5189999999999999,"15min":1.583},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.5700000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.5389999999999999,"15min":1.5960000000000001},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.696,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.597,"15min":1.645},"min":{"1min":1.0960000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.7729999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.887}]}]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.0169999999999999,"15min":1.127},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3009999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.995,"15min":1.071},"min":{"1min":0.53900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.615,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":1.325}]},{"osd":1,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0649999999999999,"15min":1.159},"min":{"1min":0.57599999999999996,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.77,"5min":5.617,"15min":5.617},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.1579999999999999,"15min":1.274},"min":{"1min":0.628,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3500000000000001,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.165}]},{"osd":2,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.345},"min":{"1min":0.61299999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.395,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.022},{"interface":"front","average":{"1min":1.125,"5min":1.3859999999999999,"15min":1.5389999999999999},"min":{"1min":0.53200000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5680000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.355}]},{"osd":4,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5089999999999999,"15min":1.6850000000000001},"min":{"1min":0.89700000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.921,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.254},{"interface":"front","average":{"1min":1.351,"5min":1.5429999999999999,"15min":1.675},"min":{"1min":1.135,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.869,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.5609999999999999,"15min":1.6970000000000001},"min":{"1min":0.98299999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9850000000000001,"5min":6.048,"15min":6.048},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.637,"15min":1.7949999999999999},"min":{"1min":0.94699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8109999999999999,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.419}]},{"osd":6,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.601,"15min":1.7190000000000001},"min":{"1min":0.995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8759999999999999,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.5580000000000001,"15min":1.7150000000000001},"min":{"1min":0.88,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6459999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.0920000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.573,"15min":1.7130000000000001},"min":{"1min":0.86899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0499999999999998,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.7310000000000001,"15min":1.911},"min":{"1min":1.0509999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.9370000000000001,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.3919999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1080000000000001,"5min":1.361,"15min":1.361},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4390000000000001,"5min":1.73,"15min":1.73},"last":1.1160000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.536,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3580000000000001,"5min":1.671,"15min":1.671},"last":1.0740000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.544,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0289999999999999,"15min":1.071},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":1.4630000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.72,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.165},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.272,"15min":1.3140000000000001},"min":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.2290000000000001}]},{"osd":5,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.218,"15min":1.236},"min":{"1min":0.746,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5549999999999999,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":0.996},{"interface":"front","average":{"1min":1.133,"5min":1.272,"15min":1.306},"min":{"1min":0.65300000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.522,"5min":2.52,"15min":2.52},"last":1.359}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.89800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.663,"5min":2.992,"15min":2.992},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.204,"5min":1.3520000000000001,"15min":1.3939999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.631,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.27}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.3859999999999999},"min":{"1min":0.76700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5349999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3700000000000001,"15min":1.4059999999999999},"min":{"1min":0.76800000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.619,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.2909999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1290240,"data_stored":897862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98299999999999998,"15min":1.0680000000000001},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.409,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.86199999999999999,"15min":0.90800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4890000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1140000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.94899999999999995,"15min":1.0169999999999999},"min":{"1min":0.52700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.3440000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.163},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.321,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.46}]},{"osd":3,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.99099999999999999,"15min":1.038},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.605,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":1.518},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.111,"15min":1.204},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4830000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.173}]},{"osd":4,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3859999999999999,"15min":1.486},"min":{"1min":0.78600000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5469999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.8},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3009999999999999,"15min":1.365},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5469999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.7689999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3540000000000001,"15min":1.4299999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.611,"5min":2.722,"15min":2.722},"last":2.0339999999999998},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.4390000000000001},"min":{"1min":0.84099999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5189999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.6739999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.427,"15min":1.5209999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.625,"5min":2.577,"15min":2.577},"last":1.587},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4410000000000001},"min":{"1min":0.85199999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6930000000000001,"5min":2.794,"15min":2.794},"last":1.921}]},{"osd":7,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4390000000000001,"15min":1.5269999999999999},"min":{"1min":0.95599999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.9870000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.365,"5min":1.5089999999999999,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8460000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.9750000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5760000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.113,"15min":1.125},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.4059999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.75600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1439999999999999,"15min":1.155},"min":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":1.976,"15min":1.976},"last":1.22}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.179,"15min":1.1739999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4530000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.84699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.6930000000000001,"5min":1.873,"15min":1.873},"last":1.3440000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3660000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6639999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.647},{"interface":"front","average":{"1min":1.335,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.92700000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.6120000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.264,"15min":1.256},"min":{"1min":0.871,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.448,"15min":1.472},"min":{"1min":0.755,"5min":0.628,"15min":0.628},"max":{"1min":1.8100000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.0389999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4350000000000001,"15min":1.4510000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.506,"5min":1.573,"15min":1.597},"min":{"1min":0.878,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.966,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.7569999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":1.0860000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1309999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.651,"15min":1.677},"min":{"1min":1.157,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.04,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.843}]}]},{"osd":5,"up_from":14,"seq":60129542194,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1290240,"data_stored":897862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.04,"15min":1.022},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.117},"min":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.0329999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1379999999999999,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.732,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.181},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.194,"15min":1.2170000000000001},"min":{"1min":0.77600000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.77,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.25}]},{"osd":2,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.2190000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.724,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.2769999999999999,"15min":1.3029999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6870000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.466}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.28,"15min":1.298},"min":{"1min":0.79800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6859999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.369,"15min":1.393},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.673,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.2909999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.3700000000000001},"min":{"1min":0.94399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9530000000000001,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.94199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8280000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.7709999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4470000000000001,"15min":1.417},"min":{"1min":1.0960000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0510000000000002,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.512},{"interface":"front","average":{"1min":1.518,"5min":1.4930000000000001,"15min":1.4950000000000001},"min":{"1min":1.167,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.0009999999999999,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.597}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4850000000000001,"15min":1.4930000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9339999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.617,"15min":1.619},"min":{"1min":1.268,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2040000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.6759999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.84299999999999997,"15min":0.83699999999999997},"min":{"1min":0.371,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":3.77,"15min":3.77},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.99299999999999999,"15min":1.0329999999999999},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4299999999999999,"5min":3.282,"15min":3.282},"last":0.81599999999999995}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":1.0009999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.357,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":1.718},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82999999999999996,"15min":0.85799999999999998},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.296,"5min":1.774,"15min":1.774},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.028,"15min":1.054},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4430000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0820000000000001},"min":{"1min":0.63200000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5589999999999999,"5min":3.121,"15min":3.121},"last":1.573}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2170000000000001},"min":{"1min":0.81000000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5489999999999999,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.3899999999999999,"15min":1.4239999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6970000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.1799999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.4379999999999999,"15min":1.464},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7709999999999999,"5min":5.75,"15min":5.75},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.4099999999999999,"15min":1.456},"min":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5089999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":2.0350000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.323,"15min":1.319},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":7.415,"15min":7.415},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.7130000000000001,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":1.5169999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.3500000000000001,"15min":1.3799999999999999},"min":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.552,"5min":3.633,"15min":3.633},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.504,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.899,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.4630000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:04:35.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-31T05:04:35.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.926+0000 7f55ffe00640 1 Processor -- start 2024-01-31T05:04:35.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.926+0000 7f55ffe00640 1 -- start start 2024-01-31T05:04:35.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.927+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 0x7f55f80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.927+0000 7f55ffe00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80d1b20 con 0x7f55f80cfca0 2024-01-31T05:04:35.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.927+0000 7f55ffe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80d1e00 con 0x7f55f80b3e90 2024-01-31T05:04:35.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.927+0000 7f55ffe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80d20e0 con 0x7f55f80d3bb0 2024-01-31T05:04:35.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.928+0000 7f55fd400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f55f80cfca0 0x7f55f80b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47524/0 (socket says 172.21.15.97:47524) 2024-01-31T05:04:35.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.928+0000 7f55fd400640 1 -- 172.21.15.97:0/2273251330 learned_addr learned my addr 172.21.15.97:0/2273251330 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:35.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.928+0000 7f55fca00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 0x7f55f80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.928+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219093846 0 0) 0x7f55f80d1b20 con 0x7f55f80cfca0 2024-01-31T05:04:35.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.928+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55d8003520 con 0x7f55f80cfca0 2024-01-31T05:04:35.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.928+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 293441536 0 0) 0x7f55d8003520 con 0x7f55f80cfca0 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 msgr2=0x7f55f80b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 0x7f55f80b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 >> v1:172.21.15.188:6789/0 conn(0x7f55f80b3e90 legacy=0x7f55f80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80d4120 con 0x7f55f80cfca0 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f55e0001670 con 0x7f55f80cfca0 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 >> v1:172.21.15.97:6789/0 conn(0x7f55f80cfca0 legacy=0x7f55f80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 0x7f55d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55d8003f20 0x7f55d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8004d00 0x7f55d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80d20e0 con 0x7f55d8003f20 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d800ba60 con 0x7f55d8004d00 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d800bc20 con 0x7f55f80d3bb0 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55fca00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55d8003f20 0x7f55d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55fca00640 1 -- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 msgr2=0x7f55d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55fca00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f80d3bb0 0x7f55d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55fca00640 1 -- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8004d00 msgr2=0x7f55d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:35.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55fca00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8004d00 0x7f55d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.929+0000 7f55fca00640 1 -- 172.21.15.97:0/2273251330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d8003520 con 0x7f55d8003f20 2024-01-31T05:04:35.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.930+0000 7f55fca00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55d8003f20 0x7f55d80047c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009fc0 tx=0x7f55d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210039342756ca2e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e802cc60 con 0x7f55d8003f20 2024-01-31T05:04:35.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e800f040 con 0x7f55d8003f20 2024-01-31T05:04:35.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55f7e00640 1 -- 172.21.15.97:0/2273251330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8034440 con 0x7f55d8003f20 2024-01-31T05:04:35.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55ffe00640 1 -- 172.21.15.97:0/2273251330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55d8003f20 msgr2=0x7f55d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2273251330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55d8003f20 0x7f55d80047c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009fc0 tx=0x7f55d80057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55fde00640 1 -- 172.21.15.97:0/2273251330 reap_dead start 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55ffe00640 1 -- 172.21.15.97:0/2273251330 shutdown_connections 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55ffe00640 1 -- 172.21.15.97:0/2273251330 >> 172.21.15.97:0/2273251330 conn(0x7f55f809c570 msgr2=0x7f55f80d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55ffe00640 1 -- 172.21.15.97:0/2273251330 shutdown_connections 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.931+0000 7f55ffe00640 1 -- 172.21.15.97:0/2273251330 wait complete. 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.933+0000 7f55ffe00640 1 Processor -- start 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.933+0000 7f55ffe00640 1 -- start start 2024-01-31T05:04:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.934+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8003f20 0x7f55f815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.934+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f815fb80 0x7f55f8160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.934+0000 7f55ffe00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 0x7f55f80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.934+0000 7f55ffe00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c6e30 con 0x7f55f80cc530 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.934+0000 7f55ffe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8160540 con 0x7f55d8003f20 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.934+0000 7f55ffe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80ccef0 con 0x7f55f815fb80 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 0x7f55f80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 0x7f55f80cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59720/0 (socket says 172.21.15.97:59720) 2024-01-31T05:04:35.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 -- 172.21.15.97:0/2793571225 learned_addr learned my addr 172.21.15.97:0/2793571225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fca00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f815fb80 0x7f55f8160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fd400640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8003f20 0x7f55f815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 -- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f815fb80 msgr2=0x7f55f8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f815fb80 0x7f55f8160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 -- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8003f20 msgr2=0x7f55f815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8003f20 0x7f55f815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 -- 172.21.15.97:0/2793571225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8160740 con 0x7f55f80cc530 2024-01-31T05:04:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.935+0000 7f55fde00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 0x7f55f80cc9b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f55f81613a0 tx=0x7f55f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.936+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00073e0 con 0x7f55f80cc530 2024-01-31T05:04:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.936+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0007540 con 0x7f55f80cc530 2024-01-31T05:04:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.936+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0005a00 con 0x7f55f80cc530 2024-01-31T05:04:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.936+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f8160a20 con 0x7f55f80cc530 2024-01-31T05:04:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.936+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f8160fe0 con 0x7f55f80cc530 2024-01-31T05:04:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.937+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f80c6f70 con 0x7f55f80cc530 2024-01-31T05:04:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.938+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f55f0005b60 con 0x7f55f80cc530 2024-01-31T05:04:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.939+0000 7f55f5600640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f55d006af00 0x7f55d006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.939+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f55f0088190 con 0x7f55f80cc530 2024-01-31T05:04:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.939+0000 7f55fd400640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f55d006af00 0x7f55d006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.940+0000 7f55fd400640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f55d006af00 0x7f55d006d3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f55e0000f60 tx=0x7f55e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:35.943+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f55f80c6f70 con 0x7f55f80cc530 2024-01-31T05:04:36.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.138+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55f80c6f70 con 0x7f55f80cc530 2024-01-31T05:04:36.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.140+0000 7f55f5600640 1 -- 172.21.15.97:0/2793571225 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13192 (secure 0 0 0) 0x7f55f80c6f70 con 0x7f55f80cc530 2024-01-31T05:04:36.140 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:36.140 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":58,"fsid":"dc34a409-d027-49db-a591-dbd89a074581","created":"2024-01-31T05:00:26.821573+0000","modified":"2024-01-31T05:04:32.056640+0000","last_up_change":"2024-01-31T05:00:38.948149+0000","last_in_change":"2024-01-31T05:00:30.525121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-31T05:00:39.613314+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c82a8d88-7203-47b9-8e70-5023862b4236","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6801","nonce":2799621490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6803","nonce":2799621490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6807","nonce":2799621490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":2799621490},{"type":"v1","addr":"172.21.15.97:6805","nonce":2799621490}]},"public_addr":"172.21.15.97:6801/2799621490","cluster_addr":"172.21.15.97:6803/2799621490","heartbeat_back_addr":"172.21.15.97:6807/2799621490","heartbeat_front_addr":"172.21.15.97:6805/2799621490","state":["exists","up"]},{"osd":1,"uuid":"f6bdaa6b-d5fa-405e-875b-e7cf225c2a17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6813","nonce":2289218318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6815","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6817","nonce":2289218318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6825","nonce":2289218318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2289218318},{"type":"v1","addr":"172.21.15.97:6820","nonce":2289218318}]},"public_addr":"172.21.15.97:6813/2289218318","cluster_addr":"172.21.15.97:6817/2289218318","heartbeat_back_addr":"172.21.15.97:6825/2289218318","heartbeat_front_addr":"172.21.15.97:6820/2289218318","state":["exists","up"]},{"osd":2,"uuid":"b73a2bad-7afc-40e1-846f-1dc55e93edd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6809","nonce":2991223614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6812","nonce":2991223614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6822","nonce":2991223614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2991223614},{"type":"v1","addr":"172.21.15.97:6816","nonce":2991223614}]},"public_addr":"172.21.15.97:6809/2991223614","cluster_addr":"172.21.15.97:6812/2991223614","heartbeat_back_addr":"172.21.15.97:6822/2991223614","heartbeat_front_addr":"172.21.15.97:6816/2991223614","state":["exists","up"]},{"osd":3,"uuid":"6dacd91a-90c8-4103-9c4c-3b5d22ad8b60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6824","nonce":4184396147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6827","nonce":4184396147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6831","nonce":4184396147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":4184396147},{"type":"v1","addr":"172.21.15.97:6829","nonce":4184396147}]},"public_addr":"172.21.15.97:6824/4184396147","cluster_addr":"172.21.15.97:6827/4184396147","heartbeat_back_addr":"172.21.15.97:6831/4184396147","heartbeat_front_addr":"172.21.15.97:6829/4184396147","state":["exists","up"]},{"osd":4,"uuid":"dfe326eb-d40b-4a65-9998-1133493036f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6825","nonce":2903566089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6827","nonce":2903566089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6831","nonce":2903566089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":2903566089},{"type":"v1","addr":"172.21.15.188:6829","nonce":2903566089}]},"public_addr":"172.21.15.188:6825/2903566089","cluster_addr":"172.21.15.188:6827/2903566089","heartbeat_back_addr":"172.21.15.188:6831/2903566089","heartbeat_front_addr":"172.21.15.188:6829/2903566089","state":["exists","up"]},{"osd":5,"uuid":"fe5fb1bb-7949-4c7c-a8c3-844d3ea4a42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6809","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6811","nonce":576621421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6813","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6815","nonce":576621421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6823","nonce":576621421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6817","nonce":576621421},{"type":"v1","addr":"172.21.15.188:6819","nonce":576621421}]},"public_addr":"172.21.15.188:6811/576621421","cluster_addr":"172.21.15.188:6815/576621421","heartbeat_back_addr":"172.21.15.188:6823/576621421","heartbeat_front_addr":"172.21.15.188:6819/576621421","state":["exists","up"]},{"osd":6,"uuid":"0f493702-f50c-47f9-abe7-82feeea46aac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6801","nonce":3025033562}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6803","nonce":3025033562}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6807","nonce":3025033562}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3025033562},{"type":"v1","addr":"172.21.15.188:6805","nonce":3025033562}]},"public_addr":"172.21.15.188:6801/3025033562","cluster_addr":"172.21.15.188:6803/3025033562","heartbeat_back_addr":"172.21.15.188:6807/3025033562","heartbeat_front_addr":"172.21.15.188:6805/3025033562","state":["exists","up"]},{"osd":7,"uuid":"016a4ce2-df66-41f4-b2fa-9e1db0c7ef34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6810","nonce":2866753840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6814","nonce":2866753840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6822","nonce":2866753840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2866753840},{"type":"v1","addr":"172.21.15.188:6818","nonce":2866753840}]},"public_addr":"172.21.15.188:6810/2866753840","cluster_addr":"172.21.15.188:6814/2866753840","heartbeat_back_addr":"172.21.15.188:6822/2866753840","heartbeat_front_addr":"172.21.15.188:6818/2866753840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.325405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.893191+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.895196+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.883071+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.873680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:37.563917+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.762334+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-31T05:00:35.838258+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/3986947126":"2024-02-01T05:04:29.039092+0000","172.21.15.97:6836/3986947126":"2024-02-01T05:04:29.039092+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f55d006af00 msgr2=0x7f55d006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f55d006af00 0x7f55d006d3c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f55e0000f60 tx=0x7f55e0009040 comp rx=0 tx=0).stop 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 msgr2=0x7f55f80cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 0x7f55f80cc9b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f55f81613a0 tx=0x7f55f000c820 comp rx=0 tx=0).stop 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 shutdown_connections 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f55d006af00 0x7f55d006d3c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:36.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55f80cc530 0x7f55f80cc9b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:36.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55f815fb80 0x7f55f8160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:36.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 --2- 172.21.15.97:0/2793571225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55d8003f20 0x7f55f815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:36.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 >> 172.21.15.97:0/2793571225 conn(0x7f55f809c570 msgr2=0x7f55f80b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:36.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 shutdown_connections 2024-01-31T05:04:36.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:36.143+0000 7f55ffe00640 1 -- 172.21.15.97:0/2793571225 wait complete. 2024-01-31T05:04:37.163 INFO:tasks.ceph:Scrubbing osd.0 2024-01-31T05:04:37.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:37.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.385+0000 7f21a3a00640 1 Processor -- start 2024-01-31T05:04:37.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.385+0000 7f21a3a00640 1 -- start start 2024-01-31T05:04:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.386+0000 7f21a3a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f219c15d360 0x7f219c15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.386+0000 7f21a3a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219c0cc9b0 con 0x7f219c0ae220 2024-01-31T05:04:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f21a3a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219c0ca050 con 0x7f219c0703b0 2024-01-31T05:04:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f21a3a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219c0cd5a0 con 0x7f219c15d360 2024-01-31T05:04:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f2193c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f219c0ae220 0x7f219c0c9940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47540/0 (socket says 172.21.15.97:47540) 2024-01-31T05:04:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f2193c00640 1 -- 172.21.15.97:0/4097983682 learned_addr learned my addr 172.21.15.97:0/4097983682 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175152765 0 0) 0x7f219c0cc9b0 con 0x7f219c0ae220 2024-01-31T05:04:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f217c003520 con 0x7f219c0ae220 2024-01-31T05:04:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f21a1000640 1 --2- 172.21.15.97:0/4097983682 >> v2:172.21.15.188:3300/0 conn(0x7f219c15d360 0x7f219c15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.387+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1155530938 0 0) 0x7f217c003520 con 0x7f219c0ae220 2024-01-31T05:04:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 >> v2:172.21.15.188:3300/0 conn(0x7f219c15d360 msgr2=0x7f219c15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 --2- 172.21.15.97:0/4097983682 >> v2:172.21.15.188:3300/0 conn(0x7f219c15d360 0x7f219c15e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 >> v1:172.21.15.188:6789/0 conn(0x7f219c0703b0 legacy=0x7f219c0707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0ae6b0 con 0x7f219c0ae220 2024-01-31T05:04:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7f2184001670 con 0x7f219c0ae220 2024-01-31T05:04:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 >> v1:172.21.15.97:6789/0 conn(0x7f219c0ae220 legacy=0x7f219c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f217c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f217c0047c0 0x7f217c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f217c005380 0x7f217c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0cd5a0 con 0x7f217c003970 2024-01-31T05:04:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c00c210 con 0x7f217c0047c0 2024-01-31T05:04:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c00c3d0 con 0x7f217c005380 2024-01-31T05:04:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1000640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f217c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f2193c00640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f217c0047c0 0x7f217c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1000640 1 -- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f217c005380 msgr2=0x7f217c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1000640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f217c005380 0x7f217c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1000640 1 -- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f217c0047c0 msgr2=0x7f217c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1000640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f217c0047c0 0x7f217c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1000640 1 -- 172.21.15.97:0/4097983682 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c003520 con 0x7f217c003970 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.388+0000 7f21a1a00640 1 -- 172.21.15.97:0/4097983682 reap_dead start 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.389+0000 7f21a1000640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f217c003d90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f218c00ba30 tx=0x7f217c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a3a2a4df8a3720 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c032680 con 0x7f217c003970 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c0327e0 con 0x7f217c003970 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f219be00640 1 -- 172.21.15.97:0/4097983682 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c032b70 con 0x7f217c003970 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 -- 172.21.15.97:0/4097983682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 msgr2=0x7f217c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f217c003d90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f218c00ba30 tx=0x7f217c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 -- 172.21.15.97:0/4097983682 shutdown_connections 2024-01-31T05:04:37.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 --2- 172.21.15.97:0/4097983682 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f217c003d90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 -- 172.21.15.97:0/4097983682 >> 172.21.15.97:0/4097983682 conn(0x7f219c0cede0 msgr2=0x7f219c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:37.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 -- 172.21.15.97:0/4097983682 shutdown_connections 2024-01-31T05:04:37.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.390+0000 7f21a3a00640 1 -- 172.21.15.97:0/4097983682 wait complete. 2024-01-31T05:04:37.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.392+0000 7f21a3a00640 1 Processor -- start 2024-01-31T05:04:37.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.392+0000 7f21a3a00640 1 -- start start 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a3a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f219c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a3a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f219c099c20 0x7f219c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a3a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219c0c7ff0 0x7f219c06c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a3a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c89c0 con 0x7f217c003970 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a3a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c8ca0 con 0x7f219c099c20 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a3a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c5f50 con 0x7f219c0c7ff0 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.393+0000 7f21a1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f219c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f219c14e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59740/0 (socket says 172.21.15.97:59740) 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 -- 172.21.15.97:0/687913583 learned_addr learned my addr 172.21.15.97:0/687913583 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1a00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219c0c7ff0 0x7f219c06c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f2193c00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f219c099c20 0x7f219c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 -- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219c0c7ff0 msgr2=0x7f219c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f219c0c7ff0 0x7f219c06c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 -- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f219c099c20 msgr2=0x7f219c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f219c099c20 0x7f219c0c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f21a1000640 1 -- 172.21.15.97:0/687913583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0c6230 con 0x7f217c003970 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.394+0000 7f2193c00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f219c099c20 0x7f219c0c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.395+0000 7f21a1000640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f219c14e370 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f218c00bb50 tx=0x7f218c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.395+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c0308a0 con 0x7f217c003970 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.395+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c030a20 con 0x7f217c003970 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.395+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c030dd0 con 0x7f217c003970 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.395+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c09a080 con 0x7f217c003970 2024-01-31T05:04:37.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.395+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c0c6510 con 0x7f217c003970 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.396+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f219c0cd1c0 con 0x7f217c003970 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.397+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f218c03b420 con 0x7f217c003970 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.398+0000 7f2199600640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f217406af00 0x7f217406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.398+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f218c0ab5c0 con 0x7f217c003970 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.398+0000 7f2193c00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f217406af00 0x7f217406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.398+0000 7f2199600640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7f2174071200 0x7f2174073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.398+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 --> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2174073d10 con 0x7f2174071200 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.398+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f218c0adc80 con 0x7f217c003970 2024-01-31T05:04:37.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.399+0000 7f21a1a00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7f2174071200 0x7f2174073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.399+0000 7f2193c00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f217406af00 0x7f217406d3c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f219c160560 tx=0x7f2184009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.399+0000 7f21a1a00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7f2174071200 0x7f2174073640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.400+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== osd.0 v2:172.21.15.97:6800/2799621490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2174073d10 con 0x7f2174071200 2024-01-31T05:04:37.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.414+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 --> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f219c0ccee0 con 0x7f2174071200 2024-01-31T05:04:37.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.415+0000 7f2199600640 1 -- 172.21.15.97:0/687913583 <== osd.0 v2:172.21.15.97:6800/2799621490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2194001850 con 0x7f2174071200 2024-01-31T05:04:37.414 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:37.415 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:37.415 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.415+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7f2174071200 msgr2=0x7f2174073640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.415+0000 7f21a3a00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6800/2799621490,v1:172.21.15.97:6801/2799621490] conn(0x7f2174071200 0x7f2174073640 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f217406af00 msgr2=0x7f217406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f217406af00 0x7f217406d3c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f219c160560 tx=0x7f2184009040 comp rx=0 tx=0).stop 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 msgr2=0x7f219c14e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 --2- 172.21.15.97:0/687913583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c003970 0x7f219c14e370 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f218c00bb50 tx=0x7f218c004730 comp rx=0 tx=0).stop 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a1a00640 1 -- 172.21.15.97:0/687913583 reap_dead start 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 shutdown_connections 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 >> 172.21.15.97:0/687913583 conn(0x7f219c0cede0 msgr2=0x7f219c0ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 shutdown_connections 2024-01-31T05:04:37.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.416+0000 7f21a3a00640 1 -- 172.21.15.97:0/687913583 wait complete. 2024-01-31T05:04:37.433 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-31T05:04:37.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.638+0000 7fd596a00640 1 Processor -- start 2024-01-31T05:04:37.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.639+0000 7fd596a00640 1 -- start start 2024-01-31T05:04:37.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd596a00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5900cfad0 0x7fd5900b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd596a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd596a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd5900cfeb0 0x7fd5901612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd596a00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5900c6bd0 con 0x7fd5900cfad0 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd596a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5900d1fd0 con 0x7fd5900cfeb0 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd596a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900d3fe0 con 0x7fd5900b41f0 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd595600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5900cfad0 0x7fd5900b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd595600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5900cfad0 0x7fd5900b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59750/0 (socket says 172.21.15.97:59750) 2024-01-31T05:04:37.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.640+0000 7fd595600640 1 -- 172.21.15.97:0/3229332534 learned_addr learned my addr 172.21.15.97:0/3229332534 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd594c00640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd596000640 1 --2- 172.21.15.97:0/3229332534 >> v2:172.21.15.188:3300/0 conn(0x7fd5900cfeb0 0x7fd5901612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd595600640 1 -- 172.21.15.97:0/3229332534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 msgr2=0x7fd5900b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd595600640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd595600640 1 -- 172.21.15.97:0/3229332534 >> v2:172.21.15.188:3300/0 conn(0x7fd5900cfeb0 msgr2=0x7fd5901612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd595600640 1 --2- 172.21.15.97:0/3229332534 >> v2:172.21.15.188:3300/0 conn(0x7fd5900cfeb0 0x7fd5901612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd595600640 1 -- 172.21.15.97:0/3229332534 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900d3d40 con 0x7fd5900cfad0 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd594c00640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.641+0000 7fd596000640 1 --2- 172.21.15.97:0/3229332534 >> v2:172.21.15.188:3300/0 conn(0x7fd5900cfeb0 0x7fd5901612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:37.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.642+0000 7fd595600640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfad0 0x7fd5900b3000 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd580009e50 tx=0x7fd58002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cfc87375a3c86f server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.642+0000 7fd58fe00640 1 -- 172.21.15.97:0/3229332534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58002cb00 con 0x7fd5900cfad0 2024-01-31T05:04:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.642+0000 7fd58fe00640 1 -- 172.21.15.97:0/3229332534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58002cc60 con 0x7fd5900cfad0 2024-01-31T05:04:37.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.642+0000 7fd58fe00640 1 -- 172.21.15.97:0/3229332534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5800343f0 con 0x7fd5900cfad0 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 -- 172.21.15.97:0/3229332534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfad0 msgr2=0x7fd5900b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfad0 0x7fd5900b3000 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd580009e50 tx=0x7fd58002bf60 comp rx=0 tx=0).stop 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 -- 172.21.15.97:0/3229332534 shutdown_connections 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 --2- 172.21.15.97:0/3229332534 >> v2:172.21.15.188:3300/0 conn(0x7fd5900cfeb0 0x7fd5901612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 --2- 172.21.15.97:0/3229332534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfad0 0x7fd5900b3000 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 -- 172.21.15.97:0/3229332534 >> 172.21.15.97:0/3229332534 conn(0x7fd5900d3080 msgr2=0x7fd5900d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:37.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 -- 172.21.15.97:0/3229332534 shutdown_connections 2024-01-31T05:04:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.643+0000 7fd596a00640 1 -- 172.21.15.97:0/3229332534 wait complete. 2024-01-31T05:04:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.645+0000 7fd596a00640 1 Processor -- start 2024-01-31T05:04:37.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.645+0000 7fd596a00640 1 -- start start 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.646+0000 7fd596a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.646+0000 7fd596a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5900cfad0 0x7fd5900b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.646+0000 7fd596a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 0x7fd59015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.646+0000 7fd596a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900b7080 con 0x7fd5900cfeb0 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.646+0000 7fd596a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5900b6da0 con 0x7fd5900cfad0 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.646+0000 7fd596a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5901f67e0 con 0x7fd5900b41f0 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 0x7fd59015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 0x7fd59015c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59752/0 (socket says 172.21.15.97:59752) 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 -- 172.21.15.97:0/1060627707 learned_addr learned my addr 172.21.15.97:0/1060627707 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd595600640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd594c00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5900cfad0 0x7fd5900b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 -- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 msgr2=0x7fd5900b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 -- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5900cfad0 msgr2=0x7fd5900b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5900cfad0 0x7fd5900b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 -- 172.21.15.97:0/1060627707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580009b30 con 0x7fd5900cfeb0 2024-01-31T05:04:37.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd594c00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5900cfad0 0x7fd5900b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:37.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.647+0000 7fd596000640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 0x7fd59015c9a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd58800c7c0 tx=0x7fd58800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.648+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588004230 con 0x7fd5900cfeb0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.648+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd588004390 con 0x7fd5900cfeb0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.648+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588007860 con 0x7fd5900cfeb0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.648+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5901f6ac0 con 0x7fd5900cfeb0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.648+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5900b2af0 con 0x7fd5900cfeb0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.649+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5901f6fe0 con 0x7fd5900cfeb0 2024-01-31T05:04:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.650+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd5880079c0 con 0x7fd5900cfeb0 2024-01-31T05:04:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.651+0000 7fd58d600640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd56806aed0 0x7fd56806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.651+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd588089680 con 0x7fd5900cfeb0 2024-01-31T05:04:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.651+0000 7fd595600640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd56806aed0 0x7fd56806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:37.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.652+0000 7fd595600640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd56806aed0 0x7fd56806d390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd580035000 tx=0x7fd580030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.656+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd5901f6fe0 con 0x7fd5900cfeb0 2024-01-31T05:04:37.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.851+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd590067fc0 con 0x7fd56806aed0 2024-01-31T05:04:37.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.852+0000 7fd58d600640 1 -- 172.21.15.97:0/1060627707 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd590067fc0 con 0x7fd56806aed0 2024-01-31T05:04:37.851 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2024-01-31T05:04:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd56806aed0 msgr2=0x7fd56806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd56806aed0 0x7fd56806d390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd580035000 tx=0x7fd580030000 comp rx=0 tx=0).stop 2024-01-31T05:04:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 msgr2=0x7fd59015c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 0x7fd59015c9a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd58800c7c0 tx=0x7fd58800cc80 comp rx=0 tx=0).stop 2024-01-31T05:04:37.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 shutdown_connections 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd56806aed0 0x7fd56806d390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5900cfeb0 0x7fd59015c9a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5900cfad0 0x7fd5900b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 --2- 172.21.15.97:0/1060627707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5900b41f0 0x7fd5900b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 >> 172.21.15.97:0/1060627707 conn(0x7fd5900d3080 msgr2=0x7fd59006b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.854+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 shutdown_connections 2024-01-31T05:04:37.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:37.855+0000 7fd596a00640 1 -- 172.21.15.97:0/1060627707 wait complete. 2024-01-31T05:04:37.872 INFO:tasks.ceph:Scrubbing osd.1 2024-01-31T05:04:37.872 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:38.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.091+0000 7ff3b6000640 1 Processor -- start 2024-01-31T05:04:38.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.091+0000 7ff3b6000640 1 -- start start 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.092+0000 7ff3b6000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3b00ad780 0x7ff3b00ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3b6000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3b0070910 0x7ff3b0070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3b6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b00712e0 0x7ff3b015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3b6000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00adb60 con 0x7ff3b0070910 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3b6000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b0072720 con 0x7ff3b00ad780 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3b6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b006e4b0 con 0x7ff3b00712e0 2024-01-31T05:04:38.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3b0070910 0x7ff3b0070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3b0070910 0x7ff3b0070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59756/0 (socket says 172.21.15.97:59756) 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 -- 172.21.15.97:0/2874483413 learned_addr learned my addr 172.21.15.97:0/2874483413 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3afe00640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b00712e0 0x7ff3b015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3af400640 1 --2- 172.21.15.97:0/2874483413 >> v2:172.21.15.188:3300/0 conn(0x7ff3b00ad780 0x7ff3b00ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 -- 172.21.15.97:0/2874483413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b00712e0 msgr2=0x7ff3b015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b00712e0 0x7ff3b015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 -- 172.21.15.97:0/2874483413 >> v2:172.21.15.188:3300/0 conn(0x7ff3b00ad780 msgr2=0x7ff3b00ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 --2- 172.21.15.97:0/2874483413 >> v2:172.21.15.188:3300/0 conn(0x7ff3b00ad780 0x7ff3b00ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3aea00640 1 -- 172.21.15.97:0/2874483413 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b006dea0 con 0x7ff3b0070910 2024-01-31T05:04:38.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.093+0000 7ff3afe00640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b00712e0 0x7ff3b015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.094+0000 7ff3aea00640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b0070910 0x7ff3b0070da0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00cb7c0 tx=0x7ff39802b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ff3b91af44fd72 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.095+0000 7ff3ae000640 1 -- 172.21.15.97:0/2874483413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980317f0 con 0x7ff3b0070910 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.095+0000 7ff3ae000640 1 -- 172.21.15.97:0/2874483413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398031950 con 0x7ff3b0070910 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.095+0000 7ff3ae000640 1 -- 172.21.15.97:0/2874483413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398031ce0 con 0x7ff3b0070910 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.095+0000 7ff3b6000640 1 -- 172.21.15.97:0/2874483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b0070910 msgr2=0x7ff3b0070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.095+0000 7ff3b6000640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b0070910 0x7ff3b0070da0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00cb7c0 tx=0x7ff39802b890 comp rx=0 tx=0).stop 2024-01-31T05:04:38.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 -- 172.21.15.97:0/2874483413 shutdown_connections 2024-01-31T05:04:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b00712e0 0x7ff3b015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 --2- 172.21.15.97:0/2874483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b0070910 0x7ff3b0070da0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 --2- 172.21.15.97:0/2874483413 >> v2:172.21.15.188:3300/0 conn(0x7ff3b00ad780 0x7ff3b00ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 -- 172.21.15.97:0/2874483413 >> 172.21.15.97:0/2874483413 conn(0x7ff3b00cede0 msgr2=0x7ff3b00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 -- 172.21.15.97:0/2874483413 shutdown_connections 2024-01-31T05:04:38.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.096+0000 7ff3b6000640 1 -- 172.21.15.97:0/2874483413 wait complete. 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.098+0000 7ff3b6000640 1 Processor -- start 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.098+0000 7ff3b6000640 1 -- start start 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3b6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b0070910 0x7ff3b00c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3b6000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b00712e0 0x7ff3b01dd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3b6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b00ad780 0x7ff3b00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3b6000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b01dd880 con 0x7ff3b00712e0 2024-01-31T05:04:38.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3b6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c4460 con 0x7ff3b00ad780 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3b6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c4fe0 con 0x7ff3b0070910 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b00712e0 0x7ff3b01dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b00712e0 0x7ff3b01dd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59766/0 (socket says 172.21.15.97:59766) 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 -- 172.21.15.97:0/718244391 learned_addr learned my addr 172.21.15.97:0/718244391 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3af400640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b0070910 0x7ff3b00c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 -- 172.21.15.97:0/718244391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b0070910 msgr2=0x7ff3b00c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3b0070910 0x7ff3b00c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3afe00640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b00ad780 0x7ff3b00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 -- 172.21.15.97:0/718244391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b00ad780 msgr2=0x7ff3b00b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3b00ad780 0x7ff3b00b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.099+0000 7ff3aea00640 1 -- 172.21.15.97:0/718244391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980094d0 con 0x7ff3b00712e0 2024-01-31T05:04:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.100+0000 7ff3aea00640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b00712e0 0x7ff3b01dd340 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.100+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398032990 con 0x7ff3b00712e0 2024-01-31T05:04:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.100+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff398032b10 con 0x7ff3b00712e0 2024-01-31T05:04:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.100+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3980453f0 con 0x7ff3b00712e0 2024-01-31T05:04:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.100+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00c46b0 con 0x7ff3b00712e0 2024-01-31T05:04:38.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.100+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b00c4c70 con 0x7ff3b00712e0 2024-01-31T05:04:38.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.101+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3b006e140 con 0x7ff3b00712e0 2024-01-31T05:04:38.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.102+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff398045550 con 0x7ff3b00712e0 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.103+0000 7ff3a7400640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38806aed0 0x7ff38806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.103+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff3980abbf0 con 0x7ff3b00712e0 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.103+0000 7ff3af400640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38806aed0 0x7ff38806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.103+0000 7ff3a7400640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7ff3880711d0 0x7ff388073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.103+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 --> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff388073ce0 con 0x7ff3880711d0 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.104+0000 7ff3afe00640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7ff3880711d0 0x7ff388073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.104+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff398076be0 con 0x7ff3b00712e0 2024-01-31T05:04:38.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.104+0000 7ff3af400640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38806aed0 0x7ff38806d390 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0006fd0 tx=0x7ff3a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.104+0000 7ff3afe00640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7ff3880711d0 0x7ff388073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.105+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== osd.1 v2:172.21.15.97:6811/2289218318 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff388073ce0 con 0x7ff3880711d0 2024-01-31T05:04:38.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.120+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 --> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff3b0071f70 con 0x7ff3880711d0 2024-01-31T05:04:38.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.121+0000 7ff3a7400640 1 -- 172.21.15.97:0/718244391 <== osd.1 v2:172.21.15.97:6811/2289218318 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff3a800d4b0 con 0x7ff3880711d0 2024-01-31T05:04:38.121 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:38.121 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:38.122 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:38.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.121+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7ff3880711d0 msgr2=0x7ff388073610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.121+0000 7ff3b6000640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6811/2289218318,v1:172.21.15.97:6813/2289218318] conn(0x7ff3880711d0 0x7ff388073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38806aed0 msgr2=0x7ff38806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff38806aed0 0x7ff38806d390 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0006fd0 tx=0x7ff3a0008040 comp rx=0 tx=0).stop 2024-01-31T05:04:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b00712e0 msgr2=0x7ff3b01dd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 --2- 172.21.15.97:0/718244391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3b00712e0 0x7ff3b01dd340 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff3980046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3afe00640 1 -- 172.21.15.97:0/718244391 reap_dead start 2024-01-31T05:04:38.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 shutdown_connections 2024-01-31T05:04:38.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 >> 172.21.15.97:0/718244391 conn(0x7ff3b00cede0 msgr2=0x7ff3b00af220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:38.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 shutdown_connections 2024-01-31T05:04:38.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.122+0000 7ff3b6000640 1 -- 172.21.15.97:0/718244391 wait complete. 2024-01-31T05:04:38.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-31T05:04:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.345+0000 7fb8b7400640 1 Processor -- start 2024-01-31T05:04:38.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.346+0000 7fb8b7400640 1 -- start start 2024-01-31T05:04:38.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8b7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8b00cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8b7400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b006d1a0 con 0x7fb8b015b400 2024-01-31T05:04:38.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8b7400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b00cd6f0 con 0x7fb8b00cb810 2024-01-31T05:04:38.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8b7400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00cc960 con 0x7fb8b00ccd00 2024-01-31T05:04:38.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8afe00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb8b015b400 0x7fb8b006d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47550/0 (socket says 172.21.15.97:47550) 2024-01-31T05:04:38.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8afe00640 1 -- 172.21.15.97:0/3390803925 learned_addr learned my addr 172.21.15.97:0/3390803925 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:38.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.347+0000 7fb8b5400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8b00cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533879978 0 0) 0x7fb8b006d1a0 con 0x7fb8b015b400 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb890003520 con 0x7fb8b015b400 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 870258651 0 0) 0x7fb890003520 con 0x7fb8b015b400 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 msgr2=0x7fb8b00cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8b00cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 >> v1:172.21.15.188:6789/0 conn(0x7fb8b00cb810 legacy=0x7fb8b00cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b006c600 con 0x7fb8b015b400 2024-01-31T05:04:38.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fb8980019f0 con 0x7fb8b015b400 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 >> v1:172.21.15.97:6789/0 conn(0x7fb8b015b400 legacy=0x7fb8b006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900039a0 0x7fb890003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb890005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00cc960 con 0x7fb890004d00 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89000ba60 con 0x7fb8900039a0 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89000bc20 con 0x7fb8b00ccd00 2024-01-31T05:04:38.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.348+0000 7fb8b5400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb890005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8b4a00640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900039a0 0x7fb890003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8afe00640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8b5400640 1 -- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 msgr2=0x7fb8900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8b5400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8b5400640 1 -- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900039a0 msgr2=0x7fb890003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8b5400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900039a0 0x7fb890003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8b5400640 1 -- 172.21.15.97:0/3390803925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890003520 con 0x7fb890004d00 2024-01-31T05:04:38.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.349+0000 7fb8afe00640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b00ccd00 0x7fb8900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:38.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.350+0000 7fb8b5400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb890005180 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009a90 tx=0x7fb8900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72cf0b8522e75db server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.350+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802c680 con 0x7fb890004d00 2024-01-31T05:04:38.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.350+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a802c7e0 con 0x7fb890004d00 2024-01-31T05:04:38.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.350+0000 7fb8af400640 1 -- 172.21.15.97:0/3390803925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802cb70 con 0x7fb890004d00 2024-01-31T05:04:38.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b7400640 1 -- 172.21.15.97:0/3390803925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 msgr2=0x7fb890005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b7400640 1 --2- 172.21.15.97:0/3390803925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb890005180 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009a90 tx=0x7fb8900057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b5400640 1 -- 172.21.15.97:0/3390803925 reap_dead start 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b7400640 1 -- 172.21.15.97:0/3390803925 shutdown_connections 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b7400640 1 -- 172.21.15.97:0/3390803925 >> 172.21.15.97:0/3390803925 conn(0x7fb8b00b0a10 msgr2=0x7fb8b00b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b7400640 1 -- 172.21.15.97:0/3390803925 shutdown_connections 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.351+0000 7fb8b7400640 1 -- 172.21.15.97:0/3390803925 wait complete. 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.353+0000 7fb8b7400640 1 Processor -- start 2024-01-31T05:04:38.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.353+0000 7fb8b7400640 1 -- start start 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b7400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb8b01f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b015e000 0x7fb8b015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b7400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b01f2580 0x7fb8b01f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b7400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0152f00 con 0x7fb890004d00 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b7400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b01f2f40 con 0x7fb8b01f2580 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b7400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0156110 con 0x7fb8b015e000 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb8b01f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b4a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb8b01f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59776/0 (socket says 172.21.15.97:59776) 2024-01-31T05:04:38.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.354+0000 7fb8b4a00640 1 -- 172.21.15.97:0/279792483 learned_addr learned my addr 172.21.15.97:0/279792483 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8afe00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b015e000 0x7fb8b015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b5400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b01f2580 0x7fb8b01f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b4a00640 1 -- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b015e000 msgr2=0x7fb8b015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b4a00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b015e000 0x7fb8b015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b4a00640 1 -- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b01f2580 msgr2=0x7fb8b01f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b4a00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b01f2580 0x7fb8b01f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b4a00640 1 -- 172.21.15.97:0/279792483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0152660 con 0x7fb890004d00 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8afe00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b015e000 0x7fb8b015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b4a00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb8b01f3120 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000e490 tx=0x7fb8a000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00052e0 con 0x7fb890004d00 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0005440 con 0x7fb890004d00 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b0152940 con 0x7fb890004d00 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.355+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0003830 con 0x7fb890004d00 2024-01-31T05:04:38.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.356+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b015e480 con 0x7fb890004d00 2024-01-31T05:04:38.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.357+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b00677c0 con 0x7fb890004d00 2024-01-31T05:04:38.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.358+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb8a0003990 con 0x7fb890004d00 2024-01-31T05:04:38.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.358+0000 7fb8acc00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb88806af00 0x7fb88806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.359+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb8a0089270 con 0x7fb890004d00 2024-01-31T05:04:38.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.359+0000 7fb8afe00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb88806af00 0x7fb88806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.359+0000 7fb8afe00640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb88806af00 0x7fb88806d3c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0154a10 tx=0x7fb898009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.363+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb8a0054260 con 0x7fb890004d00 2024-01-31T05:04:38.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.553+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8b006ce30 con 0x7fb88806af00 2024-01-31T05:04:38.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.554+0000 7fb8acc00640 1 -- 172.21.15.97:0/279792483 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb89800d040 con 0x7fb88806af00 2024-01-31T05:04:38.554 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2024-01-31T05:04:38.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb88806af00 msgr2=0x7fb88806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb88806af00 0x7fb88806d3c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0154a10 tx=0x7fb898009040 comp rx=0 tx=0).stop 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 msgr2=0x7fb8b01f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb8b01f3120 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000e490 tx=0x7fb8a000e950 comp rx=0 tx=0).stop 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 shutdown_connections 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b01f2580 0x7fb8b01f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8b015e000 0x7fb8b015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb88806af00 0x7fb88806d3c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 --2- 172.21.15.97:0/279792483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb890004d00 0x7fb8b01f3120 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 >> 172.21.15.97:0/279792483 conn(0x7fb8b00b0a10 msgr2=0x7fb8b00ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 shutdown_connections 2024-01-31T05:04:38.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.558+0000 7fb8b7400640 1 -- 172.21.15.97:0/279792483 wait complete. 2024-01-31T05:04:38.576 INFO:tasks.ceph:Scrubbing osd.2 2024-01-31T05:04:38.576 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.789+0000 7fd406200640 1 Processor -- start 2024-01-31T05:04:38.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.790+0000 7fd406200640 1 -- start start 2024-01-31T05:04:38.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd406200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd40015d3e0 0x7fd4000d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd406200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4000d0c30 0x7fd4000d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd406200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd4000d1b60 0x7fd4000cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd406200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4000d1ff0 con 0x7fd4000d1b60 2024-01-31T05:04:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd406200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4000d1090 con 0x7fd4000d0c30 2024-01-31T05:04:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd406200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40006b330 con 0x7fd40015d3e0 2024-01-31T05:04:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.791+0000 7fd405800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd4000d1b60 0x7fd4000cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd4000d1b60 0x7fd4000cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59790/0 (socket says 172.21.15.97:59790) 2024-01-31T05:04:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 -- 172.21.15.97:0/795207277 learned_addr learned my addr 172.21.15.97:0/795207277 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd3f7c00640 1 --2- 172.21.15.97:0/795207277 >> v2:172.21.15.188:3300/0 conn(0x7fd4000d0c30 0x7fd4000d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd404e00640 1 --2- 172.21.15.97:0/795207277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd40015d3e0 0x7fd4000d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 -- 172.21.15.97:0/795207277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd40015d3e0 msgr2=0x7fd4000d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 --2- 172.21.15.97:0/795207277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd40015d3e0 0x7fd4000d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 -- 172.21.15.97:0/795207277 >> v2:172.21.15.188:3300/0 conn(0x7fd4000d0c30 msgr2=0x7fd4000d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 --2- 172.21.15.97:0/795207277 >> v2:172.21.15.188:3300/0 conn(0x7fd4000d0c30 0x7fd4000d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.792+0000 7fd405800640 1 -- 172.21.15.97:0/795207277 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40006b610 con 0x7fd4000d1b60 2024-01-31T05:04:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.793+0000 7fd405800640 1 --2- 172.21.15.97:0/795207277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4000cdb70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd400072e20 tx=0x7fd3f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd1cb97f9d60f5d server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.793+0000 7fd3ffe00640 1 -- 172.21.15.97:0/795207277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8031870 con 0x7fd4000d1b60 2024-01-31T05:04:38.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.793+0000 7fd3ffe00640 1 -- 172.21.15.97:0/795207277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f80319d0 con 0x7fd4000d1b60 2024-01-31T05:04:38.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.793+0000 7fd3ffe00640 1 -- 172.21.15.97:0/795207277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8031d60 con 0x7fd4000d1b60 2024-01-31T05:04:38.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 -- 172.21.15.97:0/795207277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 msgr2=0x7fd4000cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 --2- 172.21.15.97:0/795207277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4000cdb70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd400072e20 tx=0x7fd3f802b910 comp rx=0 tx=0).stop 2024-01-31T05:04:38.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 -- 172.21.15.97:0/795207277 shutdown_connections 2024-01-31T05:04:38.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 --2- 172.21.15.97:0/795207277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4000cdb70 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 --2- 172.21.15.97:0/795207277 >> v2:172.21.15.188:3300/0 conn(0x7fd4000d0c30 0x7fd4000d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 --2- 172.21.15.97:0/795207277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd40015d3e0 0x7fd4000d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 -- 172.21.15.97:0/795207277 >> 172.21.15.97:0/795207277 conn(0x7fd4000d3060 msgr2=0x7fd4000d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 -- 172.21.15.97:0/795207277 shutdown_connections 2024-01-31T05:04:38.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.794+0000 7fd406200640 1 -- 172.21.15.97:0/795207277 wait complete. 2024-01-31T05:04:38.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.796+0000 7fd406200640 1 Processor -- start 2024-01-31T05:04:38.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.796+0000 7fd406200640 1 -- start start 2024-01-31T05:04:38.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd406200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4000d0c30 0x7fd4000cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd406200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4001f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd406200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd40015d3e0 0x7fd40015d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd406200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40015ec80 con 0x7fd4000d1b60 2024-01-31T05:04:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd406200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4001f2a10 con 0x7fd40015d3e0 2024-01-31T05:04:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd406200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40015de80 con 0x7fd4000d0c30 2024-01-31T05:04:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd3f7c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4001f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd3f7c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4001f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59792/0 (socket says 172.21.15.97:59792) 2024-01-31T05:04:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.797+0000 7fd3f7c00640 1 -- 172.21.15.97:0/3087569301 learned_addr learned my addr 172.21.15.97:0/3087569301 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd404e00640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4000d0c30 0x7fd4000cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd405800640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd40015d3e0 0x7fd40015d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3f7c00640 1 -- 172.21.15.97:0/3087569301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4000d0c30 msgr2=0x7fd4000cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3f7c00640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4000d0c30 0x7fd4000cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3f7c00640 1 -- 172.21.15.97:0/3087569301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd40015d3e0 msgr2=0x7fd40015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3f7c00640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd40015d3e0 0x7fd40015d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3f7c00640 1 -- 172.21.15.97:0/3087569301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80094d0 con 0x7fd4000d1b60 2024-01-31T05:04:38.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3f7c00640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4001f24d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800d5f0 tx=0x7fd3e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80125f0 con 0x7fd4000d1b60 2024-01-31T05:04:38.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e8012750 con 0x7fd4000d1b60 2024-01-31T05:04:38.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8012b00 con 0x7fd4000d1b60 2024-01-31T05:04:38.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40015e100 con 0x7fd4000d1b60 2024-01-31T05:04:38.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.798+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40015e6c0 con 0x7fd4000d1b60 2024-01-31T05:04:38.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.799+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd40015d7c0 con 0x7fd4000d1b60 2024-01-31T05:04:38.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.800+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd3e8012c60 con 0x7fd4000d1b60 2024-01-31T05:04:38.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.801+0000 7fd3fd600640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d806aed0 0x7fd3d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.801+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd3e80899c0 con 0x7fd4000d1b60 2024-01-31T05:04:38.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.802+0000 7fd404e00640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d806aed0 0x7fd3d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.802+0000 7fd3fd600640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fd3d80711d0 0x7fd3d8073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:38.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.802+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 --> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3d8073ce0 con 0x7fd3d80711d0 2024-01-31T05:04:38.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.802+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3e808e2a0 con 0x7fd4000d1b60 2024-01-31T05:04:38.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.802+0000 7fd405800640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fd3d80711d0 0x7fd3d8073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:38.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.802+0000 7fd404e00640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d806aed0 0x7fd3d806d390 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009940 tx=0x7fd3f0003cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.803+0000 7fd405800640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fd3d80711d0 0x7fd3d8073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:38.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.803+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== osd.2 v2:172.21.15.97:6808/2991223614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd3d8073ce0 con 0x7fd3d80711d0 2024-01-31T05:04:38.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.818+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 --> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd40006bbd0 con 0x7fd3d80711d0 2024-01-31T05:04:38.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.818+0000 7fd3fd600640 1 -- 172.21.15.97:0/3087569301 <== osd.2 v2:172.21.15.97:6808/2991223614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd3f8031bc0 con 0x7fd3d80711d0 2024-01-31T05:04:38.820 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:38.820 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:38.820 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:38.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fd3d80711d0 msgr2=0x7fd3d8073610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6808/2991223614,v1:172.21.15.97:6809/2991223614] conn(0x7fd3d80711d0 0x7fd3d8073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d806aed0 msgr2=0x7fd3d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd3d806aed0 0x7fd3d806d390 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009940 tx=0x7fd3f0003cd0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 msgr2=0x7fd4001f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:38.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 --2- 172.21.15.97:0/3087569301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4000d1b60 0x7fd4001f24d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800d5f0 tx=0x7fd3e800dab0 comp rx=0 tx=0).stop 2024-01-31T05:04:38.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd405800640 1 -- 172.21.15.97:0/3087569301 reap_dead start 2024-01-31T05:04:38.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 shutdown_connections 2024-01-31T05:04:38.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 >> 172.21.15.97:0/3087569301 conn(0x7fd4000d3060 msgr2=0x7fd40015c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:38.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 shutdown_connections 2024-01-31T05:04:38.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:38.819+0000 7fd406200640 1 -- 172.21.15.97:0/3087569301 wait complete. 2024-01-31T05:04:38.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-31T05:04:39.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.043+0000 7f2a63400640 1 Processor -- start 2024-01-31T05:04:39.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.043+0000 7f2a63400640 1 -- start start 2024-01-31T05:04:39.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.044+0000 7f2a63400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a5c0b1d10 0x7f2a5c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.044+0000 7f2a63400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.044+0000 7f2a63400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c15dc50 0x7f2a5c15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.044+0000 7f2a63400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b2630 con 0x7f2a5c15dc50 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.044+0000 7f2a63400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c06c8b0 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.044+0000 7f2a63400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15b120 con 0x7f2a5c15a600 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c15dc50 0x7f2a5c15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2a5c15dc50 0x7f2a5c15e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59796/0 (socket says 172.21.15.97:59796) 2024-01-31T05:04:39.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 -- 172.21.15.97:0/111491701 learned_addr learned my addr 172.21.15.97:0/111491701 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a53c00640 1 --2- 172.21.15.97:0/111491701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a60a00640 1 --2- 172.21.15.97:0/111491701 >> v2:172.21.15.188:3300/0 conn(0x7f2a5c0b1d10 0x7f2a5c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 -- 172.21.15.97:0/111491701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 msgr2=0x7f2a5c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 --2- 172.21.15.97:0/111491701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 -- 172.21.15.97:0/111491701 >> v2:172.21.15.188:3300/0 conn(0x7f2a5c0b1d10 msgr2=0x7f2a5c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 --2- 172.21.15.97:0/111491701 >> v2:172.21.15.188:3300/0 conn(0x7f2a5c0b1d10 0x7f2a5c0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.045+0000 7f2a61400640 1 -- 172.21.15.97:0/111491701 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c15b400 con 0x7f2a5c15dc50 2024-01-31T05:04:39.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.046+0000 7f2a61400640 1 --2- 172.21.15.97:0/111491701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c15e0e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c072e40 tx=0x7f2a5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec791e4255a9860 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a5be00640 1 -- 172.21.15.97:0/111491701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54031870 con 0x7f2a5c15dc50 2024-01-31T05:04:39.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a5be00640 1 -- 172.21.15.97:0/111491701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a540319d0 con 0x7f2a5c15dc50 2024-01-31T05:04:39.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a5be00640 1 -- 172.21.15.97:0/111491701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54031d60 con 0x7f2a5c15dc50 2024-01-31T05:04:39.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a63400640 1 -- 172.21.15.97:0/111491701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15dc50 msgr2=0x7f2a5c15e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a63400640 1 --2- 172.21.15.97:0/111491701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c15e0e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c072e40 tx=0x7f2a5402b910 comp rx=0 tx=0).stop 2024-01-31T05:04:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a63400640 1 -- 172.21.15.97:0/111491701 shutdown_connections 2024-01-31T05:04:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a63400640 1 --2- 172.21.15.97:0/111491701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c15e0e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a63400640 1 --2- 172.21.15.97:0/111491701 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.047+0000 7f2a63400640 1 --2- 172.21.15.97:0/111491701 >> v2:172.21.15.188:3300/0 conn(0x7f2a5c0b1d10 0x7f2a5c0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.048+0000 7f2a63400640 1 -- 172.21.15.97:0/111491701 >> 172.21.15.97:0/111491701 conn(0x7f2a5c0b0a10 msgr2=0x7f2a5c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:39.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.048+0000 7f2a63400640 1 -- 172.21.15.97:0/111491701 shutdown_connections 2024-01-31T05:04:39.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.048+0000 7f2a63400640 1 -- 172.21.15.97:0/111491701 wait complete. 2024-01-31T05:04:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.049+0000 7f2a63400640 1 Processor -- start 2024-01-31T05:04:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.049+0000 7f2a63400640 1 -- start start 2024-01-31T05:04:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.050+0000 7f2a63400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 0x7f2a5c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.050+0000 7f2a63400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c09b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.050+0000 7f2a63400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.050+0000 7f2a63400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c154780 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.050+0000 7f2a63400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15d6b0 con 0x7f2a5c15dc50 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.050+0000 7f2a63400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c09ba50 con 0x7f2a5c15a600 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 0x7f2a5c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 0x7f2a5c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59798/0 (socket says 172.21.15.97:59798) 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 -- 172.21.15.97:0/2595193651 learned_addr learned my addr 172.21.15.97:0/2595193651 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:39.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a61400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a53c00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c09b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 -- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 msgr2=0x7f2a5c09b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c09b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 -- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a5c15dc50 msgr2=0x7f2a5c070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 -- 172.21.15.97:0/2595193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540094d0 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a53c00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c09b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:39.055 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.051+0000 7f2a60a00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 0x7f2a5c154240 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c007e60 tx=0x7f2a4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.052+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c014620 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.052+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00ee90 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.052+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0149c0 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.052+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c0adb80 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.052+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c09bf90 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.053+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c0677c0 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.054+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f2a4c014b20 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.055+0000 7f2a59600640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3406aed0 0x7f2a3406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.055+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2a4c089db0 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.055+0000 7f2a53c00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3406aed0 0x7f2a3406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.056+0000 7f2a53c00640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3406aed0 0x7f2a3406d390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0ced30 tx=0x7f2a44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.059+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f2a4c054e40 con 0x7f2a5c0b1d10 2024-01-31T05:04:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.253+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a5c0ada00 con 0x7f2a3406aed0 2024-01-31T05:04:39.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.254+0000 7f2a59600640 1 -- 172.21.15.97:0/2595193651 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2a44009e10 con 0x7f2a3406aed0 2024-01-31T05:04:39.254 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2024-01-31T05:04:39.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3406aed0 msgr2=0x7f2a3406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3406aed0 0x7f2a3406d390 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0ced30 tx=0x7f2a44008040 comp rx=0 tx=0).stop 2024-01-31T05:04:39.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 msgr2=0x7f2a5c154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 0x7f2a5c154240 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c007e60 tx=0x7f2a4c00e4b0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 shutdown_connections 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a5c15dc50 0x7f2a5c070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a5c15a600 0x7f2a5c09b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f2a3406aed0 0x7f2a3406d390 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 --2- 172.21.15.97:0/2595193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a5c0b1d10 0x7f2a5c154240 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 >> 172.21.15.97:0/2595193651 conn(0x7f2a5c0b0a10 msgr2=0x7f2a5c157950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 shutdown_connections 2024-01-31T05:04:39.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.256+0000 7f2a63400640 1 -- 172.21.15.97:0/2595193651 wait complete. 2024-01-31T05:04:39.275 INFO:tasks.ceph:Scrubbing osd.3 2024-01-31T05:04:39.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.491+0000 7f5a07000640 1 Processor -- start 2024-01-31T05:04:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.492+0000 7f5a07000640 1 -- start start 2024-01-31T05:04:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a07000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a0015ed30 0x7f5a000721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a07000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0015f110 0x7f5a00070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a07000640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5a000c9280 0x7f5a000c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a07000640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a00072730 con 0x7f5a000c9280 2024-01-31T05:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a07000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0006e730 con 0x7f5a0015ed30 2024-01-31T05:04:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a07000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a00155c10 con 0x7f5a0015f110 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.493+0000 7f5a06600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5a000c9280 0x7f5a000c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5a000c9280 0x7f5a000c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59800/0 (socket says 172.21.15.97:59800) 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 -- 172.21.15.97:0/2393443127 learned_addr learned my addr 172.21.15.97:0/2393443127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 -- 172.21.15.97:0/2393443127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0015f110 msgr2=0x7f5a00070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 --2- 172.21.15.97:0/2393443127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0015f110 0x7f5a00070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 -- 172.21.15.97:0/2393443127 >> v2:172.21.15.188:3300/0 conn(0x7f5a0015ed30 msgr2=0x7f5a000721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a05c00640 1 --2- 172.21.15.97:0/2393443127 >> v2:172.21.15.188:3300/0 conn(0x7f5a0015ed30 0x7f5a000721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 --2- 172.21.15.97:0/2393443127 >> v2:172.21.15.188:3300/0 conn(0x7f5a0015ed30 0x7f5a000721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.494+0000 7f5a06600640 1 -- 172.21.15.97:0/2393443127 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0006e450 con 0x7f5a000c9280 2024-01-31T05:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.495+0000 7f5a06600640 1 --2- 172.21.15.97:0/2393443127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a000c9280 0x7f5a000c9710 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5a00067390 tx=0x7f59f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372c8d05bf0b91a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.495+0000 7f59ffe00640 1 -- 172.21.15.97:0/2393443127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f802c9e0 con 0x7f5a000c9280 2024-01-31T05:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.495+0000 7f59ffe00640 1 -- 172.21.15.97:0/2393443127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f802cb40 con 0x7f5a000c9280 2024-01-31T05:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.495+0000 7f59ffe00640 1 -- 172.21.15.97:0/2393443127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80343f0 con 0x7f5a000c9280 2024-01-31T05:04:39.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 -- 172.21.15.97:0/2393443127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a000c9280 msgr2=0x7f5a000c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 --2- 172.21.15.97:0/2393443127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a000c9280 0x7f5a000c9710 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5a00067390 tx=0x7f59f802be40 comp rx=0 tx=0).stop 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 -- 172.21.15.97:0/2393443127 shutdown_connections 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 --2- 172.21.15.97:0/2393443127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a000c9280 0x7f5a000c9710 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 --2- 172.21.15.97:0/2393443127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a0015f110 0x7f5a00070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 --2- 172.21.15.97:0/2393443127 >> v2:172.21.15.188:3300/0 conn(0x7f5a0015ed30 0x7f5a000721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 -- 172.21.15.97:0/2393443127 >> 172.21.15.97:0/2393443127 conn(0x7f5a000cec30 msgr2=0x7f5a000cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 -- 172.21.15.97:0/2393443127 shutdown_connections 2024-01-31T05:04:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.496+0000 7f5a07000640 1 -- 172.21.15.97:0/2393443127 wait complete. 2024-01-31T05:04:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.498+0000 7f5a07000640 1 Processor -- start 2024-01-31T05:04:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.498+0000 7f5a07000640 1 -- start start 2024-01-31T05:04:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a07000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a000c9280 0x7f5a0014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a07000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0015ed30 0x7f5a00156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a07000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0015f110 0x7f5a0014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a07000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0014ece0 con 0x7f5a0015f110 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a07000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a00158750 con 0x7f5a0015ed30 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a07000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a00158a30 con 0x7f5a000c9280 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a06600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0015f110 0x7f5a0014e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a06600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0015f110 0x7f5a0014e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59804/0 (socket says 172.21.15.97:59804) 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a06600640 1 -- 172.21.15.97:0/4006145492 learned_addr learned my addr 172.21.15.97:0/4006145492 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:39.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.499+0000 7f5a05c00640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a000c9280 0x7f5a0014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a05200640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0015ed30 0x7f5a00156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a06600640 1 -- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a000c9280 msgr2=0x7f5a0014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a06600640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a000c9280 0x7f5a0014eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a06600640 1 -- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0015ed30 msgr2=0x7f5a00156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a06600640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a0015ed30 0x7f5a00156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a06600640 1 -- 172.21.15.97:0/4006145492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009b30 con 0x7f5a0015f110 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a05c00640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a000c9280 0x7f5a0014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:39.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a06600640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0015f110 0x7f5a0014e7a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f59f8036040 tx=0x7f59f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f803e450 con 0x7f5a0015f110 2024-01-31T05:04:39.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f803e5d0 con 0x7f5a0015f110 2024-01-31T05:04:39.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f803e980 con 0x7f5a0015f110 2024-01-31T05:04:39.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a00155f80 con 0x7f5a0015f110 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.500+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a00157f00 con 0x7f5a0015f110 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.501+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a000ca450 con 0x7f5a0015f110 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.502+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f59f803eae0 con 0x7f5a0015f110 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.503+0000 7f59fd600640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f59d806aed0 0x7f59d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.503+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f59f80ac310 con 0x7f5a0015f110 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.503+0000 7f5a05c00640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f59d806aed0 0x7f59d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.503+0000 7f59fd600640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7f59d80711d0 0x7f59d8073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.503+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 --> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59d8073ce0 con 0x7f59d80711d0 2024-01-31T05:04:39.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.504+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59f80ae9a0 con 0x7f5a0015f110 2024-01-31T05:04:39.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.504+0000 7f5a05200640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7f59d80711d0 0x7f59d8073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.504+0000 7f5a05c00640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f59d806aed0 0x7f59d806d390 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5a000ce9e0 tx=0x7f59f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.504+0000 7f5a05200640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7f59d80711d0 0x7f59d8073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.505+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== osd.3 v2:172.21.15.97:6821/4184396147 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f59d8073ce0 con 0x7f59d80711d0 2024-01-31T05:04:39.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.520+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 --> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a000c4d40 con 0x7f59d80711d0 2024-01-31T05:04:39.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.520+0000 7f59fd600640 1 -- 172.21.15.97:0/4006145492 <== osd.3 v2:172.21.15.97:6821/4184396147 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5a000c4d40 con 0x7f59d80711d0 2024-01-31T05:04:39.520 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:39.520 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:39.520 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7f59d80711d0 msgr2=0x7f59d8073610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6821/4184396147,v1:172.21.15.97:6824/4184396147] conn(0x7f59d80711d0 0x7f59d8073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f59d806aed0 msgr2=0x7f59d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f59d806aed0 0x7f59d806d390 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5a000ce9e0 tx=0x7f59f0008040 comp rx=0 tx=0).stop 2024-01-31T05:04:39.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0015f110 msgr2=0x7f5a0014e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 --2- 172.21.15.97:0/4006145492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a0015f110 0x7f5a0014e7a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f59f8036040 tx=0x7f59f8004a40 comp rx=0 tx=0).stop 2024-01-31T05:04:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a06600640 1 -- 172.21.15.97:0/4006145492 reap_dead start 2024-01-31T05:04:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 shutdown_connections 2024-01-31T05:04:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 >> 172.21.15.97:0/4006145492 conn(0x7f5a000cec30 msgr2=0x7f5a000c9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 shutdown_connections 2024-01-31T05:04:39.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.521+0000 7f5a07000640 1 -- 172.21.15.97:0/4006145492 wait complete. 2024-01-31T05:04:39.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-31T05:04:39.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.743+0000 7fd431400640 1 Processor -- start 2024-01-31T05:04:39.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.744+0000 7fd431400640 1 -- start start 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd431400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd42c0b1d10 0x7fd42c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd431400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd42c15a600 0x7fd42c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd431400640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42c0b2630 con 0x7fd42c15dc50 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd431400640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42c06c8b0 con 0x7fd42c0b1d10 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd431400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42c15b120 con 0x7fd42c15a600 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd42b200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd42c15dc50 0x7fd42c15e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47566/0 (socket says 172.21.15.97:47566) 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd42b200640 1 -- 172.21.15.97:0/404248295 learned_addr learned my addr 172.21.15.97:0/404248295 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.745+0000 7fd42a800640 1 --2- 172.21.15.97:0/404248295 >> v2:172.21.15.97:3300/0 conn(0x7fd42c0b1d10 0x7fd42c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd429e00640 1 --2- 172.21.15.97:0/404248295 >> v2:172.21.15.188:3300/0 conn(0x7fd42c15a600 0x7fd42c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd429400640 1 -- 172.21.15.97:0/404248295 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930657272 0 0) 0x7fd42c0b2630 con 0x7fd42c15dc50 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd429400640 1 -- 172.21.15.97:0/404248295 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd40c003520 con 0x7fd42c15dc50 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd42a800640 1 -- 172.21.15.97:0/404248295 >> v2:172.21.15.188:3300/0 conn(0x7fd42c15a600 msgr2=0x7fd42c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd42a800640 1 --2- 172.21.15.97:0/404248295 >> v2:172.21.15.188:3300/0 conn(0x7fd42c15a600 0x7fd42c15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd42a800640 1 -- 172.21.15.97:0/404248295 >> v1:172.21.15.97:6789/0 conn(0x7fd42c15dc50 legacy=0x7fd42c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.746+0000 7fd42a800640 1 -- 172.21.15.97:0/404248295 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c15b400 con 0x7fd42c0b1d10 2024-01-31T05:04:39.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.747+0000 7fd42a800640 1 --2- 172.21.15.97:0/404248295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c0b1d10 0x7fd42c0b20f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd42c158250 tx=0x7fd41402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e3409de5c2faf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.747+0000 7fd429400640 1 -- 172.21.15.97:0/404248295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414031830 con 0x7fd42c0b1d10 2024-01-31T05:04:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.747+0000 7fd429400640 1 -- 172.21.15.97:0/404248295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414031990 con 0x7fd42c0b1d10 2024-01-31T05:04:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.747+0000 7fd429400640 1 -- 172.21.15.97:0/404248295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414031d20 con 0x7fd42c0b1d10 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 -- 172.21.15.97:0/404248295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c0b1d10 msgr2=0x7fd42c0b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 --2- 172.21.15.97:0/404248295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c0b1d10 0x7fd42c0b20f0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fd42c158250 tx=0x7fd41402b8d0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 -- 172.21.15.97:0/404248295 shutdown_connections 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 --2- 172.21.15.97:0/404248295 >> v2:172.21.15.188:3300/0 conn(0x7fd42c15a600 0x7fd42c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 --2- 172.21.15.97:0/404248295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c0b1d10 0x7fd42c0b20f0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 -- 172.21.15.97:0/404248295 >> 172.21.15.97:0/404248295 conn(0x7fd42c0b0a10 msgr2=0x7fd42c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 -- 172.21.15.97:0/404248295 shutdown_connections 2024-01-31T05:04:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.748+0000 7fd431400640 1 -- 172.21.15.97:0/404248295 wait complete. 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.750+0000 7fd431400640 1 Processor -- start 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.750+0000 7fd431400640 1 -- start start 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd431400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42c0b1d10 0x7fd42c1f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd431400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd42c15a600 0x7fd42c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd431400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 0x7fd42c09a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd431400640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c09b3b0 con 0x7fd42c15dc50 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd431400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c15bd10 con 0x7fd42c0b1d10 2024-01-31T05:04:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd431400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c15c570 con 0x7fd42c15a600 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd42b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 0x7fd42c09a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd42b200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 0x7fd42c09a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59826/0 (socket says 172.21.15.97:59826) 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd42b200640 1 -- 172.21.15.97:0/3557598522 learned_addr learned my addr 172.21.15.97:0/3557598522 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.751+0000 7fd42a800640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42c0b1d10 0x7fd42c1f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd429e00640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd42c15a600 0x7fd42c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd42b200640 1 -- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd42c15a600 msgr2=0x7fd42c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd42b200640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd42c15a600 0x7fd42c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd42b200640 1 -- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42c0b1d10 msgr2=0x7fd42c1f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd42b200640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42c0b1d10 0x7fd42c1f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd42b200640 1 -- 172.21.15.97:0/3557598522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009400 con 0x7fd42c15dc50 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd429e00640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd42c15a600 0x7fd42c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd42b200640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 0x7fd42c09a570 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c390 tx=0x7fd42400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424005ce0 con 0x7fd42c15dc50 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd424005e40 con 0x7fd42c15dc50 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c0adb80 con 0x7fd42c15dc50 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424007310 con 0x7fd42c15dc50 2024-01-31T05:04:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.752+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c1f3b40 con 0x7fd42c15dc50 2024-01-31T05:04:39.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.754+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c0677c0 con 0x7fd42c15dc50 2024-01-31T05:04:39.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.754+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fd424007470 con 0x7fd42c15dc50 2024-01-31T05:04:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.755+0000 7fd422a00640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd40406aed0 0x7fd40406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.755+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd424088ad0 con 0x7fd42c15dc50 2024-01-31T05:04:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.756+0000 7fd42a800640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd40406aed0 0x7fd40406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:39.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.756+0000 7fd42a800640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd40406aed0 0x7fd40406d390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd414036000 tx=0x7fd41403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:39.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.760+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fd424053b60 con 0x7fd42c15dc50 2024-01-31T05:04:39.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.954+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd42c0cdfa0 con 0x7fd40406aed0 2024-01-31T05:04:39.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.955+0000 7fd422a00640 1 -- 172.21.15.97:0/3557598522 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd414031b00 con 0x7fd40406aed0 2024-01-31T05:04:39.955 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd40406aed0 msgr2=0x7fd40406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd40406aed0 0x7fd40406d390 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd414036000 tx=0x7fd41403c040 comp rx=0 tx=0).stop 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 msgr2=0x7fd42c09a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 0x7fd42c09a570 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c390 tx=0x7fd42400c850 comp rx=0 tx=0).stop 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 shutdown_connections 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fd40406aed0 0x7fd40406d390 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd42c15dc50 0x7fd42c09a570 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd42c15a600 0x7fd42c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 --2- 172.21.15.97:0/3557598522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd42c0b1d10 0x7fd42c1f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 >> 172.21.15.97:0/3557598522 conn(0x7fd42c0b0a10 msgr2=0x7fd42c0ace30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 shutdown_connections 2024-01-31T05:04:39.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:39.958+0000 7fd431400640 1 -- 172.21.15.97:0/3557598522 wait complete. 2024-01-31T05:04:39.980 INFO:tasks.ceph:Scrubbing osd.4 2024-01-31T05:04:39.980 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.195+0000 7f41c1800640 1 Processor -- start 2024-01-31T05:04:40.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.195+0000 7f41c1800640 1 -- start start 2024-01-31T05:04:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.196+0000 7f41c1800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41bc0999c0 0x7f41bc099da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.196+0000 7f41c1800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0b36b0 0x7f41bc09a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.196+0000 7f41c1800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc0d0040 con 0x7f41bc0cec70 2024-01-31T05:04:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.196+0000 7f41c1800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc0cd090 con 0x7f41bc0999c0 2024-01-31T05:04:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.196+0000 7f41c1800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc06e900 con 0x7f41bc0b36b0 2024-01-31T05:04:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41baa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f41bc0999c0 0x7f41bc099da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41bb400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f41bc0cec70 0x7f41bc0cf0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47576/0 (socket says 172.21.15.97:47576) 2024-01-31T05:04:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41bb400640 1 -- 172.21.15.97:0/135814601 learned_addr learned my addr 172.21.15.97:0/135814601 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41b1e00640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0b36b0 0x7f41bc09a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41baa00640 1 -- 172.21.15.97:0/135814601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0b36b0 msgr2=0x7f41bc09a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41baa00640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0b36b0 0x7f41bc09a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41baa00640 1 -- 172.21.15.97:0/135814601 >> v1:172.21.15.97:6789/0 conn(0x7f41bc0cec70 legacy=0x7f41bc0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41baa00640 1 -- 172.21.15.97:0/135814601 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0cc7f0 con 0x7f41bc0999c0 2024-01-31T05:04:40.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41ba000640 1 -- 172.21.15.97:0/135814601 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857916984 0 0) 0x7f41b4001110 con 0x7f41bc0cec70 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.197+0000 7f41b1e00640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0b36b0 0x7f41bc09a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.198+0000 7f41baa00640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0999c0 0x7f41bc099da0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad07a1d0ec1d931 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41ba000640 1 -- 172.21.15.97:0/135814601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac02cbc0 con 0x7f41bc0999c0 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41ba000640 1 -- 172.21.15.97:0/135814601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac02cd20 con 0x7f41bc0999c0 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41ba000640 1 -- 172.21.15.97:0/135814601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac034660 con 0x7f41bc0999c0 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 -- 172.21.15.97:0/135814601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0999c0 msgr2=0x7f41bc099da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0999c0 0x7f41bc099da0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02be10 comp rx=0 tx=0).stop 2024-01-31T05:04:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 -- 172.21.15.97:0/135814601 shutdown_connections 2024-01-31T05:04:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0b36b0 0x7f41bc09a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 --2- 172.21.15.97:0/135814601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0999c0 0x7f41bc099da0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 -- 172.21.15.97:0/135814601 >> 172.21.15.97:0/135814601 conn(0x7f41bc0b2b90 msgr2=0x7f41bc0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 -- 172.21.15.97:0/135814601 shutdown_connections 2024-01-31T05:04:40.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.199+0000 7f41c1800640 1 -- 172.21.15.97:0/135814601 wait complete. 2024-01-31T05:04:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.201+0000 7f41c1800640 1 Processor -- start 2024-01-31T05:04:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.201+0000 7f41c1800640 1 -- start start 2024-01-31T05:04:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.202+0000 7f41c1800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0999c0 0x7f41bc0b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.202+0000 7f41c1800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0b36b0 0x7f41bc0b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.202+0000 7f41c1800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0cec70 0x7f41bc0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.202+0000 7f41c1800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc14f9e0 con 0x7f41bc0b36b0 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.202+0000 7f41c1800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc15c7a0 con 0x7f41bc0999c0 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.202+0000 7f41c1800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc15d600 con 0x7f41bc0cec70 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0b36b0 0x7f41bc0b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0b36b0 0x7f41bc0b6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59834/0 (socket says 172.21.15.97:59834) 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 -- 172.21.15.97:0/2320525553 learned_addr learned my addr 172.21.15.97:0/2320525553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41baa00640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0999c0 0x7f41bc0b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41bb400640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0cec70 0x7f41bc0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 -- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0cec70 msgr2=0x7f41bc0b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0cec70 0x7f41bc0b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 -- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0999c0 msgr2=0x7f41bc0b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0999c0 0x7f41bc0b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41b1e00640 1 -- 172.21.15.97:0/2320525553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac009a70 con 0x7f41bc0b36b0 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.203+0000 7f41bb400640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc0cec70 0x7f41bc0b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.204+0000 7f41b1e00640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0b36b0 0x7f41bc0b6bc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c820 tx=0x7f41a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.204+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4004230 con 0x7f41bc0b36b0 2024-01-31T05:04:40.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.204+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a4004390 con 0x7f41bc0b36b0 2024-01-31T05:04:40.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.204+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4007a70 con 0x7f41bc0b36b0 2024-01-31T05:04:40.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.204+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc15c9f0 con 0x7f41bc0b36b0 2024-01-31T05:04:40.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.204+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc15cfb0 con 0x7f41bc0b36b0 2024-01-31T05:04:40.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.205+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41bc0ccdb0 con 0x7f41bc0b36b0 2024-01-31T05:04:40.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.206+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f41a4007bd0 con 0x7f41bc0b36b0 2024-01-31T05:04:40.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.207+0000 7f41b3400640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f419406af00 0x7f419406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.207+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f41a4089320 con 0x7f41bc0b36b0 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.207+0000 7f41b3400640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f4194071200 0x7f4194073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.207+0000 7f41baa00640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f419406af00 0x7f419406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.207+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 --> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4194073d10 con 0x7f4194071200 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.207+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a408b9e0 con 0x7f41bc0b36b0 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.208+0000 7f41bb400640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f4194071200 0x7f4194073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.208+0000 7f41baa00640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f419406af00 0x7f419406d3c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f41ac036000 tx=0x7f41ac030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.209+0000 7f41bb400640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f4194071200 0x7f4194073640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.211+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== osd.4 v2:172.21.15.188:6824/2903566089 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4194073d10 con 0x7f4194071200 2024-01-31T05:04:40.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.225+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 --> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f41bc067830 con 0x7f4194071200 2024-01-31T05:04:40.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41b3400640 1 -- 172.21.15.97:0/2320525553 <== osd.4 v2:172.21.15.188:6824/2903566089 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f41bc067830 con 0x7f4194071200 2024-01-31T05:04:40.226 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:40.226 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:40.226 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:40.227 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f4194071200 msgr2=0x7f4194073640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41c1800640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.188:6824/2903566089,v1:172.21.15.188:6825/2903566089] conn(0x7f4194071200 0x7f4194073640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f419406af00 msgr2=0x7f419406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41c1800640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f419406af00 0x7f419406d3c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f41ac036000 tx=0x7f41ac030000 comp rx=0 tx=0).stop 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0b36b0 msgr2=0x7f41bc0b6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41c1800640 1 --2- 172.21.15.97:0/2320525553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41bc0b36b0 0x7f41bc0b6bc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f41a400c820 tx=0x7f41a400cce0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.227+0000 7f41bb400640 1 -- 172.21.15.97:0/2320525553 reap_dead start 2024-01-31T05:04:40.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.228+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 shutdown_connections 2024-01-31T05:04:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.228+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 >> 172.21.15.97:0/2320525553 conn(0x7f41bc0b2b90 msgr2=0x7f41bc0cbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.228+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 shutdown_connections 2024-01-31T05:04:40.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.228+0000 7f41c1800640 1 -- 172.21.15.97:0/2320525553 wait complete. 2024-01-31T05:04:40.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-31T05:04:40.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.450+0000 7fb0eaa00640 1 Processor -- start 2024-01-31T05:04:40.451 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.451+0000 7fb0eaa00640 1 -- start start 2024-01-31T05:04:40.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.452+0000 7fb0eaa00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0e415eb70 0x7fb0e415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.452+0000 7fb0eaa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e40ca940 0x7fb0e40cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.452+0000 7fb0eaa00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e40ca050 con 0x7fb0e40ca4e0 2024-01-31T05:04:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.452+0000 7fb0eaa00640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e415f490 con 0x7fb0e415eb70 2024-01-31T05:04:40.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.452+0000 7fb0eaa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40c9440 con 0x7fb0e40ca940 2024-01-31T05:04:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e9600640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0e415eb70 0x7fb0e415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e8c00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb0e40ca4e0 0x7fb0e4070400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47586/0 (socket says 172.21.15.97:47586) 2024-01-31T05:04:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e8c00640 1 -- 172.21.15.97:0/2826822571 learned_addr learned my addr 172.21.15.97:0/2826822571 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:40.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0ea000640 1 --2- 172.21.15.97:0/2826822571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e40ca940 0x7fb0e40cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e9600640 1 -- 172.21.15.97:0/2826822571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e40ca940 msgr2=0x7fb0e40cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e9600640 1 --2- 172.21.15.97:0/2826822571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e40ca940 0x7fb0e40cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e9600640 1 -- 172.21.15.97:0/2826822571 >> v1:172.21.15.97:6789/0 conn(0x7fb0e40ca4e0 legacy=0x7fb0e4070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e3e00640 1 -- 172.21.15.97:0/2826822571 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521803277 0 0) 0x7fb0e40ca050 con 0x7fb0e40ca4e0 2024-01-31T05:04:40.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.453+0000 7fb0e9600640 1 -- 172.21.15.97:0/2826822571 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40c9720 con 0x7fb0e415eb70 2024-01-31T05:04:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.455+0000 7fb0e9600640 1 --2- 172.21.15.97:0/2826822571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e415eb70 0x7fb0e415ef50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40097a0 tx=0x7fb0d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d534f45fba60b8e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.455+0000 7fb0e3e00640 1 -- 172.21.15.97:0/2826822571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d402d040 con 0x7fb0e415eb70 2024-01-31T05:04:40.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.455+0000 7fb0e3e00640 1 -- 172.21.15.97:0/2826822571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d4031810 con 0x7fb0e415eb70 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.455+0000 7fb0e3e00640 1 -- 172.21.15.97:0/2826822571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4031ba0 con 0x7fb0e415eb70 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.455+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2826822571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e415eb70 msgr2=0x7fb0e415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2826822571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e415eb70 0x7fb0e415ef50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40097a0 tx=0x7fb0d402b8b0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2826822571 shutdown_connections 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2826822571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e40ca940 0x7fb0e40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2826822571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e415eb70 0x7fb0e415ef50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2826822571 >> 172.21.15.97:0/2826822571 conn(0x7fb0e40cebd0 msgr2=0x7fb0e40cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:40.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2826822571 shutdown_connections 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.456+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2826822571 wait complete. 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.458+0000 7fb0eaa00640 1 Processor -- start 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.458+0000 7fb0eaa00640 1 -- start start 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0eaa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40ca4e0 0x7fb0e4133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0eaa00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 0x7fb0e413f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0eaa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e415eb70 0x7fb0e40ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0eaa00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40b4c60 con 0x7fb0e40ca940 2024-01-31T05:04:40.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0eaa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e413fd00 con 0x7fb0e40ca4e0 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0eaa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e4134380 con 0x7fb0e415eb70 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 0x7fb0e413f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 0x7fb0e413f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59842/0 (socket says 172.21.15.97:59842) 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 -- 172.21.15.97:0/2752711594 learned_addr learned my addr 172.21.15.97:0/2752711594 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0ea000640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e415eb70 0x7fb0e40ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e9600640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40ca4e0 0x7fb0e4133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 -- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e415eb70 msgr2=0x7fb0e40ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e415eb70 0x7fb0e40ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 -- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40ca4e0 msgr2=0x7fb0e4133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40ca4e0 0x7fb0e4133e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.459+0000 7fb0e8c00640 1 -- 172.21.15.97:0/2752711594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4009400 con 0x7fb0e40ca940 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.460+0000 7fb0e8c00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 0x7fb0e413f230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c4d0 tx=0x7fb0cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.460+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0074e0 con 0x7fb0e40ca940 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.460+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc007660 con 0x7fb0e40ca940 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.460+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc004a20 con 0x7fb0e40ca940 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.460+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e40b3bc0 con 0x7fb0e40ca940 2024-01-31T05:04:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.460+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e40b4180 con 0x7fb0e40ca940 2024-01-31T05:04:40.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.461+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e40715d0 con 0x7fb0e40ca940 2024-01-31T05:04:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.462+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb0cc004b80 con 0x7fb0e40ca940 2024-01-31T05:04:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.463+0000 7fb0e1600640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0bc06aed0 0x7fb0bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.463+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb0cc088b10 con 0x7fb0e40ca940 2024-01-31T05:04:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.463+0000 7fb0e9600640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0bc06aed0 0x7fb0bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.464+0000 7fb0e9600640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0bc06aed0 0x7fb0bc06d390 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4036000 tx=0x7fb0d403c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.467+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb0cc053a70 con 0x7fb0e40ca940 2024-01-31T05:04:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.661+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e40c9ce0 con 0x7fb0bc06aed0 2024-01-31T05:04:40.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.662+0000 7fb0e1600640 1 -- 172.21.15.97:0/2752711594 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb0d4031b00 con 0x7fb0bc06aed0 2024-01-31T05:04:40.662 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2024-01-31T05:04:40.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.664+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0bc06aed0 msgr2=0x7fb0bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.664+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0bc06aed0 0x7fb0bc06d390 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4036000 tx=0x7fb0d403c040 comp rx=0 tx=0).stop 2024-01-31T05:04:40.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 msgr2=0x7fb0e413f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 0x7fb0e413f230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c4d0 tx=0x7fb0cc00c990 comp rx=0 tx=0).stop 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 shutdown_connections 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0e415eb70 0x7fb0e40ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb0bc06aed0 0x7fb0bc06d390 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e40ca940 0x7fb0e413f230 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 --2- 172.21.15.97:0/2752711594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40ca4e0 0x7fb0e4133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 >> 172.21.15.97:0/2752711594 conn(0x7fb0e40cebd0 msgr2=0x7fb0e40ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 shutdown_connections 2024-01-31T05:04:40.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.665+0000 7fb0eaa00640 1 -- 172.21.15.97:0/2752711594 wait complete. 2024-01-31T05:04:40.683 INFO:tasks.ceph:Scrubbing osd.5 2024-01-31T05:04:40.683 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.897+0000 7f4c0d200640 1 Processor -- start 2024-01-31T05:04:40.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.898+0000 7f4c0d200640 1 -- start start 2024-01-31T05:04:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.899+0000 7f4c0d200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4c080d1e60 0x7f4c080cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.899+0000 7f4c0d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c080d3850 0x7f4c080d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.899+0000 7f4c0d200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080d0780 con 0x7f4c0815d3f0 2024-01-31T05:04:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.899+0000 7f4c0d200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080ce7e0 con 0x7f4c080d1e60 2024-01-31T05:04:40.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.899+0000 7f4c0d200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080d3c80 con 0x7f4c080d3850 2024-01-31T05:04:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4c07400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4c0815d3f0 0x7f4c080d1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47598/0 (socket says 172.21.15.97:47598) 2024-01-31T05:04:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4c07400640 1 -- 172.21.15.97:0/2513056860 learned_addr learned my addr 172.21.15.97:0/2513056860 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:40.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4bfe800640 1 --2- 172.21.15.97:0/2513056860 >> v2:172.21.15.97:3300/0 conn(0x7f4c080d1e60 0x7f4c080cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4c06a00640 1 -- 172.21.15.97:0/2513056860 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358580646 0 0) 0x7f4c080d0780 con 0x7f4c0815d3f0 2024-01-31T05:04:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4c06a00640 1 -- 172.21.15.97:0/2513056860 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be8003520 con 0x7f4c0815d3f0 2024-01-31T05:04:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4c07e00640 1 --2- 172.21.15.97:0/2513056860 >> v2:172.21.15.188:3300/0 conn(0x7f4c080d3850 0x7f4c080d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.901 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4bfe800640 1 -- 172.21.15.97:0/2513056860 >> v2:172.21.15.188:3300/0 conn(0x7f4c080d3850 msgr2=0x7f4c080d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4bfe800640 1 --2- 172.21.15.97:0/2513056860 >> v2:172.21.15.188:3300/0 conn(0x7f4c080d3850 0x7f4c080d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4bfe800640 1 -- 172.21.15.97:0/2513056860 >> v1:172.21.15.97:6789/0 conn(0x7f4c0815d3f0 legacy=0x7f4c080d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4bfe800640 1 -- 172.21.15.97:0/2513056860 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080d09b0 con 0x7f4c080d1e60 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.900+0000 7f4c07e00640 1 --2- 172.21.15.97:0/2513056860 >> v2:172.21.15.188:3300/0 conn(0x7f4c080d3850 0x7f4c080d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.901+0000 7f4bfe800640 1 --2- 172.21.15.97:0/2513056860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d1e60 0x7f4c080cec70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4c080cd770 tx=0x7f4bf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e0736fbbf6d379 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c06a00640 1 -- 172.21.15.97:0/2513056860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0031830 con 0x7f4c080d1e60 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c06a00640 1 -- 172.21.15.97:0/2513056860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0031990 con 0x7f4c080d1e60 2024-01-31T05:04:40.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c06a00640 1 -- 172.21.15.97:0/2513056860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0031d20 con 0x7f4c080d1e60 2024-01-31T05:04:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 -- 172.21.15.97:0/2513056860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d1e60 msgr2=0x7f4c080cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 --2- 172.21.15.97:0/2513056860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d1e60 0x7f4c080cec70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4c080cd770 tx=0x7f4bf002b8d0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 -- 172.21.15.97:0/2513056860 shutdown_connections 2024-01-31T05:04:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 --2- 172.21.15.97:0/2513056860 >> v2:172.21.15.188:3300/0 conn(0x7f4c080d3850 0x7f4c080d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 --2- 172.21.15.97:0/2513056860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d1e60 0x7f4c080cec70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 -- 172.21.15.97:0/2513056860 >> 172.21.15.97:0/2513056860 conn(0x7f4c0808cb00 msgr2=0x7f4c0809c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 -- 172.21.15.97:0/2513056860 shutdown_connections 2024-01-31T05:04:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.902+0000 7f4c0d200640 1 -- 172.21.15.97:0/2513056860 wait complete. 2024-01-31T05:04:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.904+0000 7f4c0d200640 1 Processor -- start 2024-01-31T05:04:40.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.904+0000 7f4c0d200640 1 -- start start 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c080d1e60 0x7f4c080c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d3850 0x7f4c0814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0815d3f0 0x7f4c080cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4c0d200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080c6290 con 0x7f4c080d3850 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4c0d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080cba60 con 0x7f4c080d1e60 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4c0d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080cbd40 con 0x7f4c0815d3f0 2024-01-31T05:04:40.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.905+0000 7f4bfe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d3850 0x7f4c0814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d3850 0x7f4c0814d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59858/0 (socket says 172.21.15.97:59858) 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 -- 172.21.15.97:0/1897361657 learned_addr learned my addr 172.21.15.97:0/1897361657 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 -- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0815d3f0 msgr2=0x7f4c080cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0815d3f0 0x7f4c080cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4c07400640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c080d1e60 0x7f4c080c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 -- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c080d1e60 msgr2=0x7f4c080c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c080d1e60 0x7f4c080c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 -- 172.21.15.97:0/1897361657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0009400 con 0x7f4c080d3850 2024-01-31T05:04:40.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bfe800640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d3850 0x7f4c0814d0c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.906+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf003a8f0 con 0x7f4c080d3850 2024-01-31T05:04:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.907+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf003aa70 con 0x7f4c080d3850 2024-01-31T05:04:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.907+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf003ae20 con 0x7f4c080d3850 2024-01-31T05:04:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.907+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c080cc020 con 0x7f4c080d3850 2024-01-31T05:04:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.907+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c080cb710 con 0x7f4c080d3850 2024-01-31T05:04:40.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.908+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c080ce470 con 0x7f4c080d3850 2024-01-31T05:04:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.909+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f4bf004d450 con 0x7f4c080d3850 2024-01-31T05:04:40.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.910+0000 7f4bffe00640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4be006aed0 0x7f4be006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.910+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4bf00ab540 con 0x7f4c080d3850 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.910+0000 7f4c07400640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4be006aed0 0x7f4be006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.910+0000 7f4bffe00640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7f4be00711d0 0x7f4be0073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.910+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 --> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4be0073ce0 con 0x7f4be00711d0 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.910+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bf00adb20 con 0x7f4c080d3850 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.911+0000 7f4c07e00640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7f4be00711d0 0x7f4be0073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.911+0000 7f4c07400640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4be006aed0 0x7f4be006d390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8000f60 tx=0x7f4bf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.911+0000 7f4c07e00640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7f4be00711d0 0x7f4be0073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:40.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.914+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== osd.5 v2:172.21.15.188:6809/576621421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4be0073ce0 con 0x7f4be00711d0 2024-01-31T05:04:40.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.928+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 --> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4c080ce190 con 0x7f4be00711d0 2024-01-31T05:04:40.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.929+0000 7f4bffe00640 1 -- 172.21.15.97:0/1897361657 <== osd.5 v2:172.21.15.188:6809/576621421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4c00008950 con 0x7f4be00711d0 2024-01-31T05:04:40.929 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:40.929 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:40.929 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7f4be00711d0 msgr2=0x7f4be0073610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.188:6809/576621421,v1:172.21.15.188:6811/576621421] conn(0x7f4be00711d0 0x7f4be0073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4be006aed0 msgr2=0x7f4be006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f4be006aed0 0x7f4be006d390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8000f60 tx=0x7f4bf8009040 comp rx=0 tx=0).stop 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d3850 msgr2=0x7f4c0814d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 --2- 172.21.15.97:0/1897361657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c080d3850 0x7f4c0814d0c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0000c00 tx=0x7f4bf00046c0 comp rx=0 tx=0).stop 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c07e00640 1 -- 172.21.15.97:0/1897361657 reap_dead start 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 shutdown_connections 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 >> 172.21.15.97:0/1897361657 conn(0x7f4c0808cb00 msgr2=0x7f4c08160fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 shutdown_connections 2024-01-31T05:04:40.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:40.930+0000 7f4c0d200640 1 -- 172.21.15.97:0/1897361657 wait complete. 2024-01-31T05:04:40.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-31T05:04:41.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.158+0000 7fa941800640 1 Processor -- start 2024-01-31T05:04:41.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.158+0000 7fa941800640 1 -- start start 2024-01-31T05:04:41.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa941800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 0x7fa93c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa941800640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c06d1a0 con 0x7fa93c0ccd00 2024-01-31T05:04:41.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa941800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93c06c600 con 0x7fa93c15b400 2024-01-31T05:04:41.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa941800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0cc960 con 0x7fa93c0cb810 2024-01-31T05:04:41.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa93b400640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa93c0ccd00 0x7fa93c071cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47602/0 (socket says 172.21.15.97:47602) 2024-01-31T05:04:41.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa93b400640 1 -- 172.21.15.97:0/4243184095 learned_addr learned my addr 172.21.15.97:0/4243184095 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:41.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa93aa00640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 0x7fa93c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.160+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230963210 0 0) 0x7fa93c06d1a0 con 0x7fa93c0ccd00 2024-01-31T05:04:41.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003520 con 0x7fa93c0ccd00 2024-01-31T05:04:41.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490500619 0 0) 0x7fa91c003520 con 0x7fa93c0ccd00 2024-01-31T05:04:41.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 msgr2=0x7fa93c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 0x7fa93c0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 >> v1:172.21.15.188:6789/0 conn(0x7fa93c15b400 legacy=0x7fa93c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0cc130 con 0x7fa93c0ccd00 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa934001670 con 0x7fa93c0ccd00 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 >> v1:172.21.15.97:6789/0 conn(0x7fa93c0ccd00 legacy=0x7fa93c071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0039a0 0x7fa91c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa91c0047c0 0x7fa91c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 0x7fa91c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0cc960 con 0x7fa91c0047c0 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c00ba60 con 0x7fa91c0039a0 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c00bc20 con 0x7fa93c0cb810 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.161+0000 7fa931e00640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa91c0047c0 0x7fa91c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.162+0000 7fa931e00640 1 -- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 msgr2=0x7fa91c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.162+0000 7fa931e00640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c0cb810 0x7fa91c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.162+0000 7fa931e00640 1 -- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0039a0 msgr2=0x7fa91c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.162+0000 7fa931e00640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0039a0 0x7fa91c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.162+0000 7fa931e00640 1 -- 172.21.15.97:0/4243184095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c003520 con 0x7fa91c0047c0 2024-01-31T05:04:41.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.163+0000 7fa931e00640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa91c0047c0 0x7fa91c004c40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa93c15b7e0 tx=0x7fa91c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef77f5b61b6cf061 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.163+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240052d0 con 0x7fa91c0047c0 2024-01-31T05:04:41.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.163+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92402e050 con 0x7fa91c0047c0 2024-01-31T05:04:41.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.163+0000 7fa93a000640 1 -- 172.21.15.97:0/4243184095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240035f0 con 0x7fa91c0047c0 2024-01-31T05:04:41.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa941800640 1 -- 172.21.15.97:0/4243184095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa91c0047c0 msgr2=0x7fa91c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.168 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa941800640 1 --2- 172.21.15.97:0/4243184095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa91c0047c0 0x7fa91c004c40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa93c15b7e0 tx=0x7fa91c0057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa93b400640 1 -- 172.21.15.97:0/4243184095 reap_dead start 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa941800640 1 -- 172.21.15.97:0/4243184095 shutdown_connections 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa941800640 1 -- 172.21.15.97:0/4243184095 >> 172.21.15.97:0/4243184095 conn(0x7fa93c0b0a10 msgr2=0x7fa93c0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa941800640 1 -- 172.21.15.97:0/4243184095 shutdown_connections 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.164+0000 7fa941800640 1 -- 172.21.15.97:0/4243184095 wait complete. 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.166+0000 7fa941800640 1 Processor -- start 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.166+0000 7fa941800640 1 -- start start 2024-01-31T05:04:41.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa941800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0047c0 0x7fa93c153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa941800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c1f0f10 0x7fa93c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa941800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 0x7fa93c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa941800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c0999c0 con 0x7fa93c1f20d0 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa941800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c15d740 con 0x7fa91c0047c0 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa941800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93c15dd50 con 0x7fa93c1f0f10 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa93b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 0x7fa93c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa93b400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 0x7fa93c0cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59866/0 (socket says 172.21.15.97:59866) 2024-01-31T05:04:41.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa93b400640 1 -- 172.21.15.97:0/2508371740 learned_addr learned my addr 172.21.15.97:0/2508371740 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.167+0000 7fa931e00640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c1f0f10 0x7fa93c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa93b400640 1 -- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c1f0f10 msgr2=0x7fa93c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa93b400640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c1f0f10 0x7fa93c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa93b400640 1 -- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0047c0 msgr2=0x7fa93c153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa93b400640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0047c0 0x7fa93c153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa93b400640 1 -- 172.21.15.97:0/2508371740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c1f1440 con 0x7fa93c1f20d0 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa931e00640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c1f0f10 0x7fa93c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:41.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa93b400640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 0x7fa93c0cd970 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa93c072e70 tx=0x7fa93400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934007440 con 0x7fa93c1f20d0 2024-01-31T05:04:41.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9340075a0 con 0x7fa93c1f20d0 2024-01-31T05:04:41.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93c1f1720 con 0x7fa93c1f20d0 2024-01-31T05:04:41.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934004a20 con 0x7fa93c1f20d0 2024-01-31T05:04:41.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.168+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93c1f1ce0 con 0x7fa93c1f20d0 2024-01-31T05:04:41.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.169+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93c0cb680 con 0x7fa93c1f20d0 2024-01-31T05:04:41.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.170+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa934004b80 con 0x7fa93c1f20d0 2024-01-31T05:04:41.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.171+0000 7fa933400640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa91406aed0 0x7fa91406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.171+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa934088d40 con 0x7fa93c1f20d0 2024-01-31T05:04:41.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.171+0000 7fa93aa00640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa91406aed0 0x7fa91406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.172+0000 7fa93aa00640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa91406aed0 0x7fa91406d390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa92c004b30 tx=0x7fa92c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.176+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fa934053d30 con 0x7fa93c1f20d0 2024-01-31T05:04:41.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.367+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa93c06cb50 con 0x7fa91406aed0 2024-01-31T05:04:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.368+0000 7fa933400640 1 -- 172.21.15.97:0/2508371740 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa92c00ec70 con 0x7fa91406aed0 2024-01-31T05:04:41.368 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2024-01-31T05:04:41.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa91406aed0 msgr2=0x7fa91406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa91406aed0 0x7fa91406d390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa92c004b30 tx=0x7fa92c00d040 comp rx=0 tx=0).stop 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 msgr2=0x7fa93c0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 0x7fa93c0cd970 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa93c072e70 tx=0x7fa93400c910 comp rx=0 tx=0).stop 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 shutdown_connections 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa91406aed0 0x7fa91406d390 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa93c1f20d0 0x7fa93c0cd970 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa93c1f0f10 0x7fa93c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 --2- 172.21.15.97:0/2508371740 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa91c0047c0 0x7fa93c153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 >> 172.21.15.97:0/2508371740 conn(0x7fa93c0b0a10 msgr2=0x7fa93c0ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:41.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 shutdown_connections 2024-01-31T05:04:41.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.371+0000 7fa941800640 1 -- 172.21.15.97:0/2508371740 wait complete. 2024-01-31T05:04:41.390 INFO:tasks.ceph:Scrubbing osd.6 2024-01-31T05:04:41.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.592+0000 7fa410600640 1 Processor -- start 2024-01-31T05:04:41.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.592+0000 7fa410600640 1 -- start start 2024-01-31T05:04:41.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.594+0000 7fa410600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 0x7fa40815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.594+0000 7fa410600640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40806e4b0 con 0x7fa4080ae220 2024-01-31T05:04:41.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.594+0000 7fa410600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40806dea0 con 0x7fa4080ad780 2024-01-31T05:04:41.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.594+0000 7fa410600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4080adb60 con 0x7fa40815ba60 2024-01-31T05:04:41.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.594+0000 7fa40d200640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa4080ae220 0x7fa408071020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47606/0 (socket says 172.21.15.97:47606) 2024-01-31T05:04:41.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.594+0000 7fa40d200640 1 -- 172.21.15.97:0/3047868714 learned_addr learned my addr 172.21.15.97:0/3047868714 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:41.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa40e600640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 0x7fa40815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601399074 0 0) 0x7fa40806e4b0 con 0x7fa4080ae220 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e8003520 con 0x7fa4080ae220 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322900473 0 0) 0x7fa3e8003520 con 0x7fa4080ae220 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 msgr2=0x7fa40815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 0x7fa40815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 >> v1:172.21.15.188:6789/0 conn(0x7fa4080ad780 legacy=0x7fa408070910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408071c90 con 0x7fa4080ae220 2024-01-31T05:04:41.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fa3f0001670 con 0x7fa4080ae220 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 >> v1:172.21.15.97:6789/0 conn(0x7fa4080ae220 legacy=0x7fa408071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 0x7fa3e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e8003f20 0x7fa3e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e8004d00 0x7fa3e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e800bb10 con 0x7fa3e8004d00 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e800bc50 con 0x7fa3e8003f20 2024-01-31T05:04:41.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.595+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e800be10 con 0x7fa40815ba60 2024-01-31T05:04:41.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40e600640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e8004d00 0x7fa3e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40dc00640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 0x7fa3e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40d200640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e8003f20 0x7fa3e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40e600640 1 -- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 msgr2=0x7fa3e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40e600640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40815ba60 0x7fa3e80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40e600640 1 -- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e8003f20 msgr2=0x7fa3e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40e600640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e8003f20 0x7fa3e80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.596+0000 7fa40e600640 1 -- 172.21.15.97:0/3047868714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8003520 con 0x7fa3e8004d00 2024-01-31T05:04:41.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.597+0000 7fa40e600640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e8004d00 0x7fa3e8005180 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa400009fc0 tx=0x7fa3e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941dc1328a791392 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.597+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40002cc20 con 0x7fa3e8004d00 2024-01-31T05:04:41.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40000f040 con 0x7fa3e8004d00 2024-01-31T05:04:41.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa407e00640 1 -- 172.21.15.97:0/3047868714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400034440 con 0x7fa3e8004d00 2024-01-31T05:04:41.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa410600640 1 -- 172.21.15.97:0/3047868714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e8004d00 msgr2=0x7fa3e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa410600640 1 --2- 172.21.15.97:0/3047868714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3e8004d00 0x7fa3e8005180 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa400009fc0 tx=0x7fa3e80057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa40e600640 1 -- 172.21.15.97:0/3047868714 reap_dead start 2024-01-31T05:04:41.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa410600640 1 -- 172.21.15.97:0/3047868714 shutdown_connections 2024-01-31T05:04:41.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa410600640 1 -- 172.21.15.97:0/3047868714 >> 172.21.15.97:0/3047868714 conn(0x7fa4080cede0 msgr2=0x7fa4080cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:41.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa410600640 1 -- 172.21.15.97:0/3047868714 shutdown_connections 2024-01-31T05:04:41.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.598+0000 7fa410600640 1 -- 172.21.15.97:0/3047868714 wait complete. 2024-01-31T05:04:41.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.600+0000 7fa410600640 1 Processor -- start 2024-01-31T05:04:41.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.600+0000 7fa410600640 1 -- start start 2024-01-31T05:04:41.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa410600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e8004d00 0x7fa4081544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa410600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4080999c0 0x7fa408099e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa410600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4080cc9b0 0x7fa4080cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa410600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4080ce320 con 0x7fa4080999c0 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa410600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa408154a30 con 0x7fa4080cc9b0 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa410600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40809a380 con 0x7fa3e8004d00 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.601+0000 7fa40d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4080999c0 0x7fa408099e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4080999c0 0x7fa408099e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59878/0 (socket says 172.21.15.97:59878) 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 -- 172.21.15.97:0/2397664747 learned_addr learned my addr 172.21.15.97:0/2397664747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40e600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4080cc9b0 0x7fa4080cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40dc00640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e8004d00 0x7fa4081544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 -- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e8004d00 msgr2=0x7fa4081544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e8004d00 0x7fa4081544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 -- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4080cc9b0 msgr2=0x7fa4080cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4080cc9b0 0x7fa4080cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 -- 172.21.15.97:0/2397664747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4080cd6a0 con 0x7fa4080999c0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.602+0000 7fa40d200640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4080999c0 0x7fa408099e40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d8e0 tx=0x7fa3f000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.603+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0001d80 con 0x7fa4080999c0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.603+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f00074e0 con 0x7fa4080999c0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.603+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0004a00 con 0x7fa4080999c0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.603+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4080cd980 con 0x7fa4080999c0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.603+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4080cdf40 con 0x7fa4080999c0 2024-01-31T05:04:41.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.604+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa40815bf40 con 0x7fa4080999c0 2024-01-31T05:04:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.605+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fa3f0004b60 con 0x7fa4080999c0 2024-01-31T05:04:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.606+0000 7fa405600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa3e006af00 0x7fa3e006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.606+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa3f0089080 con 0x7fa4080999c0 2024-01-31T05:04:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.606+0000 7fa40dc00640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa3e006af00 0x7fa3e006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.606+0000 7fa405600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7fa3e0071200 0x7fa3e0073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.606+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 --> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3e0073d10 con 0x7fa3e0071200 2024-01-31T05:04:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.606+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f008b740 con 0x7fa4080999c0 2024-01-31T05:04:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.607+0000 7fa40dc00640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa3e006af00 0x7fa3e006d3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001590 tx=0x7fa3f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.607+0000 7fa40e600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7fa3e0071200 0x7fa3e0073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.608+0000 7fa40e600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7fa3e0071200 0x7fa3e0073640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.612+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== osd.6 v2:172.21.15.188:6800/3025033562 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa3e0073d10 con 0x7fa3e0071200 2024-01-31T05:04:41.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.626+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 --> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa408071730 con 0x7fa3e0071200 2024-01-31T05:04:41.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.627+0000 7fa405600640 1 -- 172.21.15.97:0/2397664747 <== osd.6 v2:172.21.15.188:6800/3025033562 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa40002cae0 con 0x7fa3e0071200 2024-01-31T05:04:41.627 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:41.627 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:41.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7fa3e0071200 msgr2=0x7fa3e0073640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.188:6800/3025033562,v1:172.21.15.188:6801/3025033562] conn(0x7fa3e0071200 0x7fa3e0073640 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa3e006af00 msgr2=0x7fa3e006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fa3e006af00 0x7fa3e006d3c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001590 tx=0x7fa3f8009040 comp rx=0 tx=0).stop 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4080999c0 msgr2=0x7fa408099e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 --2- 172.21.15.97:0/2397664747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4080999c0 0x7fa408099e40 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000d8e0 tx=0x7fa3f000dda0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa40e600640 1 -- 172.21.15.97:0/2397664747 reap_dead start 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 shutdown_connections 2024-01-31T05:04:41.629 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 >> 172.21.15.97:0/2397664747 conn(0x7fa4080cede0 msgr2=0x7fa4080ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 shutdown_connections 2024-01-31T05:04:41.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.628+0000 7fa410600640 1 -- 172.21.15.97:0/2397664747 wait complete. 2024-01-31T05:04:41.646 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-31T05:04:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.847+0000 7fb3d5800640 1 Processor -- start 2024-01-31T05:04:41.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.848+0000 7fb3d5800640 1 -- start start 2024-01-31T05:04:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3d5800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3d015b540 0x7fb3d00cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3d5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b920 0x7fb3d00cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3d5800640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3d00b21c0 0x7fb3d00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3d5800640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00ccef0 con 0x7fb3d00b21c0 2024-01-31T05:04:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3d5800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00d1d30 con 0x7fb3d015b540 2024-01-31T05:04:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3d5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00cc8a0 con 0x7fb3d015b920 2024-01-31T05:04:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3cf400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3d00b21c0 0x7fb3d00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3cf400640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb3d00b21c0 0x7fb3d00cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59892/0 (socket says 172.21.15.97:59892) 2024-01-31T05:04:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.849+0000 7fb3cf400640 1 -- 172.21.15.97:0/3523231861 learned_addr learned my addr 172.21.15.97:0/3523231861 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3cea00640 1 --2- 172.21.15.97:0/3523231861 >> v2:172.21.15.188:3300/0 conn(0x7fb3d015b540 0x7fb3d00cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3ce000640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b920 0x7fb3d00cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3cf400640 1 -- 172.21.15.97:0/3523231861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b920 msgr2=0x7fb3d00cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3cf400640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b920 0x7fb3d00cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3cf400640 1 -- 172.21.15.97:0/3523231861 >> v2:172.21.15.188:3300/0 conn(0x7fb3d015b540 msgr2=0x7fb3d00cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3cf400640 1 --2- 172.21.15.97:0/3523231861 >> v2:172.21.15.188:3300/0 conn(0x7fb3d015b540 0x7fb3d00cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3cf400640 1 -- 172.21.15.97:0/3523231861 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00ccb80 con 0x7fb3d00b21c0 2024-01-31T05:04:41.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.850+0000 7fb3ce000640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b920 0x7fb3d00cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:41.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.851+0000 7fb3cf400640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d00cee50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb3d006e0c0 tx=0x7fb3c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1e853895022f24 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.851+0000 7fb3cd600640 1 -- 172.21.15.97:0/3523231861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8031870 con 0x7fb3d00b21c0 2024-01-31T05:04:41.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.851+0000 7fb3cd600640 1 -- 172.21.15.97:0/3523231861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c80319d0 con 0x7fb3d00b21c0 2024-01-31T05:04:41.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.851+0000 7fb3cd600640 1 -- 172.21.15.97:0/3523231861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8031d60 con 0x7fb3d00b21c0 2024-01-31T05:04:41.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 -- 172.21.15.97:0/3523231861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 msgr2=0x7fb3d00cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d00cee50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb3d006e0c0 tx=0x7fb3c802b910 comp rx=0 tx=0).stop 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 -- 172.21.15.97:0/3523231861 shutdown_connections 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d00cee50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 --2- 172.21.15.97:0/3523231861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b920 0x7fb3d00cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 --2- 172.21.15.97:0/3523231861 >> v2:172.21.15.188:3300/0 conn(0x7fb3d015b540 0x7fb3d00cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 -- 172.21.15.97:0/3523231861 >> 172.21.15.97:0/3523231861 conn(0x7fb3d00d1070 msgr2=0x7fb3d00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 -- 172.21.15.97:0/3523231861 shutdown_connections 2024-01-31T05:04:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.852+0000 7fb3d5800640 1 -- 172.21.15.97:0/3523231861 wait complete. 2024-01-31T05:04:41.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.854+0000 7fb3d5800640 1 Processor -- start 2024-01-31T05:04:41.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.854+0000 7fb3d5800640 1 -- start start 2024-01-31T05:04:41.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3d5800640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d01581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3d5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b540 0x7fb3d00c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3d5800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3d015b920 0x7fb3d0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3d5800640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0151de0 con 0x7fb3d00b21c0 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3d5800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0158710 con 0x7fb3d015b920 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3d5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00c9100 con 0x7fb3d015b540 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d01581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3cea00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d01581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59902/0 (socket says 172.21.15.97:59902) 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.855+0000 7fb3cea00640 1 -- 172.21.15.97:0/324060357 learned_addr learned my addr 172.21.15.97:0/324060357 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3ce000640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b540 0x7fb3d00c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cf400640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3d015b920 0x7fb3d0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cea00640 1 -- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b540 msgr2=0x7fb3d00c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cea00640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b540 0x7fb3d00c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cea00640 1 -- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3d015b920 msgr2=0x7fb3d0159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cea00640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3d015b920 0x7fb3d0159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cea00640 1 -- 172.21.15.97:0/324060357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80094d0 con 0x7fb3d00b21c0 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3ce000640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b540 0x7fb3d00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:41.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3cea00640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d01581d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000d800 tx=0x7fb3c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.856+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0009640 con 0x7fb3d00b21c0 2024-01-31T05:04:41.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.857+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c00097a0 con 0x7fb3d00b21c0 2024-01-31T05:04:41.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.857+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0009b30 con 0x7fb3d00b21c0 2024-01-31T05:04:41.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.857+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d00c9d70 con 0x7fb3d00b21c0 2024-01-31T05:04:41.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.857+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d015c090 con 0x7fb3d00b21c0 2024-01-31T05:04:41.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.858+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d00b2960 con 0x7fb3d00b21c0 2024-01-31T05:04:41.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.859+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fb3c0009c90 con 0x7fb3d00b21c0 2024-01-31T05:04:41.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.860+0000 7fb3c6a00640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb3a806aed0 0x7fb3a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:41.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.860+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb3c00898e0 con 0x7fb3d00b21c0 2024-01-31T05:04:41.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.860+0000 7fb3ce000640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb3a806aed0 0x7fb3a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:41.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.860+0000 7fb3ce000640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb3a806aed0 0x7fb3a806d390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb3d015f1c0 tx=0x7fb3b8007490 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:41.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:41.864+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fb3c0054960 con 0x7fb3d00b21c0 2024-01-31T05:04:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.062+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3d0067790 con 0x7fb3a806aed0 2024-01-31T05:04:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.062+0000 7fb3c6a00640 1 -- 172.21.15.97:0/324060357 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb3d0067790 con 0x7fb3a806aed0 2024-01-31T05:04:42.062 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2024-01-31T05:04:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb3a806aed0 msgr2=0x7fb3a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb3a806aed0 0x7fb3a806d390 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb3d015f1c0 tx=0x7fb3b8007490 comp rx=0 tx=0).stop 2024-01-31T05:04:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 msgr2=0x7fb3d01581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d01581d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000d800 tx=0x7fb3c000dcc0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 shutdown_connections 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3d015b920 0x7fb3d0159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3d015b540 0x7fb3d00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fb3a806aed0 0x7fb3a806d390 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 --2- 172.21.15.97:0/324060357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3d00b21c0 0x7fb3d01581d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.065+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 >> 172.21.15.97:0/324060357 conn(0x7fb3d00d1070 msgr2=0x7fb3d006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.066+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 shutdown_connections 2024-01-31T05:04:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.066+0000 7fb3d5800640 1 -- 172.21.15.97:0/324060357 wait complete. 2024-01-31T05:04:42.083 INFO:tasks.ceph:Scrubbing osd.7 2024-01-31T05:04:42.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-31T05:04:42.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.297+0000 7f198d600640 1 Processor -- start 2024-01-31T05:04:42.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.297+0000 7f198d600640 1 -- start start 2024-01-31T05:04:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.298+0000 7f198d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 0x7f198806d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.298+0000 7f198d600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19880710a0 0x7f198806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.298+0000 7f198d600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198806ced0 con 0x7f198815cfa0 2024-01-31T05:04:42.298 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.298+0000 7f198d600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1988071610 con 0x7f19880710a0 2024-01-31T05:04:42.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.299+0000 7f198d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880718f0 con 0x7f19880c9f70 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.299+0000 7f197fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 0x7f198806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.299+0000 7f197fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 0x7f198806d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:48058/0 (socket says 172.21.15.97:48058) 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.299+0000 7f197fe00640 1 -- 172.21.15.97:0/3996481054 learned_addr learned my addr 172.21.15.97:0/3996481054 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.300+0000 7f1987400640 1 --2- 172.21.15.97:0/3996481054 >> v2:172.21.15.188:3300/0 conn(0x7f19880710a0 0x7f198806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.300+0000 7f197fe00640 1 -- 172.21.15.97:0/3996481054 >> v2:172.21.15.188:3300/0 conn(0x7f19880710a0 msgr2=0x7f198806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.300+0000 7f197fe00640 1 --2- 172.21.15.97:0/3996481054 >> v2:172.21.15.188:3300/0 conn(0x7f19880710a0 0x7f198806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.300+0000 7f197fe00640 1 -- 172.21.15.97:0/3996481054 >> v1:172.21.15.188:6789/0 conn(0x7f198815cfa0 legacy=0x7f19880c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.300+0000 7f197fe00640 1 -- 172.21.15.97:0/3996481054 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198806cb60 con 0x7f19880c9f70 2024-01-31T05:04:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.301+0000 7f197fe00640 1 --2- 172.21.15.97:0/3996481054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 0x7f198806d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1970009870 tx=0x7f197002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bb0e38085c239a server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f1986000640 1 -- 172.21.15.97:0/3996481054 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970030840 con 0x7f19880c9f70 2024-01-31T05:04:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f1986000640 1 -- 172.21.15.97:0/3996481054 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19700309a0 con 0x7f19880c9f70 2024-01-31T05:04:42.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f1986000640 1 -- 172.21.15.97:0/3996481054 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970030d30 con 0x7f19880c9f70 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f198d600640 1 -- 172.21.15.97:0/3996481054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 msgr2=0x7f198806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f198d600640 1 --2- 172.21.15.97:0/3996481054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 0x7f198806d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1970009870 tx=0x7f197002b980 comp rx=0 tx=0).stop 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f198d600640 1 -- 172.21.15.97:0/3996481054 shutdown_connections 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f198d600640 1 --2- 172.21.15.97:0/3996481054 >> v2:172.21.15.188:3300/0 conn(0x7f19880710a0 0x7f198806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.302+0000 7f198d600640 1 --2- 172.21.15.97:0/3996481054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19880c9f70 0x7f198806d360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.303+0000 7f198d600640 1 -- 172.21.15.97:0/3996481054 >> 172.21.15.97:0/3996481054 conn(0x7f19880cede0 msgr2=0x7f19880cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:42.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.303+0000 7f198d600640 1 -- 172.21.15.97:0/3996481054 shutdown_connections 2024-01-31T05:04:42.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.303+0000 7f198d600640 1 -- 172.21.15.97:0/3996481054 wait complete. 2024-01-31T05:04:42.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.304+0000 7f198d600640 1 Processor -- start 2024-01-31T05:04:42.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.304+0000 7f198d600640 1 -- start start 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.305+0000 7f198d600640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19880710a0 0x7f19881f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.305+0000 7f198d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19880c9f70 0x7f19880cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.305+0000 7f198d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198815cfa0 0x7f19881f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.305+0000 7f198d600640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988155930 con 0x7f19880710a0 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.305+0000 7f198d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881f3800 con 0x7f19880c9f70 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.305+0000 7f198d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19881f3ad0 con 0x7f198815cfa0 2024-01-31T05:04:42.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19880710a0 0x7f19881f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19880710a0 0x7f19881f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59912/0 (socket says 172.21.15.97:59912) 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 -- 172.21.15.97:0/4104598384 learned_addr learned my addr 172.21.15.97:0/4104598384 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f197fe00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19880c9f70 0x7f19880cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1987400640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198815cfa0 0x7f19881f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 -- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198815cfa0 msgr2=0x7f19881f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198815cfa0 0x7f19881f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 -- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19880c9f70 msgr2=0x7f19880cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19880c9f70 0x7f19880cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 -- 172.21.15.97:0/4104598384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700094d0 con 0x7f19880710a0 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.306+0000 7f1986a00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19880710a0 0x7f19881f32c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f197800c400 tx=0x7f197800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.307+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780052d0 con 0x7f19880710a0 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.307+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978005450 con 0x7f19880710a0 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.307+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978007260 con 0x7f19880710a0 2024-01-31T05:04:42.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.307+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198815d500 con 0x7f19880710a0 2024-01-31T05:04:42.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.307+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881f4090 con 0x7f19880710a0 2024-01-31T05:04:42.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.308+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f198815d380 con 0x7f19880710a0 2024-01-31T05:04:42.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.309+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f19780073c0 con 0x7f19880710a0 2024-01-31T05:04:42.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.310+0000 7f197ea00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f196006aed0 0x7f196006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.310+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f19780894e0 con 0x7f19880710a0 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.310+0000 7f197ea00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7f19600711d0 0x7f1960073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.310+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 --> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1960073ce0 con 0x7f19600711d0 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.310+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1978089850 con 0x7f19880710a0 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.310+0000 7f197fe00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f196006aed0 0x7f196006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.311+0000 7f1987400640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7f19600711d0 0x7f1960073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.311+0000 7f197fe00640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f196006aed0 0x7f196006d390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1970036000 tx=0x7f1970008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.312+0000 7f1987400640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7f19600711d0 0x7f1960073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.314+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== osd.7 v2:172.21.15.188:6808/2866753840 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1960073ce0 con 0x7f19600711d0 2024-01-31T05:04:42.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.328+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 --> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1988072400 con 0x7f19600711d0 2024-01-31T05:04:42.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f197ea00640 1 -- 172.21.15.97:0/4104598384 <== osd.7 v2:172.21.15.188:6808/2866753840 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1980009b70 con 0x7f19600711d0 2024-01-31T05:04:42.330 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-01-31T05:04:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-31T05:04:42.330 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-01-31T05:04:42.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7f19600711d0 msgr2=0x7f1960073610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f198d600640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.188:6808/2866753840,v1:172.21.15.188:6810/2866753840] conn(0x7f19600711d0 0x7f1960073610 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f196006aed0 msgr2=0x7f196006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f198d600640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f196006aed0 0x7f196006d390 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1970036000 tx=0x7f1970008030 comp rx=0 tx=0).stop 2024-01-31T05:04:42.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19880710a0 msgr2=0x7f19881f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.330+0000 7f198d600640 1 --2- 172.21.15.97:0/4104598384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19880710a0 0x7f19881f32c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f197800c400 tx=0x7f197800c8c0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.331+0000 7f1987400640 1 -- 172.21.15.97:0/4104598384 reap_dead start 2024-01-31T05:04:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.331+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 shutdown_connections 2024-01-31T05:04:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.331+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 >> 172.21.15.97:0/4104598384 conn(0x7f19880cede0 msgr2=0x7f19880ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.331+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 shutdown_connections 2024-01-31T05:04:42.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.331+0000 7f198d600640 1 -- 172.21.15.97:0/4104598384 wait complete. 2024-01-31T05:04:42.346 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-31T05:04:42.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.547+0000 7fdcea000640 1 Processor -- start 2024-01-31T05:04:42.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.548+0000 7fdcea000640 1 -- start start 2024-01-31T05:04:42.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdcea000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdce40ccd00 0x7fdce40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdcea000640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce406d1a0 con 0x7fdce415b400 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdcea000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce406c600 con 0x7fdce40cb810 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdcea000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce40cc960 con 0x7fdce40ccd00 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdce2a00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdce415b400 0x7fdce406d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47622/0 (socket says 172.21.15.97:47622) 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdce2a00640 1 -- 172.21.15.97:0/1412768527 learned_addr learned my addr 172.21.15.97:0/1412768527 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.549+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282976461 0 0) 0x7fdce406d1a0 con 0x7fdce415b400 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc4003520 con 0x7fdce415b400 2024-01-31T05:04:42.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce3e00640 1 --2- 172.21.15.97:0/1412768527 >> v2:172.21.15.188:3300/0 conn(0x7fdce40ccd00 0x7fdce40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1143967904 0 0) 0x7fdcc4003520 con 0x7fdce415b400 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 >> v2:172.21.15.188:3300/0 conn(0x7fdce40ccd00 msgr2=0x7fdce40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 --2- 172.21.15.97:0/1412768527 >> v2:172.21.15.188:3300/0 conn(0x7fdce40ccd00 0x7fdce40cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 >> v1:172.21.15.188:6789/0 conn(0x7fdce40cb810 legacy=0x7fdce40cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40cd6d0 con 0x7fdce415b400 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7fdccc001670 con 0x7fdce415b400 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 >> v1:172.21.15.97:6789/0 conn(0x7fdce415b400 legacy=0x7fdce406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4003970 0x7fdcc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc40047c0 0x7fdcc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4005180 0x7fdcc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40cc960 con 0x7fdcc4005180 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc400c0d0 con 0x7fdcc40047c0 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc400c290 con 0x7fdcc4003970 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.550+0000 7fdce3e00640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4005180 0x7fdcc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce2a00640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc40047c0 0x7fdcc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3400640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4003970 0x7fdcc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3e00640 1 -- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4003970 msgr2=0x7fdcc4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3e00640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4003970 0x7fdcc4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3e00640 1 -- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc40047c0 msgr2=0x7fdcc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3e00640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc40047c0 0x7fdcc4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3e00640 1 -- 172.21.15.97:0/1412768527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4003520 con 0x7fdcc4005180 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce3400640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4003970 0x7fdcc4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.551+0000 7fdce2a00640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc40047c0 0x7fdcc4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.552+0000 7fdce3e00640 1 -- 172.21.15.97:0/1412768527 reap_dead start 2024-01-31T05:04:42.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.552+0000 7fdce3e00640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4005180 0x7fdcc400b980 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007360 tx=0x7fdcc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e14daea7e3ca5e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.552+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02ccf0 con 0x7fdcc4005180 2024-01-31T05:04:42.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.552+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc02ce90 con 0x7fdcc4005180 2024-01-31T05:04:42.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.552+0000 7fdce2000640 1 -- 172.21.15.97:0/1412768527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc003bd0 con 0x7fdcc4005180 2024-01-31T05:04:42.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 -- 172.21.15.97:0/1412768527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4005180 msgr2=0x7fdcc400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4005180 0x7fdcc400b980 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007360 tx=0x7fdcc40057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 -- 172.21.15.97:0/1412768527 shutdown_connections 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 --2- 172.21.15.97:0/1412768527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcc4005180 0x7fdcc400b980 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 -- 172.21.15.97:0/1412768527 >> 172.21.15.97:0/1412768527 conn(0x7fdce40b0a10 msgr2=0x7fdce40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 -- 172.21.15.97:0/1412768527 shutdown_connections 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.553+0000 7fdcea000640 1 -- 172.21.15.97:0/1412768527 wait complete. 2024-01-31T05:04:42.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.554+0000 7fdcea000640 1 Processor -- start 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.555+0000 7fdcea000640 1 -- start start 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdcea000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4005180 0x7fdce414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdcea000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 0x7fdce4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdcea000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce415df00 0x7fdce4154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdcea000640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415d5f0 con 0x7fdce415da80 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdcea000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce41503b0 con 0x7fdce415df00 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdcea000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce415c9e0 con 0x7fdcc4005180 2024-01-31T05:04:42.560 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 0x7fdce4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 0x7fdce4154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59934/0 (socket says 172.21.15.97:59934) 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 -- 172.21.15.97:0/1060846734 learned_addr learned my addr 172.21.15.97:0/1060846734 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 -- 172.21.15.97:0/1060846734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4005180 msgr2=0x7fdce414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4005180 0x7fdce414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 -- 172.21.15.97:0/1060846734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce415df00 msgr2=0x7fdce4154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce415df00 0x7fdce4154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.556+0000 7fdce2a00640 1 -- 172.21.15.97:0/1060846734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce415ccc0 con 0x7fdce415da80 2024-01-31T05:04:42.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.557+0000 7fdce2a00640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 0x7fdce4154590 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fdccc000f60 tx=0x7fdccc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.557+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0074e0 con 0x7fdce415da80 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.557+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc007640 con 0x7fdce415da80 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.557+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce415cfa0 con 0x7fdce415da80 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.557+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0049e0 con 0x7fdce415da80 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.557+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce4157910 con 0x7fdce415da80 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.558+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce40677c0 con 0x7fdce415da80 2024-01-31T05:04:42.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.559+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7fdccc004b40 con 0x7fdce415da80 2024-01-31T05:04:42.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.561+0000 7fdcdb400640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdcbc06aed0 0x7fdcbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.561+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fdccc0891f0 con 0x7fdce415da80 2024-01-31T05:04:42.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.561+0000 7fdce3400640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdcbc06aed0 0x7fdcbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.562+0000 7fdce3400640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdcbc06aed0 0x7fdcbc06d390 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001650 tx=0x7fdcd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.565+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7fdccc0541e0 con 0x7fdce415da80 2024-01-31T05:04:42.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.762+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fdce406ce30 con 0x7fdcbc06aed0 2024-01-31T05:04:42.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.762+0000 7fdcdb400640 1 -- 172.21.15.97:0/1060846734 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdcd400d520 con 0x7fdcbc06aed0 2024-01-31T05:04:42.762 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2024-01-31T05:04:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdcbc06aed0 msgr2=0x7fdcbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdcbc06aed0 0x7fdcbc06d390 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4001650 tx=0x7fdcd4009040 comp rx=0 tx=0).stop 2024-01-31T05:04:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 msgr2=0x7fdce4154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 0x7fdce4154590 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fdccc000f60 tx=0x7fdccc00dc90 comp rx=0 tx=0).stop 2024-01-31T05:04:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 shutdown_connections 2024-01-31T05:04:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdce415df00 0x7fdce4154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7fdcbc06aed0 0x7fdcbc06d390 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdce415da80 0x7fdce4154590 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 --2- 172.21.15.97:0/1060846734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcc4005180 0x7fdce414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 >> 172.21.15.97:0/1060846734 conn(0x7fdce40b0a10 msgr2=0x7fdce40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 shutdown_connections 2024-01-31T05:04:42.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.765+0000 7fdcea000640 1 -- 172.21.15.97:0/1060846734 wait complete. 2024-01-31T05:04:42.781 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-31T05:04:42.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.983+0000 7ff198a00640 1 Processor -- start 2024-01-31T05:04:42.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.984+0000 7ff198a00640 1 -- start start 2024-01-31T05:04:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.985+0000 7ff198a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff190161640 0x7ff1900cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.985+0000 7ff198a00640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1900ce7c0 con 0x7ff1901578d0 2024-01-31T05:04:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.985+0000 7ff198a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff190071fd0 con 0x7ff19006d590 2024-01-31T05:04:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.985+0000 7ff198a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff190072870 con 0x7ff190161640 2024-01-31T05:04:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.985+0000 7ff196000640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff1901578d0 0x7ff190071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47638/0 (socket says 172.21.15.97:47638) 2024-01-31T05:04:42.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.985+0000 7ff196000640 1 -- 172.21.15.97:0/2940420058 learned_addr learned my addr 172.21.15.97:0/2940420058 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344433098 0 0) 0x7ff1900ce7c0 con 0x7ff1901578d0 2024-01-31T05:04:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170003520 con 0x7ff1901578d0 2024-01-31T05:04:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff196a00640 1 --2- 172.21.15.97:0/2940420058 >> v2:172.21.15.188:3300/0 conn(0x7ff190161640 0x7ff1900cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3621429747 0 0) 0x7ff170003520 con 0x7ff1901578d0 2024-01-31T05:04:42.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 >> v2:172.21.15.188:3300/0 conn(0x7ff190161640 msgr2=0x7ff1900cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 --2- 172.21.15.97:0/2940420058 >> v2:172.21.15.188:3300/0 conn(0x7ff190161640 0x7ff1900cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 >> v1:172.21.15.188:6789/0 conn(0x7ff19006d590 legacy=0x7ff190160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900722b0 con 0x7ff1901578d0 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4251403820 0 0) 0x7ff1800019f0 con 0x7ff1901578d0 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 >> v1:172.21.15.97:6789/0 conn(0x7ff1901578d0 legacy=0x7ff190071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff170003970 0x7ff170003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff170004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff170005180 0x7ff17000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.986+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190072870 con 0x7ff1700047c0 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000c0d0 con 0x7ff170005180 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000c290 con 0x7ff170003970 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff195600640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff170004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff195600640 1 -- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff170003970 msgr2=0x7ff170003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff195600640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff170003970 0x7ff170003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff195600640 1 -- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff170005180 msgr2=0x7ff17000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:42.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff196a00640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff170005180 0x7ff17000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff195600640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff170005180 0x7ff17000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff195600640 1 -- 172.21.15.97:0/2940420058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170003520 con 0x7ff1700047c0 2024-01-31T05:04:42.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.987+0000 7ff196a00640 1 -- 172.21.15.97:0/2940420058 reap_dead start 2024-01-31T05:04:42.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.988+0000 7ff195600640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff170004c40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff178001750 tx=0x7ff1700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ad8df0c1cec46e server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.988+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1780052d0 con 0x7ff1700047c0 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.988+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17802e050 con 0x7ff1700047c0 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.988+0000 7ff194c00640 1 -- 172.21.15.97:0/2940420058 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff178003830 con 0x7ff1700047c0 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 -- 172.21.15.97:0/2940420058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 msgr2=0x7ff170004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff170004c40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff178001750 tx=0x7ff1700057f0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 -- 172.21.15.97:0/2940420058 shutdown_connections 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 --2- 172.21.15.97:0/2940420058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff170004c40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 -- 172.21.15.97:0/2940420058 >> 172.21.15.97:0/2940420058 conn(0x7ff1900b1a90 msgr2=0x7ff1900b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:42.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 -- 172.21.15.97:0/2940420058 shutdown_connections 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.989+0000 7ff198a00640 1 -- 172.21.15.97:0/2940420058 wait complete. 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.991+0000 7ff198a00640 1 Processor -- start 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.991+0000 7ff198a00640 1 -- start start 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff198a00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff1901f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff198a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff190152c70 0x7ff19014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff198a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1901530f0 0x7ff1901e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff198a00640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900acd70 con 0x7ff1700047c0 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff198a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1901535e0 con 0x7ff1901530f0 2024-01-31T05:04:42.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff198a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900acfc0 con 0x7ff190152c70 2024-01-31T05:04:42.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff196000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff1901f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff196000640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff1901f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59964/0 (socket says 172.21.15.97:59964) 2024-01-31T05:04:42.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.992+0000 7ff196000640 1 -- 172.21.15.97:0/4659501 learned_addr learned my addr 172.21.15.97:0/4659501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:42.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1901530f0 0x7ff1901e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.995 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff195600640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff190152c70 0x7ff19014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196000640 1 -- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff190152c70 msgr2=0x7ff19014d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196000640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff190152c70 0x7ff19014d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196000640 1 -- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1901530f0 msgr2=0x7ff1901e6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196000640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1901530f0 0x7ff1901e6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196000640 1 -- 172.21.15.97:0/4659501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900ad2a0 con 0x7ff1700047c0 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff196000640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff1901f36a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff180001370 tx=0x7ff18000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18000a890 con 0x7ff1700047c0 2024-01-31T05:04:42.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18000aa10 con 0x7ff1700047c0 2024-01-31T05:04:42.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1900ad580 con 0x7ff1700047c0 2024-01-31T05:04:42.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18000adc0 con 0x7ff1700047c0 2024-01-31T05:04:42.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.993+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19014de60 con 0x7ff1700047c0 2024-01-31T05:04:42.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.995+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1900ad7c0 con 0x7ff1700047c0 2024-01-31T05:04:42.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.995+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7ff1800070d0 con 0x7ff1700047c0 2024-01-31T05:04:42.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.997+0000 7ff18e000640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff16806aed0 0x7ff16806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:42.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.997+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff180088ff0 con 0x7ff1700047c0 2024-01-31T05:04:42.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.997+0000 7ff195600640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff16806aed0 0x7ff16806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:42.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:42.998+0000 7ff195600640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff16806aed0 0x7ff16806d390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff178033000 tx=0x7ff178009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.001+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7ff1800bc190 con 0x7ff1700047c0 2024-01-31T05:04:43.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.191+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 --> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1900cf120 con 0x7ff16806aed0 2024-01-31T05:04:43.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.197+0000 7ff18e000640 1 -- 172.21.15.97:0/4659501 <== mgr.4104 v2:172.21.15.97:6832/631419576 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48316 (secure 0 0 0) 0x7ff1900cf120 con 0x7ff16806aed0 2024-01-31T05:04:43.197 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:43.197 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-01-31T05:04:43.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.199+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff16806aed0 msgr2=0x7ff16806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.199+0000 7ff198a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff16806aed0 0x7ff16806d390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff178033000 tx=0x7ff178009030 comp rx=0 tx=0).stop 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 msgr2=0x7ff1901f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff1901f36a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff180001370 tx=0x7ff18000a060 comp rx=0 tx=0).stop 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 shutdown_connections 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1901530f0 0x7ff1901e6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff190152c70 0x7ff19014d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7ff16806aed0 0x7ff16806d390 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 --2- 172.21.15.97:0/4659501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1700047c0 0x7ff1901f36a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 >> 172.21.15.97:0/4659501 conn(0x7ff1900b1a90 msgr2=0x7ff19015fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:43.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 shutdown_connections 2024-01-31T05:04:43.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.200+0000 7ff198a00640 1 -- 172.21.15.97:0/4659501 wait complete. 2024-01-31T05:04:43.219 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-01-31T05:04:41.628361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222776,"kb_used_data":6600,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751944,"statfs":{"total":773094113280,"available":772865990656,"internally_reserved":0,"allocated":6758400,"data_stored":3640688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002748"},"pg_stats":[{"pgid":"1.0","version":"58'119","reported_seq":249,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-01-31T05:04:39.515734+0000","last_change":"2024-01-31T05:04:39.515765+0000","last_active":"2024-01-31T05:04:39.515734+0000","last_peered":"2024-01-31T05:04:39.515734+0000","last_clean":"2024-01-31T05:04:39.515734+0000","last_became_active":"2024-01-31T05:00:41.347499+0000","last_became_peered":"2024-01-31T05:00:41.347499+0000","last_unstale":"2024-01-31T05:04:39.515734+0000","last_undegraded":"2024-01-31T05:04:39.515734+0000","last_fullsized":"2024-01-31T05:04:39.515734+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'119","last_scrub_stamp":"2024-01-31T05:04:39.515662+0000","last_deep_scrub":"58'119","last_deep_scrub_stamp":"2024-01-31T05:04:39.515662+0000","last_clean_scrub_stamp":"2024-01-31T05:04:39.515662+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-02-01T08:10:14.740765+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.64900000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":1.514,"15min":1.514},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.0469999999999999,"15min":1.012},"min":{"1min":0.69499999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.76300000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0960000000000001,"15min":1.0880000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0259999999999998,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6890000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.095}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.67100000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8100000000000001,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2869999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"max":{"1min":1.8720000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.2110000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.78000000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7270000000000001,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.153},{"interface":"front","average":{"1min":1.246,"5min":1.2230000000000001,"15min":1.2250000000000001},"min":{"1min":0.749,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.792,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.2829999999999999}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3540000000000001,"15min":1.3520000000000001},"min":{"1min":1.0309999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9670000000000001,"5min":2.25,"15min":2.25},"last":1.244},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3660000000000001},"min":{"1min":0.95499999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.8049999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.47}]},{"osd":5,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.476,"15min":1.476},"min":{"1min":1.0169999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.905,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.357},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.363,"15min":1.363},"min":{"1min":0.92900000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.784,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.4259999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4570000000000001,"15min":1.4810000000000001},"min":{"1min":1.044,"5min":0.94299999999999995,"15min":0.94299999999999995},"max":{"1min":1.617,"5min":2.5750000000000002,"15min":2.5750000000000002},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5409999999999999,"15min":1.5569999999999999},"min":{"1min":1.0209999999999999,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8660000000000001,"5min":2.629,"15min":2.629},"last":1.5289999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0329999999999999,"15min":1.03},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.403,"5min":1.476,"15min":1.476},"last":1.141},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":0.99299999999999999},"min":{"1min":0.623,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":1.5129999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.167,"15min":1.1830000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.502,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.087},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.129,"15min":1.1339999999999999},"min":{"1min":0.80800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6659999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1930000000000001,"15min":1.1830000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.5800000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.036},{"interface":"front","average":{"1min":1.04,"5min":1.1499999999999999,"15min":1.1739999999999999},"min":{"1min":0.60599999999999998,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.5900000000000001,"5min":1.883,"15min":1.883},"last":1.24}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.345,"15min":1.383},"min":{"1min":0.84099999999999997,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.536,"5min":1.73,"15min":1.73},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.413,"15min":1.425},"min":{"1min":1.0549999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.6259999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.2789999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.3460000000000001,"15min":1.403},"min":{"1min":0.80800000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.478,"5min":2.032,"15min":2.032},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.395,"5min":1.5289999999999999,"15min":1.571},"min":{"1min":1.0680000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"max":{"1min":1.7,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":1.458}]},{"osd":6,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.488,"15min":1.522},"min":{"1min":0.98099999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7190000000000001,"5min":2.335,"15min":2.335},"last":1.196},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.5189999999999999,"15min":1.583},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.651,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":1.3560000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.5389999999999999,"15min":1.5960000000000001},"min":{"1min":0.89100000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.696,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.399},{"interface":"front","average":{"1min":1.474,"5min":1.597,"15min":1.645},"min":{"1min":1.0960000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.7729999999999999,"5min":2.6269999999999998,"15min":2.6269999999999998},"last":1.3380000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":1.0169999999999999,"15min":1.127},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.3009999999999999,"5min":4.8529999999999998,"15min":4.8529999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.995,"15min":1.071},"min":{"1min":0.53900000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.615,"5min":4.5350000000000001,"15min":4.5350000000000001},"last":0.75700000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":1.0649999999999999,"15min":1.159},"min":{"1min":0.57599999999999996,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.77,"5min":5.617,"15min":5.617},"last":1.704},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.1579999999999999,"15min":1.274},"min":{"1min":0.628,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3500000000000001,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":1.1799999999999999}]},{"osd":2,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.224,"15min":1.345},"min":{"1min":0.61299999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.395,"5min":5.6760000000000002,"15min":5.6760000000000002},"last":1.02},{"interface":"front","average":{"1min":1.125,"5min":1.3859999999999999,"15min":1.5389999999999999},"min":{"1min":0.53200000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.5680000000000001,"5min":7.2850000000000001,"15min":7.2850000000000001},"last":1.419}]},{"osd":4,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5089999999999999,"15min":1.6850000000000001},"min":{"1min":0.89700000000000002,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.921,"5min":8.2850000000000001,"15min":8.2850000000000001},"last":1.377},{"interface":"front","average":{"1min":1.351,"5min":1.5429999999999999,"15min":1.675},"min":{"1min":1.135,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.869,"5min":6.2949999999999999,"15min":6.2949999999999999},"last":1.54}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.5609999999999999,"15min":1.6970000000000001},"min":{"1min":0.98299999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.9850000000000001,"5min":6.048,"15min":6.048},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.637,"15min":1.7949999999999999},"min":{"1min":0.94699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8109999999999999,"5min":7.3789999999999996,"15min":7.3789999999999996},"last":1.5800000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.601,"15min":1.7190000000000001},"min":{"1min":0.995,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.8759999999999999,"5min":5.0490000000000004,"15min":5.0490000000000004},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.5580000000000001,"15min":1.7150000000000001},"min":{"1min":0.88,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6459999999999999,"5min":5.1669999999999998,"15min":5.1669999999999998},"last":1.2849999999999999}]},{"osd":7,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.573,"15min":1.7130000000000001},"min":{"1min":0.86899999999999999,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0499999999999998,"5min":4.9050000000000002,"15min":4.9050000000000002},"last":1.621},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.7310000000000001,"15min":1.911},"min":{"1min":1.0509999999999999,"5min":0.746,"15min":0.746},"max":{"1min":1.9370000000000001,"5min":7.7839999999999998,"15min":7.7839999999999998},"last":1.829}]}]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.78300000000000003,"15min":0.80500000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1080000000000001,"5min":1.361,"15min":1.361},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.4390000000000001,"5min":1.73,"15min":1.73},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.536,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.0309999999999999,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3580000000000001,"5min":1.671,"15min":1.671},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.544,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0289999999999999,"15min":1.071},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.387,"5min":1.798,"15min":1.798},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.1679999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.72,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.272,"15min":1.3140000000000001},"min":{"1min":0.77200000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.4790000000000001,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.389}]},{"osd":5,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.218,"15min":1.236},"min":{"1min":0.746,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.5549999999999999,"5min":2.9470000000000001,"15min":2.9470000000000001},"last":1.488},{"interface":"front","average":{"1min":1.133,"5min":1.272,"15min":1.306},"min":{"1min":0.65300000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.522,"5min":2.52,"15min":2.52},"last":1.0940000000000001}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.371,"15min":1.4039999999999999},"min":{"1min":0.89800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.663,"5min":2.992,"15min":2.992},"last":1.45},{"interface":"front","average":{"1min":1.204,"5min":1.3520000000000001,"15min":1.3939999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.631,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.335}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3480000000000001,"15min":1.3859999999999999},"min":{"1min":0.76700000000000002,"5min":0.621,"15min":0.621},"max":{"1min":1.5349999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.3700000000000001,"15min":1.4059999999999999},"min":{"1min":0.76800000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.619,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":1.375}]}]},{"osd":1,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1290240,"data_stored":897862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.98299999999999998,"15min":1.0680000000000001},"min":{"1min":0.441,"5min":0.371,"15min":0.371},"max":{"1min":1.409,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.86199999999999999,"15min":0.90800000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4890000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.339}]},{"osd":2,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.94899999999999995,"15min":1.0169999999999999},"min":{"1min":0.52700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.3440000000000001,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.163},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.321,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.01}]},{"osd":3,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.99099999999999999,"15min":1.038},"min":{"1min":0.48399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.605,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.111,"15min":1.204},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4830000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.1120000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.3859999999999999,"15min":1.486},"min":{"1min":0.78600000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.5469999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3009999999999999,"15min":1.365},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5469999999999999,"5min":2.8929999999999998,"15min":2.8929999999999998},"last":1.778}]},{"osd":5,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3540000000000001,"15min":1.4299999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.611,"5min":2.722,"15min":2.722},"last":1.621},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.4390000000000001},"min":{"1min":0.84099999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.5189999999999999,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.294}]},{"osd":6,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.427,"15min":1.5209999999999999},"min":{"1min":0.78600000000000003,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.625,"5min":2.577,"15min":2.577},"last":1.585},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4410000000000001},"min":{"1min":0.85199999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6930000000000001,"5min":2.794,"15min":2.794},"last":1.8620000000000001}]},{"osd":7,"last update":"Wed Jan 31 05:03:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4390000000000001,"15min":1.5269999999999999},"min":{"1min":0.95599999999999996,"5min":0.876,"15min":0.876},"max":{"1min":1.9870000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.397},{"interface":"front","average":{"1min":1.365,"5min":1.5089999999999999,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8460000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.71}]}]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5760000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.113,"15min":1.125},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.079}]},{"osd":1,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1419999999999999,"15min":1.1339999999999999},"min":{"1min":0.75600000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1439999999999999,"15min":1.155},"min":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5900000000000001,"5min":1.976,"15min":1.976},"last":1.2250000000000001}]},{"osd":2,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.179,"15min":1.1739999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.4530000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.84699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.6930000000000001,"5min":1.873,"15min":1.873},"last":1.4630000000000001}]},{"osd":3,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3660000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.6639999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3},{"interface":"front","average":{"1min":1.335,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.92700000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0899999999999999,"5min":2.0899999999999999,"15min":2.0899999999999999},"last":1.268}]},{"osd":4,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.264,"15min":1.256},"min":{"1min":0.871,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.448,"15min":1.472},"min":{"1min":0.755,"5min":0.628,"15min":0.628},"max":{"1min":1.8100000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.411}]},{"osd":5,"last update":"Wed Jan 31 05:03:47 2024","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4350000000000001,"15min":1.4510000000000001},"min":{"1min":0.96199999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.506,"5min":1.573,"15min":1.597},"min":{"1min":0.878,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.966,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.617}]},{"osd":7,"last update":"Wed Jan 31 05:03:48 2024","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5369999999999999,"15min":1.5740000000000001},"min":{"1min":1.0860000000000001,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1309999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.651,"15min":1.677},"min":{"1min":1.157,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":2.04,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.6719999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1290240,"data_stored":897862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.04,"15min":1.022},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.615,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.147},{"interface":"front","average":{"1min":1.101,"5min":1.115,"15min":1.117},"min":{"1min":0.63100000000000001,"5min":0.626,"15min":0.626},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.73699999999999999}]},{"osd":1,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1379999999999999,"15min":1.1439999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.732,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.002},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.194,"15min":1.2170000000000001},"min":{"1min":0.77600000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.77,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.069}]},{"osd":2,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.196,"15min":1.2190000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.724,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.399},{"interface":"front","average":{"1min":1.222,"5min":1.2769999999999999,"15min":1.3029999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6870000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.86399999999999999}]},{"osd":3,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.28,"15min":1.298},"min":{"1min":0.79800000000000004,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.6859999999999999,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.369,"15min":1.393},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.673,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.2110000000000001}]},{"osd":4,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.3700000000000001},"min":{"1min":0.94399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9530000000000001,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.94199999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8280000000000001,"5min":3.0960000000000001,"15min":3.0960000000000001},"last":1.3069999999999999}]},{"osd":6,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4470000000000001,"15min":1.417},"min":{"1min":1.0960000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0510000000000002,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.4930000000000001,"15min":1.4950000000000001},"min":{"1min":1.167,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.0009999999999999,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":1.484}]},{"osd":7,"last update":"Wed Jan 31 05:03:43 2024","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4850000000000001,"15min":1.4930000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9339999999999999,"5min":2.8370000000000002,"15min":2.8370000000000002},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.617,"15min":1.619},"min":{"1min":1.268,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2040000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.546}]}]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":307494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.84299999999999997,"15min":0.83699999999999997},"min":{"1min":0.371,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":3.77,"15min":3.77},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.99299999999999999,"15min":1.0329999999999999},"min":{"1min":0.504,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4299999999999999,"5min":3.282,"15min":3.282},"last":0.623}]},{"osd":2,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":1.0009999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.357,"5min":2.9860000000000002,"15min":2.9860000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82999999999999996,"15min":0.85799999999999998},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.296,"5min":1.774,"15min":1.774},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.028,"15min":1.054},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4430000000000001,"5min":3.6880000000000002,"15min":3.6880000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0820000000000001},"min":{"1min":0.63200000000000001,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5589999999999999,"5min":3.121,"15min":3.121},"last":0.71399999999999997}]},{"osd":4,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.2150000000000001,"15min":1.2170000000000001},"min":{"1min":0.81000000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5489999999999999,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.3899999999999999,"15min":1.4239999999999999},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6970000000000001,"5min":3.8500000000000001,"15min":3.8500000000000001},"last":1.0509999999999999}]},{"osd":5,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.4379999999999999,"15min":1.464},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.7709999999999999,"5min":5.75,"15min":5.75},"last":1.141},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.4099999999999999,"15min":1.456},"min":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5089999999999999,"5min":4.8799999999999999,"15min":4.8799999999999999},"last":0.95199999999999996}]},{"osd":6,"last update":"Wed Jan 31 05:03:41 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.323,"15min":1.319},"min":{"1min":0.746,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":7.415,"15min":7.415},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":1.7130000000000001,"5min":7.5650000000000004,"15min":7.5650000000000004},"last":0.97499999999999998}]},{"osd":7,"last update":"Wed Jan 31 05:03:44 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.3500000000000001,"15min":1.3799999999999999},"min":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.552,"5min":3.633,"15min":3.633},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.504,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.899,"5min":3.9169999999999998,"15min":3.9169999999999998},"last":1.087}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-31T05:04:43.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-31T05:04:43.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.418+0000 7f89b3200640 1 Processor -- start 2024-01-31T05:04:43.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.418+0000 7f89b3200640 1 -- start start 2024-01-31T05:04:43.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89b3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 0x7f89ac06d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89b3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89ac161020 0x7f89ac1614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89b3200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89ac0ceb40 con 0x7f89ac15d690 2024-01-31T05:04:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89b3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89ac15e7c0 con 0x7f89ac161020 2024-01-31T05:04:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89b3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac06dea0 con 0x7f89ac06d580 2024-01-31T05:04:43.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89abe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 0x7f89ac06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89abe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 0x7f89ac06d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.97:48108/0 (socket says 172.21.15.97:48108) 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89abe00640 1 -- 172.21.15.97:0/3602331479 learned_addr learned my addr 172.21.15.97:0/3602331479 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.420+0000 7f89ab400640 1 --1- 172.21.15.97:0/3602331479 >> v1:172.21.15.188:6789/0 conn(0x7f89ac15d690 0x7f89ac0ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.97:49294/0 (socket says 172.21.15.97:49294) 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.421+0000 7f89b1200640 1 --2- 172.21.15.97:0/3602331479 >> v2:172.21.15.188:3300/0 conn(0x7f89ac161020 0x7f89ac1614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.421+0000 7f89abe00640 1 -- 172.21.15.97:0/3602331479 >> v2:172.21.15.188:3300/0 conn(0x7f89ac161020 msgr2=0x7f89ac1614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.421+0000 7f89abe00640 1 --2- 172.21.15.97:0/3602331479 >> v2:172.21.15.188:3300/0 conn(0x7f89ac161020 0x7f89ac1614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.421+0000 7f89abe00640 1 -- 172.21.15.97:0/3602331479 >> v1:172.21.15.188:6789/0 conn(0x7f89ac15d690 legacy=0x7f89ac0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.421+0000 7f89aaa00640 1 -- 172.21.15.97:0/3602331479 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848450592 0 0) 0x7f89ac0ceb40 con 0x7f89ac15d690 2024-01-31T05:04:43.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.421+0000 7f89abe00640 1 -- 172.21.15.97:0/3602331479 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac15db40 con 0x7f89ac06d580 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.423+0000 7f89abe00640 1 --2- 172.21.15.97:0/3602331479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 0x7f89ac06d960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f899c009850 tx=0x7f899c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e131fde32d6245bf server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.423+0000 7f89aaa00640 1 -- 172.21.15.97:0/3602331479 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c004230 con 0x7f89ac06d580 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.423+0000 7f89aaa00640 1 -- 172.21.15.97:0/3602331479 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c004390 con 0x7f89ac06d580 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.423+0000 7f89aaa00640 1 -- 172.21.15.97:0/3602331479 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c003810 con 0x7f89ac06d580 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 -- 172.21.15.97:0/3602331479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 msgr2=0x7f89ac06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 --2- 172.21.15.97:0/3602331479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 0x7f89ac06d960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f899c009850 tx=0x7f899c02b960 comp rx=0 tx=0).stop 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 -- 172.21.15.97:0/3602331479 shutdown_connections 2024-01-31T05:04:43.426 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 --2- 172.21.15.97:0/3602331479 >> v2:172.21.15.188:3300/0 conn(0x7f89ac161020 0x7f89ac1614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 --2- 172.21.15.97:0/3602331479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac06d580 0x7f89ac06d960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 -- 172.21.15.97:0/3602331479 >> 172.21.15.97:0/3602331479 conn(0x7f89ac0b0a10 msgr2=0x7f89ac0b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 -- 172.21.15.97:0/3602331479 shutdown_connections 2024-01-31T05:04:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.424+0000 7f89b3200640 1 -- 172.21.15.97:0/3602331479 wait complete. 2024-01-31T05:04:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.426+0000 7f89b3200640 1 Processor -- start 2024-01-31T05:04:43.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.426+0000 7f89b3200640 1 -- start start 2024-01-31T05:04:43.429 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89b3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89ac06d580 0x7f89ac06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89b3200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 0x7f89ac09a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89b3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac161020 0x7f89ac09b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89b3200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac09a390 con 0x7f89ac15d690 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89b3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac09ad60 con 0x7f89ac06d580 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89b3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac1f4110 con 0x7f89ac161020 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89ab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 0x7f89ac09a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89ab400640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 0x7f89ac09a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59968/0 (socket says 172.21.15.97:59968) 2024-01-31T05:04:43.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.427+0000 7f89ab400640 1 -- 172.21.15.97:0/664589579 learned_addr learned my addr 172.21.15.97:0/664589579 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89b1200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac161020 0x7f89ac09b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89abe00640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89ac06d580 0x7f89ac06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89ab400640 1 -- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac161020 msgr2=0x7f89ac09b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89ab400640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac161020 0x7f89ac09b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89ab400640 1 -- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89ac06d580 msgr2=0x7f89ac06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89ab400640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89ac06d580 0x7f89ac06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89ab400640 1 -- 172.21.15.97:0/664589579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c009460 con 0x7f89ac15d690 2024-01-31T05:04:43.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89abe00640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89ac06d580 0x7f89ac06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89ab400640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 0x7f89ac09a820 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f899400c4b0 tx=0x7f899400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940074e0 con 0x7f89ac15d690 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8994007660 con 0x7f89ac15d690 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.428+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac099a60 con 0x7f89ac15d690 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.429+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994004a20 con 0x7f89ac15d690 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.429+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac09a020 con 0x7f89ac15d690 2024-01-31T05:04:43.432 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.430+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac09a540 con 0x7f89ac15d690 2024-01-31T05:04:43.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.431+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f8994004b80 con 0x7f89ac15d690 2024-01-31T05:04:43.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.431+0000 7f89a3e00640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898406af00 0x7f898406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.432+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f89940887c0 con 0x7f89ac15d690 2024-01-31T05:04:43.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.432+0000 7f89abe00640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898406af00 0x7f898406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.432+0000 7f89abe00640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898406af00 0x7f898406d3c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f899c035000 tx=0x7f899c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.436+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f89940537f0 con 0x7f89ac15d690 2024-01-31T05:04:43.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:43.443 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.633+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f89ac067830 con 0x7f89ac15d690 2024-01-31T05:04:43.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.639+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f89940180c0 con 0x7f89ac15d690 2024-01-31T05:04:43.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.640+0000 7f89a3e00640 1 -- 172.21.15.97:0/664589579 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f89940537f0 con 0x7f89ac15d690 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898406af00 msgr2=0x7f898406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898406af00 0x7f898406d3c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f899c035000 tx=0x7f899c008030 comp rx=0 tx=0).stop 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 msgr2=0x7f89ac09a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 0x7f89ac09a820 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f899400c4b0 tx=0x7f899400c970 comp rx=0 tx=0).stop 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 shutdown_connections 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89ac161020 0x7f89ac09b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f898406af00 0x7f898406d3c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89ac15d690 0x7f89ac09a820 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 --2- 172.21.15.97:0/664589579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89ac06d580 0x7f89ac06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 >> 172.21.15.97:0/664589579 conn(0x7f89ac0b0a10 msgr2=0x7f89ac0af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:43.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 shutdown_connections 2024-01-31T05:04:43.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.642+0000 7f89b3200640 1 -- 172.21.15.97:0/664589579 wait complete. 2024-01-31T05:04:43.661 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-31T05:04:43.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-31T05:04:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.871+0000 7f95cb200640 1 Processor -- start 2024-01-31T05:04:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.871+0000 7f95cb200640 1 -- start start 2024-01-31T05:04:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95cb200640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95c40ccd00 0x7f95c40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95cb200640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c406d1a0 con 0x7f95c40cb810 2024-01-31T05:04:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95cb200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c406c600 con 0x7f95c415b400 2024-01-31T05:04:43.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95cb200640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c40cc960 con 0x7f95c40ccd00 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95c3e00640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f95c40cb810 0x7f95c40cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:47648/0 (socket says 172.21.15.97:47648) 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95c3e00640 1 -- 172.21.15.97:0/2462939680 learned_addr learned my addr 172.21.15.97:0/2462939680 (peer_addr_for_me v1:172.21.15.97:0/0) 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95c9200640 1 --2- 172.21.15.97:0/2462939680 >> v2:172.21.15.97:3300/0 conn(0x7f95c40ccd00 0x7f95c40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95c3400640 1 -- 172.21.15.97:0/2462939680 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691109811 0 0) 0x7f95c406d1a0 con 0x7f95c40cb810 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.873+0000 7f95c3400640 1 -- 172.21.15.97:0/2462939680 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a4003520 con 0x7f95c40cb810 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.874+0000 7f95c9200640 1 -- 172.21.15.97:0/2462939680 >> v1:172.21.15.188:6789/0 conn(0x7f95c415b400 legacy=0x7f95c406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.874+0000 7f95c9200640 1 -- 172.21.15.97:0/2462939680 >> v1:172.21.15.97:6789/0 conn(0x7f95c40cb810 legacy=0x7f95c40cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.874+0000 7f95c3400640 1 -- 172.21.15.97:0/2462939680 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1317348030 0 0) 0x7f95a4003520 con 0x7f95c40cb810 2024-01-31T05:04:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.874+0000 7f95c9200640 1 -- 172.21.15.97:0/2462939680 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c40cd6d0 con 0x7f95c40ccd00 2024-01-31T05:04:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.875+0000 7f95c9200640 1 --2- 172.21.15.97:0/2462939680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40ccd00 0x7f95c40cd190 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009d90 tx=0x7f95bc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf1036d326e72ae server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.875+0000 7f95c3400640 1 -- 172.21.15.97:0/2462939680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4001670 con 0x7f95c40ccd00 2024-01-31T05:04:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.875+0000 7f95c3400640 1 -- 172.21.15.97:0/2462939680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f95bc02c940 con 0x7f95c40ccd00 2024-01-31T05:04:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.875+0000 7f95c3400640 1 -- 172.21.15.97:0/2462939680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc02ccb0 con 0x7f95c40ccd00 2024-01-31T05:04:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 -- 172.21.15.97:0/2462939680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40ccd00 msgr2=0x7f95c40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 --2- 172.21.15.97:0/2462939680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40ccd00 0x7f95c40cd190 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009d90 tx=0x7f95bc02be10 comp rx=0 tx=0).stop 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 -- 172.21.15.97:0/2462939680 shutdown_connections 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 --2- 172.21.15.97:0/2462939680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40ccd00 0x7f95c40cd190 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 -- 172.21.15.97:0/2462939680 >> 172.21.15.97:0/2462939680 conn(0x7f95c40b0a10 msgr2=0x7f95c40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 -- 172.21.15.97:0/2462939680 shutdown_connections 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.876+0000 7f95cb200640 1 -- 172.21.15.97:0/2462939680 wait complete. 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.878+0000 7f95cb200640 1 Processor -- start 2024-01-31T05:04:43.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.878+0000 7f95cb200640 1 -- start start 2024-01-31T05:04:43.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95cb200640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 0x7f95c415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95cb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95c40ccd00 0x7f95c41f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95cb200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95c415b400 0x7f95c41f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95cb200640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c40cee80 con 0x7f95c40cb810 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95cb200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c41f29c0 con 0x7f95c415b400 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95cb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c40ce270 con 0x7f95c40ccd00 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 0x7f95c415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 0x7f95c415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59980/0 (socket says 172.21.15.97:59980) 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 -- 172.21.15.97:0/1926891591 learned_addr learned my addr 172.21.15.97:0/1926891591 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 -- 172.21.15.97:0/1926891591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95c40ccd00 msgr2=0x7f95c41f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95c40ccd00 0x7f95c41f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 -- 172.21.15.97:0/1926891591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95c415b400 msgr2=0x7f95c41f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95c415b400 0x7f95c41f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.879+0000 7f95c3e00640 1 -- 172.21.15.97:0/1926891591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc009a70 con 0x7f95c40cb810 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.880+0000 7f95c3e00640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 0x7f95c415b810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c760 tx=0x7f95b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.880+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4005ad0 con 0x7f95c40cb810 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.880+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f95b4005c30 con 0x7f95c40cb810 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.880+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4007070 con 0x7f95c40cb810 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.880+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c40ce550 con 0x7f95c40cb810 2024-01-31T05:04:43.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.880+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c40ceb10 con 0x7f95c40cb810 2024-01-31T05:04:43.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.881+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c415bd50 con 0x7f95c40cb810 2024-01-31T05:04:43.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.882+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89206+0+0 (secure 0 0 0) 0x7f95b40071d0 con 0x7f95c40cb810 2024-01-31T05:04:43.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.883+0000 7f95c0c00640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f959c06aed0 0x7f959c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-31T05:04:43.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.883+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f95b40892b0 con 0x7f95c40cb810 2024-01-31T05:04:43.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.883+0000 7f95bb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f959c06aed0 0x7f959c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-31T05:04:43.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.884+0000 7f95bb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f959c06aed0 0x7f959c06d390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f95ac000f60 tx=0x7f95ac0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-31T05:04:43.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:43.887+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180608 (secure 0 0 0) 0x7f95b40542b0 con 0x7f95c40cb810 2024-01-31T05:04:44.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.120+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95c4071c70 con 0x7f95c40cb810 2024-01-31T05:04:44.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.121+0000 7f95c0c00640 1 -- 172.21.15.97:0/1926891591 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f95b40578d0 con 0x7f95c40cb810 2024-01-31T05:04:44.121 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-01-31T05:04:44.121 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/1785584262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1785584262},{"type":"v1","addr":"172.21.15.188:6833","nonce":1785584262}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5197,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/3953130259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":3953130259},{"type":"v1","addr":"172.21.15.188:6835","nonce":3953130259}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6234,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1130605051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1130605051},{"type":"v1","addr":"172.21.15.97:6835","nonce":1130605051}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/665459995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":665459995},{"type":"v1","addr":"172.21.15.97:6837","nonce":665459995}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-01-31T05:04:44.122 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 29 2024-01-31T05:04:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.124+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f959c06aed0 msgr2=0x7f959c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.124+0000 7f95cb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f959c06aed0 0x7f959c06d390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f95ac000f60 tx=0x7f95ac0079b0 comp rx=0 tx=0).stop 2024-01-31T05:04:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.124+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 msgr2=0x7f95c415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-31T05:04:44.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.124+0000 7f95cb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 0x7f95c415b810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c760 tx=0x7f95b400cc20 comp rx=0 tx=0).stop 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 shutdown_connections 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95c415b400 0x7f95c41f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95c40ccd00 0x7f95c41f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:6832/631419576,v1:172.21.15.97:6833/631419576] conn(0x7f959c06aed0 0x7f959c06d390 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 --2- 172.21.15.97:0/1926891591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95c40cb810 0x7f95c415b810 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 >> 172.21.15.97:0/1926891591 conn(0x7f95c40b0a10 msgr2=0x7f95c40abe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 shutdown_connections 2024-01-31T05:04:44.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-01-31T05:04:44.125+0000 7f95cb200640 1 -- 172.21.15.97:0/1926891591 wait complete. 2024-01-31T05:04:44.144 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-31T05:04:44.145 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-31T05:04:44.145 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-31T05:04:44.145 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.166 INFO:tasks.ceph.mds.a:Stopped 2024-01-31T05:04:44.166 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-31T05:04:44.166 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.231 INFO:tasks.ceph.mds.c:Stopped 2024-01-31T05:04:44.232 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-31T05:04:44.232 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.254 INFO:tasks.ceph.mds.b:Stopped 2024-01-31T05:04:44.255 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-31T05:04:44.255 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.317 INFO:tasks.ceph.mds.d:Stopped 2024-01-31T05:04:44.317 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-31T05:04:44.317 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-31T05:04:44.317 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.376 INFO:tasks.ceph.osd.0:Stopped 2024-01-31T05:04:44.376 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-31T05:04:44.376 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.427 INFO:tasks.ceph.osd.1:Stopped 2024-01-31T05:04:44.428 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-31T05:04:44.428 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.492 INFO:tasks.ceph.osd.2:Stopped 2024-01-31T05:04:44.493 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-31T05:04:44.493 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.559 INFO:tasks.ceph.osd.3:Stopped 2024-01-31T05:04:44.559 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-31T05:04:44.559 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.633 INFO:tasks.ceph.osd.4:Stopped 2024-01-31T05:04:44.633 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-31T05:04:44.633 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.713 INFO:tasks.ceph.osd.5:Stopped 2024-01-31T05:04:44.714 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-31T05:04:44.714 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.767 INFO:tasks.ceph.osd.6:Stopped 2024-01-31T05:04:44.768 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-31T05:04:44.768 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.841 INFO:tasks.ceph.osd.7:Stopped 2024-01-31T05:04:44.842 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-31T05:04:44.842 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-31T05:04:44.842 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:44.925 INFO:tasks.ceph.mgr.y:Stopped 2024-01-31T05:04:44.925 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-31T05:04:44.925 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:45.018 INFO:tasks.ceph.mgr.x:Stopped 2024-01-31T05:04:45.019 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-31T05:04:45.019 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-31T05:04:45.019 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:45.069 INFO:tasks.ceph.mon.a:Stopped 2024-01-31T05:04:45.070 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-31T05:04:45.070 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:45.094 INFO:tasks.ceph.mon.b:Stopped 2024-01-31T05:04:45.094 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-31T05:04:45.095 INFO:teuthology.orchestra.run:waiting for 300 2024-01-31T05:04:45.159 INFO:tasks.ceph.mon.c:Stopped 2024-01-31T05:04:45.160 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-31T05:04:45.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-31T05:04:45.262 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:45.263 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-31T05:04:45.518 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:45.519 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-31T05:04:45.669 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:45.669 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-31T05:04:45.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:45.809 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-31T05:04:45.967 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:45.967 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-31T05:04:46.230 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:46.231 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-31T05:04:46.397 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:46.397 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-31T05:04:46.556 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:46.557 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-31T05:04:46.713 INFO:tasks.ceph:Archiving mon data... 2024-01-31T05:04:46.714 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/data/mon.a.tgz 2024-01-31T05:04:46.714 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-01-31T05:04:46.771 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.MeLGXD4Jum 2024-01-31T05:04:46.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MeLGXD4Jum 2024-01-31T05:04:46.979 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.MeLGXD4Jum 2024-01-31T05:04:47.129 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.MeLGXD4Jum is 373KB 2024-01-31T05:04:47.236 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.MeLGXD4Jum 2024-01-31T05:04:47.293 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/data/mon.b.tgz 2024-01-31T05:04:47.293 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-01-31T05:04:47.348 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.u2bq6pgGKJ 2024-01-31T05:04:47.349 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.u2bq6pgGKJ 2024-01-31T05:04:47.565 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.u2bq6pgGKJ 2024-01-31T05:04:47.707 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.u2bq6pgGKJ is 491KB 2024-01-31T05:04:47.813 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.u2bq6pgGKJ 2024-01-31T05:04:47.866 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/data/mon.c.tgz 2024-01-31T05:04:47.867 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-01-31T05:04:47.914 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.w7oQ6BBq8Q 2024-01-31T05:04:47.914 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.w7oQ6BBq8Q 2024-01-31T05:04:48.114 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.w7oQ6BBq8Q 2024-01-31T05:04:48.266 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.w7oQ6BBq8Q is 491KB 2024-01-31T05:04:48.372 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.w7oQ6BBq8Q 2024-01-31T05:04:48.427 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-31T05:04:48.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-31T05:04:48.435 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-31T05:04:48.579 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-31T05:04:48.636 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-31T05:04:48.636 INFO:tasks.ceph:Archiving crash dumps... 2024-01-31T05:04:48.638 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/remote/smithi097/crash 2024-01-31T05:04:48.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-31T05:04:48.736 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/remote/smithi188/crash 2024-01-31T05:04:48.737 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-31T05:04:48.830 INFO:tasks.ceph:Shutting down logrotate 2024-01-31T05:04:48.830 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:48.838 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:48.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:49.061 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-31T05:04:49.158 INFO:tasks.ceph:Compressing logs... 2024-01-31T05:04:49.158 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-31T05:04:49.166 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-31T05:04:49.253 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54394.log 2024-01-31T05:04:49.253 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36911.log 2024-01-31T05:04:49.254 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54494.log 2024-01-31T05:04:49.255 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-31T05:04:49.256 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-01-31T05:04:49.256 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54394.log.gz 2024-01-31T05:04:49.257 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36831.log 2024-01-31T05:04:49.258 INFO:teuthology.orchestra.run.smithi188.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36911.log.gz 2024-01-31T05:04:49.258 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-31T05:04:49.259 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57524.log 2024-01-31T05:04:49.259 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54494.log.gz 2024-01-31T05:04:49.259 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-01-31T05:04:49.259 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-01-31T05:04:49.259 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57524.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57524.log.gz 2024-01-31T05:04:49.260 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61197.log 2024-01-31T05:04:49.261 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42317.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-01-31T05:04:49.261 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-31T05:04:49.262 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57346.log 2024-01-31T05:04:49.262 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61197.log.gz 2024-01-31T05:04:49.262 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36831.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-31T05:04:49.263 INFO:teuthology.orchestra.run.smithi188.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36831.log.gz 2024-01-31T05:04:49.263 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36711.log 2024-01-31T05:04:49.263 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-31T05:04:49.263 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36711.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36711.log.gz 2024-01-31T05:04:49.263 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36951.log 2024-01-31T05:04:49.263 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-31T05:04:49.264 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-01-31T05:04:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49131.log 2024-01-31T05:04:49.265 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57346.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57346.log.gz 2024-01-31T05:04:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61239.log 2024-01-31T05:04:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54824.log 2024-01-31T05:04:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61239.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49131.log.gz 2024-01-31T05:04:49.266 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61239.log.gz 2024-01-31T05:04:49.266 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52924.log 2024-01-31T05:04:49.268 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48701.log 2024-01-31T05:04:49.268 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54824.log.gz 2024-01-31T05:04:49.269 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.36951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41945.log 2024-01-31T05:04:49.269 INFO:teuthology.orchestra.run.smithi188.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36951.log.gz 2024-01-31T05:04:49.269 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36871.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz -5 2024-01-31T05:04:49.270 INFO:teuthology.orchestra.run.smithi188.stderr: --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-31T05:04:49.270 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.41945.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-31T05:04:49.270 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: 91.5% -- replaced with /var/log/ceph/ceph-client.1.41945.log.gz 2024-01-31T05:04:49.270 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-31T05:04:49.271 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36671.log 2024-01-31T05:04:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-01-31T05:04:49.272 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52924.log.gz 2024-01-31T05:04:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48701.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48701.log.gz 2024-01-31T05:04:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60987.log 2024-01-31T05:04:49.272 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-01-31T05:04:49.273 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-01-31T05:04:49.273 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 2024-01-31T05:04:49.273 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60987.log.gz 2024-01-31T05:04:49.274 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-31T05:04:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42401.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-01-31T05:04:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56161.log 2024-01-31T05:04:49.275 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56751.log 2024-01-31T05:04:49.275 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz 2024-01-31T05:04:49.277 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36671.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36671.log.gz 2024-01-31T05:04:49.277 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36751.log 2024-01-31T05:04:49.278 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56161.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56161.log.gz 2024-01-31T05:04:49.278 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43904.log 2024-01-31T05:04:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51682.log 2024-01-31T05:04:49.279 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56751.log.gz 2024-01-31T05:04:49.279 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62252.log 2024-01-31T05:04:49.279 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43904.log.gz 2024-01-31T05:04:49.282 INFO:teuthology.orchestra.run.smithi188.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-31T05:04:49.282 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-31T05:04:49.282 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36751.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-31T05:04:49.282 INFO:teuthology.orchestra.run.smithi188.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36751.log.gz 2024-01-31T05:04:49.282 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42465.log 2024-01-31T05:04:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51682.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51682.log.gz 2024-01-31T05:04:49.283 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62000.log 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57976.log 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62252.log.gz 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55434.log 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62000.log.gz 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57976.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57976.log.gz 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2024-01-31T05:04:49.284 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61281.log 2024-01-31T05:04:49.285 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-31T05:04:49.286 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.42465.log: /var/log/ceph/ceph-mds.d.log: 92.2% -- replaced with /var/log/ceph/ceph-client.3.42465.log.gz 2024-01-31T05:04:49.286 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55434.log.gz 2024-01-31T05:04:49.286 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50019.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.50181.log -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2024-01-31T05:04:49.287 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T05:04:49.287 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48801.log 2024-01-31T05:04:49.287 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61281.log.gz 2024-01-31T05:04:49.289 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36791.log 2024-01-31T05:04:49.289 INFO:teuthology.orchestra.run.smithi188.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-31T05:04:49.290 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-31T05:04:49.290 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44202.log 2024-01-31T05:04:49.291 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50181.log.gz 2024-01-31T05:04:49.291 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48801.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48801.log.gz 2024-01-31T05:04:49.291 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49893.log 2024-01-31T05:04:49.291 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2024-01-31T05:04:49.291 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44202.log.gz 2024-01-31T05:04:49.292 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.36791.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36791.log.gz 2024-01-31T05:04:49.292 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53514.log 2024-01-31T05:04:49.292 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49893.log.gz 2024-01-31T05:04:49.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49709.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2024-01-31T05:04:49.293 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2024-01-31T05:04:49.293 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53682.log 2024-01-31T05:04:49.294 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53514.log.gz 2024-01-31T05:04:49.294 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43862.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz /var/log/ceph/ceph-client.admin.48533.log 2024-01-31T05:04:49.294 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T05:04:49.295 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55392.log 2024-01-31T05:04:49.296 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53682.log.gz 2024-01-31T05:04:49.297 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48533.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48533.log.gz 2024-01-31T05:04:49.297 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53414.log 2024-01-31T05:04:49.298 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45032.log 2024-01-31T05:04:49.298 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55392.log.gz 2024-01-31T05:04:49.299 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62378.log 2024-01-31T05:04:49.299 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53414.log.gz 2024-01-31T05:04:49.300 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49089.log 2024-01-31T05:04:49.300 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45032.log.gz 2024-01-31T05:04:49.301 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.c.log: 90.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-31T05:04:49.302 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62378.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62378.log.gz 2024-01-31T05:04:49.302 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2024-01-31T05:04:49.303 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2024-01-31T05:04:49.303 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49089.log.gz 2024-01-31T05:04:49.304 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53372.log 2024-01-31T05:04:49.304 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz 2024-01-31T05:04:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44990.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2024-01-31T05:04:49.306 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54782.log 2024-01-31T05:04:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53372.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53372.log.gz 2024-01-31T05:04:49.307 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-01-31T05:04:49.308 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-01-31T05:04:49.308 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54782.log.gz 2024-01-31T05:04:49.309 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53288.log 2024-01-31T05:04:49.309 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-01-31T05:04:49.310 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52148.log 2024-01-31T05:04:49.310 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2024-01-31T05:04:49.311 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59890.log 2024-01-31T05:04:49.311 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53288.log.gz 2024-01-31T05:04:49.312 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60582.log 2024-01-31T05:04:49.312 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52148.log.gz 2024-01-31T05:04:49.314 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44157.log 2024-01-31T05:04:49.314 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59890.log.gz 2024-01-31T05:04:49.315 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60582.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60582.log.gz 2024-01-31T05:04:49.315 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2024-01-31T05:04:49.316 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-01-31T05:04:49.316 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44157.log.gz 2024-01-31T05:04:49.317 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48575.log 2024-01-31T05:04:49.317 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2024-01-31T05:04:49.319 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57568.log 2024-01-31T05:04:49.319 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-01-31T05:04:49.320 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48575.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48575.log.gz 2024-01-31T05:04:49.320 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57766.log 2024-01-31T05:04:49.321 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55350.log 2024-01-31T05:04:49.321 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57568.log.gz 2024-01-31T05:04:49.322 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53070.log 2024-01-31T05:04:49.322 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57766.log.gz 2024-01-31T05:04:49.323 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42443.log 2024-01-31T05:04:49.323 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55350.log.gz 2024-01-31T05:04:49.325 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61748.log 2024-01-31T05:04:49.325 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53070.log.gz 2024-01-31T05:04:49.326 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57482.log 2024-01-31T05:04:49.326 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42443.log.gz 2024-01-31T05:04:49.327 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48312.log 2024-01-31T05:04:49.327 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61748.log.gz 2024-01-31T05:04:49.329 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57482.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57482.log.gz 2024-01-31T05:04:49.329 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61468.log 2024-01-31T05:04:49.330 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57218.log 2024-01-31T05:04:49.330 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48312.log.gz 2024-01-31T05:04:49.331 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60903.log 2024-01-31T05:04:49.331 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61468.log.gz 2024-01-31T05:04:49.332 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52274.log 2024-01-31T05:04:49.332 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57218.log.gz 2024-01-31T05:04:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60903.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60903.log.gz 2024-01-31T05:04:49.333 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61874.log 2024-01-31T05:04:49.334 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44382.log 2024-01-31T05:04:49.335 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52274.log.gz 2024-01-31T05:04:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2024-01-31T05:04:49.336 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61874.log.gz 2024-01-31T05:04:49.336 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45950.log 2024-01-31T05:04:49.337 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44382.log.gz 2024-01-31T05:04:49.338 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42551.log 2024-01-31T05:04:49.338 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-01-31T05:04:49.339 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47176.log 2024-01-31T05:04:49.339 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45950.log.gz 2024-01-31T05:04:49.340 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48927.log 2024-01-31T05:04:49.340 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42551.log.gz 2024-01-31T05:04:49.341 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-01-31T05:04:49.342 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47176.log.gz 2024-01-31T05:04:49.343 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58969.log 2024-01-31T05:04:49.343 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48927.log.gz 2024-01-31T05:04:49.344 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46120.log 2024-01-31T05:04:49.344 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-01-31T05:04:49.345 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52106.log 2024-01-31T05:04:49.345 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58969.log.gz 2024-01-31T05:04:49.346 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56709.log 2024-01-31T05:04:49.347 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46120.log.gz 2024-01-31T05:04:49.348 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52106.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52106.log.gz 2024-01-31T05:04:49.348 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62126.log 2024-01-31T05:04:49.349 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57850.log 2024-01-31T05:04:49.349 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56709.log.gz 2024-01-31T05:04:49.350 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47892.log 2024-01-31T05:04:49.350 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62126.log.gz 2024-01-31T05:04:49.351 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59522.log 2024-01-31T05:04:49.351 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57850.log.gz 2024-01-31T05:04:49.352 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53640.log 2024-01-31T05:04:49.353 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47892.log.gz 2024-01-31T05:04:49.354 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-01-31T05:04:49.354 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59522.log.gz 2024-01-31T05:04:49.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58126.log 2024-01-31T05:04:49.355 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53640.log.gz 2024-01-31T05:04:49.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51598.log 2024-01-31T05:04:49.356 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-01-31T05:04:49.357 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58126.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58126.log.gz 2024-01-31T05:04:49.357 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51766.log 2024-01-31T05:04:49.358 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57440.log 2024-01-31T05:04:49.359 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51598.log.gz 2024-01-31T05:04:49.359 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50787.log 2024-01-31T05:04:49.360 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51766.log.gz 2024-01-31T05:04:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42359.log 2024-01-31T05:04:49.361 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57440.log.gz 2024-01-31T05:04:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50787.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.61341.log 2024-01-31T05:04:49.362 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50787.log.gz 2024-01-31T05:04:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54662.log 2024-01-31T05:04:49.363 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42359.log.gz 2024-01-31T05:04:49.364 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51039.log 2024-01-31T05:04:49.364 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61341.log.gz 2024-01-31T05:04:49.365 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44736.log 2024-01-31T05:04:49.366 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54662.log.gz 2024-01-31T05:04:49.367 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51039.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.51039.log.gz -5 2024-01-31T05:04:49.367 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2024-01-31T05:04:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52190.log 2024-01-31T05:04:49.368 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44736.log.gz 2024-01-31T05:04:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41602.log 2024-01-31T05:04:49.369 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2024-01-31T05:04:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52400.log 2024-01-31T05:04:49.370 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52190.log.gz 2024-01-31T05:04:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41602.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41602.log.gz 2024-01-31T05:04:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49257.log 2024-01-31T05:04:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2024-01-31T05:04:49.373 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52400.log.gz 2024-01-31T05:04:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59848.log 2024-01-31T05:04:49.374 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49257.log.gz 2024-01-31T05:04:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58743.log 2024-01-31T05:04:49.375 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2024-01-31T05:04:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47134.log 2024-01-31T05:04:49.377 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59848.log.gz 2024-01-31T05:04:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2024-01-31T05:04:49.378 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58743.log.gz 2024-01-31T05:04:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47134.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47134.log.gz 2024-01-31T05:04:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45488.log 2024-01-31T05:04:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2024-01-31T05:04:49.380 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-01-31T05:04:49.381 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56119.log 2024-01-31T05:04:49.381 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45488.log.gz 2024-01-31T05:04:49.382 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56793.log 2024-01-31T05:04:49.383 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2024-01-31T05:04:49.384 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56331.log 2024-01-31T05:04:49.384 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56119.log.gz 2024-01-31T05:04:49.385 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50871.log 2024-01-31T05:04:49.385 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56793.log.gz 2024-01-31T05:04:49.386 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59706.log 2024-01-31T05:04:49.386 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56331.log.gz 2024-01-31T05:04:49.387 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58018.log 2024-01-31T05:04:49.387 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50871.log.gz 2024-01-31T05:04:49.388 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45908.log 2024-01-31T05:04:49.389 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59706.log.gz 2024-01-31T05:04:49.389 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58018.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58018.log.gz 2024-01-31T05:04:49.390 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48270.log 2024-01-31T05:04:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54620.log 2024-01-31T05:04:49.391 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45908.log.gz 2024-01-31T05:04:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51556.log 2024-01-31T05:04:49.395 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48270.log.gz 2024-01-31T05:04:49.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54620.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-31T05:04:49.397 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54620.log.gz 2024-01-31T05:04:49.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51556.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51556.log.gz 2024-01-31T05:04:49.397 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60861.log 2024-01-31T05:04:49.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48396.log 2024-01-31T05:04:49.397 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-01-31T05:04:49.398 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60861.log.gz 2024-01-31T05:04:49.398 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-31T05:04:49.398 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48102.log 2024-01-31T05:04:49.399 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48396.log.gz 2024-01-31T05:04:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49793.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-01-31T05:04:49.399 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51936.log 2024-01-31T05:04:49.400 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51980.log 2024-01-31T05:04:49.401 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48102.log.gz 2024-01-31T05:04:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58060.log 2024-01-31T05:04:49.402 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51936.log.gz 2024-01-31T05:04:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-01-31T05:04:49.403 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51980.log.gz 2024-01-31T05:04:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58060.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30306.log 2024-01-31T05:04:49.404 INFO:teuthology.orchestra.run.smithi097.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58060.log.gz 2024-01-31T05:04:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48144.log 2024-01-31T05:04:49.405 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-01-31T05:04:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.30306.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30306.log.gz 2024-01-31T05:04:49.406 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-01-31T05:04:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48144.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48144.log.gz 2024-01-31T05:04:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56373.log 2024-01-31T05:04:49.409 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44401.log 2024-01-31T05:04:49.409 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-01-31T05:04:49.410 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53330.log 2024-01-31T05:04:49.410 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56373.log.gz 2024-01-31T05:04:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44401.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44401.log.gz 2024-01-31T05:04:49.411 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45824.log 2024-01-31T05:04:49.412 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51390.log 2024-01-31T05:04:49.413 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53330.log.gz 2024-01-31T05:04:49.413 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52442.log 2024-01-31T05:04:49.414 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45824.log.gz 2024-01-31T05:04:49.415 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-01-31T05:04:49.415 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51390.log.gz 2024-01-31T05:04:49.416 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60070.log 2024-01-31T05:04:49.417 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45404.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52442.log.gz 2024-01-31T05:04:49.417 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-01-31T05:04:49.417 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53598.log 2024-01-31T05:04:49.418 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46525.log 2024-01-31T05:04:49.418 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60070.log.gz 2024-01-31T05:04:49.419 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60624.log 2024-01-31T05:04:49.420 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53598.log.gz 2024-01-31T05:04:49.421 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.46525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47934.log 2024-01-31T05:04:49.422 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60624.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.46525.log.gz 2024-01-31T05:04:49.422 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60624.log.gz 2024-01-31T05:04:49.422 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47808.log 2024-01-31T05:04:49.423 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-01-31T05:04:49.423 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47934.log.gz 2024-01-31T05:04:49.424 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61384.log 2024-01-31T05:04:49.424 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47808.log.gz 2024-01-31T05:04:49.425 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52358.log 2024-01-31T05:04:49.426 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-01-31T05:04:49.426 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61384.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.61384.log.gz --verbose 2024-01-31T05:04:49.426 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.56289.log 2024-01-31T05:04:49.427 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2024-01-31T05:04:49.428 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52358.log.gz 2024-01-31T05:04:49.428 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36322.log 2024-01-31T05:04:49.429 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56289.log.gz 2024-01-31T05:04:49.430 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51310.log 2024-01-31T05:04:49.430 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2024-01-31T05:04:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61664.log 2024-01-31T05:04:49.431 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36322.log.gz 2024-01-31T05:04:49.431 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51808.log 2024-01-31T05:04:49.432 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51310.log.gz 2024-01-31T05:04:49.433 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62042.log 2024-01-31T05:04:49.433 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61664.log.gz 2024-01-31T05:04:49.434 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-01-31T05:04:49.434 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51808.log.gz 2024-01-31T05:04:49.435 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62042.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62042.log.gz 2024-01-31T05:04:49.435 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53556.log 2024-01-31T05:04:49.436 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56972.log 2024-01-31T05:04:49.436 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-01-31T05:04:49.437 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60666.log 2024-01-31T05:04:49.437 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53556.log.gz 2024-01-31T05:04:49.438 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56972.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56972.log.gz 2024-01-31T05:04:49.438 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57656.log 2024-01-31T05:04:49.439 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54436.log 2024-01-31T05:04:49.440 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60666.log.gz 2024-01-31T05:04:49.440 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42009.log 2024-01-31T05:04:49.441 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57656.log.gz 2024-01-31T05:04:49.441 INFO:teuthology.orchestra.run.smithi188.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-31T05:04:49.442 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62420.log 2024-01-31T05:04:49.442 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54436.log.gz 2024-01-31T05:04:49.443 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62462.log 2024-01-31T05:04:49.443 INFO:teuthology.orchestra.run.smithi188.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-31T05:04:49.444 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42009.log.gz 2024-01-31T05:04:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2024-01-31T05:04:49.444 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62420.log.gz 2024-01-31T05:04:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61155.log 2024-01-31T05:04:49.445 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62462.log.gz 2024-01-31T05:04:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2024-01-31T05:04:49.446 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2024-01-31T05:04:49.447 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53246.log 2024-01-31T05:04:49.447 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61155.log.gz 2024-01-31T05:04:49.448 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41915.log 2024-01-31T05:04:49.449 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2024-01-31T05:04:49.449 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-01-31T05:04:49.449 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53246.log.gz 2024-01-31T05:04:49.450 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41915.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41915.log.gz 2024-01-31T05:04:49.450 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44404.log 2024-01-31T05:04:49.451 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57611.log 2024-01-31T05:04:49.452 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44404.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-01-31T05:04:49.452 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44404.log.gz 2024-01-31T05:04:49.452 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2024-01-31T05:04:49.453 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43333.log 2024-01-31T05:04:49.454 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57611.log.gz 2024-01-31T05:04:49.455 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58701.log 2024-01-31T05:04:49.455 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2024-01-31T05:04:49.456 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43333.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-osd.1.log 2024-01-31T05:04:49.456 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43333.log.gz 2024-01-31T05:04:49.456 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50349.log 2024-01-31T05:04:49.457 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58701.log.gz 2024-01-31T05:04:49.458 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47412.log 2024-01-31T05:04:49.459 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-01-31T05:04:49.459 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50349.log.gz 2024-01-31T05:04:49.461 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47412.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47412.log.gz 2024-01-31T05:04:49.461 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51724.log 2024-01-31T05:04:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53028.log 2024-01-31T05:04:49.462 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-01-31T05:04:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51724.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51724.log.gz 2024-01-31T05:04:49.464 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62294.log 2024-01-31T05:04:49.465 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55702.log 2024-01-31T05:04:49.465 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53028.log.gz 2024-01-31T05:04:49.466 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62294.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62294.log.gz 2024-01-31T05:04:49.467 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44738.log 2024-01-31T05:04:49.468 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59806.log 2024-01-31T05:04:49.468 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55702.log.gz 2024-01-31T05:04:49.469 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44730.log 2024-01-31T05:04:49.469 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44738.log.gz 2024-01-31T05:04:49.470 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59806.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.50661.log -- replaced with /var/log/ceph/ceph-client.admin.59806.log.gz 2024-01-31T05:04:49.470 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T05:04:49.471 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50997.log 2024-01-31T05:04:49.471 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44730.log.gz 2024-01-31T05:04:49.472 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50661.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50661.log.gz 2024-01-31T05:04:49.472 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62504.log 2024-01-31T05:04:49.473 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45078.log 2024-01-31T05:04:49.473 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50997.log.gz 2024-01-31T05:04:49.474 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57260.log 2024-01-31T05:04:49.474 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62504.log.gz 2024-01-31T05:04:49.475 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50913.log 2024-01-31T05:04:49.476 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45078.log.gz 2024-01-31T05:04:49.476 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57260.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.48885.log -- replaced with /var/log/ceph/ceph-client.admin.57260.log.gz 2024-01-31T05:04:49.477 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T05:04:49.478 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45530.log 2024-01-31T05:04:49.478 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50913.log.gz 2024-01-31T05:04:49.479 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2024-01-31T05:04:49.479 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48885.log.gz 2024-01-31T05:04:49.480 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43946.log 2024-01-31T05:04:49.480 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45530.log.gz 2024-01-31T05:04:49.481 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55476.log 2024-01-31T05:04:49.482 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz 2024-01-31T05:04:49.482 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45362.log 2024-01-31T05:04:49.483 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43946.log.gz 2024-01-31T05:04:49.484 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55476.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55476.log.gz 2024-01-31T05:04:49.484 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47766.log 2024-01-31T05:04:49.485 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62336.log 2024-01-31T05:04:49.485 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45362.log.gz 2024-01-31T05:04:49.486 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60503.log 2024-01-31T05:04:49.486 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47766.log.gz 2024-01-31T05:04:49.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54352.log 2024-01-31T05:04:49.487 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62336.log.gz 2024-01-31T05:04:49.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58885.log 2024-01-31T05:04:49.488 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60503.log.gz 2024-01-31T05:04:49.489 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-01-31T05:04:49.489 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54352.log.gz 2024-01-31T05:04:49.490 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57390.log 88.8% 2024-01-31T05:04:49.491 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58885.log.gz 2024-01-31T05:04:49.492 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57723.log 2024-01-31T05:04:49.492 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-01-31T05:04:49.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2024-01-31T05:04:49.493 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57390.log.gz 2024-01-31T05:04:49.494 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2024-01-31T05:04:49.494 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57723.log.gz 2024-01-31T05:04:49.495 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48060.log 2024-01-31T05:04:49.495 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2024-01-31T05:04:49.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45043.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gzgzip 2024-01-31T05:04:49.496 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43375.log 2024-01-31T05:04:49.497 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46637.log 2024-01-31T05:04:49.498 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48060.log.gz 2024-01-31T05:04:49.499 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43375.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43375.log.gz 2024-01-31T05:04:49.499 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44746.log 2024-01-31T05:04:49.500 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.46637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49935.log 2024-01-31T05:04:49.500 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46637.log.gz 2024-01-31T05:04:49.501 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49667.log 2024-01-31T05:04:49.501 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44746.log.gz 2024-01-31T05:04:49.502 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51081.log 2024-01-31T05:04:49.502 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49935.log.gz 2024-01-31T05:04:49.503 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49173.log 2024-01-31T05:04:49.503 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2024-01-31T05:04:49.504 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49299.log 2024-01-31T05:04:49.504 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51081.log.gz 2024-01-31T05:04:49.505 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54536.log 2024-01-31T05:04:49.506 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49173.log.gz 2024-01-31T05:04:49.507 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59974.log 2024-01-31T05:04:49.507 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49299.log.gz 2024-01-31T05:04:49.508 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54536.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54536.log.gz 2024-01-31T05:04:49.508 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56077.log 2024-01-31T05:04:49.509 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52232.log 2024-01-31T05:04:49.509 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59974.log.gz 2024-01-31T05:04:49.510 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55101.log 2024-01-31T05:04:49.510 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56077.log.gz 2024-01-31T05:04:49.511 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44072.log 2024-01-31T05:04:49.511 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52232.log.gz 2024-01-31T05:04:49.513 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.55101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60945.log 2024-01-31T05:04:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44072.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.55101.log.gz 2024-01-31T05:04:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47622.log 2024-01-31T05:04:49.515 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44072.log.gz 2024-01-31T05:04:49.515 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-01-31T05:04:49.516 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60945.log.gz 2024-01-31T05:04:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47622.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47622.log.gz -- 2024-01-31T05:04:49.517 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.56835.log 2024-01-31T05:04:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58559.log 2024-01-31T05:04:49.518 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-01-31T05:04:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2024-01-31T05:04:49.519 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56835.log.gz 2024-01-31T05:04:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61071.log 2024-01-31T05:04:49.520 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58559.log.gz 2024-01-31T05:04:49.521 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42191.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-31T05:04:49.522 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz 2024-01-31T05:04:49.523 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52508.log 2024-01-31T05:04:49.523 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61071.log.gz 2024-01-31T05:04:49.524 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58843.log 2024-01-31T05:04:49.525 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49047.log 2024-01-31T05:04:49.526 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52508.log.gz 2024-01-31T05:04:49.527 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44712.log 2024-01-31T05:04:49.527 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58843.log.gz 2024-01-31T05:04:49.528 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61426.log 2024-01-31T05:04:49.529 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49047.log.gz 2024-01-31T05:04:49.530 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48186.log 2024-01-31T05:04:49.530 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44712.log.gz 2024-01-31T05:04:49.531 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57808.log 2024-01-31T05:04:49.531 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61426.log.gz 2024-01-31T05:04:49.532 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2024-01-31T05:04:49.533 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48186.log.gz 2024-01-31T05:04:49.534 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48491.log 2024-01-31T05:04:49.534 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57808.log.gz 2024-01-31T05:04:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51892.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz 2024-01-31T05:04:49.535 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62084.log 2024-01-31T05:04:49.536 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59748.log 2024-01-31T05:04:49.537 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48491.log.gz 2024-01-31T05:04:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56457.log 2024-01-31T05:04:49.538 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62084.log.gz 2024-01-31T05:04:49.540 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54704.log 2024-01-31T05:04:49.540 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59748.log.gz 2024-01-31T05:04:49.541 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59089.log 2024-01-31T05:04:49.541 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56457.log.gz 2024-01-31T05:04:49.542 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54704.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.54704.log.gz --verbose -- 2024-01-31T05:04:49.543 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/ceph-client.admin.61029.log 2024-01-31T05:04:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48617.log 2024-01-31T05:04:49.544 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59089.log.gz 2024-01-31T05:04:49.545 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50955.log 2024-01-31T05:04:49.545 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61029.log.gz 2024-01-31T05:04:49.547 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41960.log 2024-01-31T05:04:49.547 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48617.log.gz 2024-01-31T05:04:49.548 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60750.log 2024-01-31T05:04:49.548 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50955.log.gz 2024-01-31T05:04:49.549 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48354.log 2024-01-31T05:04:49.549 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41960.log.gz 2024-01-31T05:04:49.551 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60750.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.60750.log.gz -5 2024-01-31T05:04:49.551 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53472.log 2024-01-31T05:04:49.552 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-01-31T05:04:49.552 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48354.log.gz 2024-01-31T05:04:49.553 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49851.log 2024-01-31T05:04:49.553 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53472.log.gz 2024-01-31T05:04:49.555 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61706.log 2024-01-31T05:04:49.555 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-01-31T05:04:49.556 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49851.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49851.log.gz 2024-01-31T05:04:49.556 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-01-31T05:04:49.557 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49425.log 2024-01-31T05:04:49.558 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61706.log.gz 2024-01-31T05:04:49.559 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43658.log 2024-01-31T05:04:49.559 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-01-31T05:04:49.560 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50745.log 2024-01-31T05:04:49.560 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49425.log.gz 2024-01-31T05:04:49.561 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55618.log 2024-01-31T05:04:49.562 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43658.log.gz 2024-01-31T05:04:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50745.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2024-01-31T05:04:49.563 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49583.log 2024-01-31T05:04:49.564 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-01-31T05:04:49.565 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55618.log.gz 2024-01-31T05:04:49.565 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49583.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49583.log.gz 2024-01-31T05:04:49.565 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47850.log 2024-01-31T05:04:49.566 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-01-31T05:04:49.567 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-01-31T05:04:49.567 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47850.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47850.log.gz 2024-01-31T05:04:49.568 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50391.log 2024-01-31T05:04:49.568 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58403.log 2024-01-31T05:04:49.569 INFO:teuthology.orchestra.run.smithi097.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-01-31T05:04:49.570 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50391.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50391.log.gz 2024-01-31T05:04:49.570 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57302.log 2024-01-31T05:04:49.571 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.58403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51268.log 2024-01-31T05:04:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57302.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57302.log.gz 2024-01-31T05:04:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59622.log 2024-01-31T05:04:49.573 INFO:teuthology.orchestra.run.smithi097.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.58403.log.gz 2024-01-31T05:04:49.574 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51268.log: /var/log/ceph/ceph-client.admin.45278.log 2024-01-31T05:04:49.574 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51268.log.gz 2024-01-31T05:04:49.574 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59622.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59622.log.gz 2024-01-31T05:04:49.575 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52785.log 2024-01-31T05:04:49.576 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-01-31T05:04:49.576 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-01-31T05:04:49.577 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.52785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56247.log 2024-01-31T05:04:49.578 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55290.log 2024-01-31T05:04:49.579 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.52785.log.gz 2024-01-31T05:04:49.579 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-01-31T05:04:49.579 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45656.log 2024-01-31T05:04:49.580 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56247.log.gz 2024-01-31T05:04:49.580 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55290.log.gz 2024-01-31T05:04:49.581 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56499.log 2024-01-31T05:04:49.582 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50307.log 2024-01-31T05:04:49.582 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-01-31T05:04:49.583 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56499.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56499.log.gz 2024-01-31T05:04:49.583 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59932.log 2024-01-31T05:04:49.584 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-01-31T05:04:49.584 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50307.log.gz 2024-01-31T05:04:49.585 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59932.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59932.log.gz 2024-01-31T05:04:49.585 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55786.log 2024-01-31T05:04:49.586 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45077.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-31T05:04:49.587 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-01-31T05:04:49.587 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55786.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55786.log.gz 2024-01-31T05:04:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49625.log 2024-01-31T05:04:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60708.log 2024-01-31T05:04:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49625.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49625.log.gz 2024-01-31T05:04:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43988.log 2024-01-31T05:04:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.60708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52316.log 2024-01-31T05:04:49.591 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60708.log.gz 2024-01-31T05:04:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43988.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43988.log.gz 2024-01-31T05:04:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-01-31T05:04:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51850.log 2024-01-31T05:04:49.594 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52316.log.gz 2024-01-31T05:04:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50475.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-01-31T05:04:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61790.log 2024-01-31T05:04:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55518.log 2024-01-31T05:04:49.596 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51850.log.gz 2024-01-31T05:04:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61790.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61790.log.gz -5 2024-01-31T05:04:49.597 INFO:teuthology.orchestra.run.smithi097.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58927.log 2024-01-31T05:04:49.598 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49509.log 2024-01-31T05:04:49.599 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55518.log.gz 2024-01-31T05:04:49.599 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58927.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58927.log.gz 2024-01-31T05:04:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-01-31T05:04:49.600 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56035.log 2024-01-31T05:04:49.601 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49509.log.gz 2024-01-31T05:04:49.602 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44406.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-01-31T05:04:49.602 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48843.log 2024-01-31T05:04:49.603 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45782.log 2024-01-31T05:04:49.604 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56035.log.gz 2024-01-31T05:04:49.604 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48843.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.48843.log.gz 2024-01-31T05:04:49.604 INFO:teuthology.orchestra.run.smithi097.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42103.log 2024-01-31T05:04:49.605 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2024-01-31T05:04:49.606 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45782.log.gz 2024-01-31T05:04:49.607 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42103.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42103.log.gz 2024-01-31T05:04:49.607 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50139.log 2024-01-31T05:04:49.608 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50703.log 2024-01-31T05:04:49.608 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2024-01-31T05:04:49.609 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-01-31T05:04:49.610 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50139.log.gz 2024-01-31T05:04:49.611 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50703.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50703.log.gz 2024-01-31T05:04:49.611 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49341.log 2024-01-31T05:04:49.612 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54037.log 2024-01-31T05:04:49.612 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-01-31T05:04:49.613 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49341.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49341.log.gz 2024-01-31T05:04:49.613 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44711.log 2024-01-31T05:04:49.614 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.54037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45075.log 2024-01-31T05:04:49.615 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44711.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44711.log.gz 2024-01-31T05:04:49.616 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54310.log 2024-01-31T05:04:49.616 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.54037.log.gz 2024-01-31T05:04:49.616 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-31T05:04:49.617 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45075.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45075.log.gz 2024-01-31T05:04:49.617 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54226.log 2024-01-31T05:04:49.618 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47496.log 2024-01-31T05:04:49.619 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54310.log.gz 2024-01-31T05:04:49.620 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58785.log 2024-01-31T05:04:49.620 INFO:teuthology.orchestra.run.smithi097.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54226.log.gz 2024-01-31T05:04:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47496.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47496.log.gz 2024-01-31T05:04:49.621 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45698.log 2024-01-31T05:04:49.622 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43249.log 2024-01-31T05:04:49.622 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58785.log.gz 2024-01-31T05:04:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45698.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45698.log.gz 2024-01-31T05:04:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55744.log 2024-01-31T05:04:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52022.log 2024-01-31T05:04:49.624 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43249.log.gz 2024-01-31T05:04:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45866.log 2024-01-31T05:04:49.625 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55744.log.gz 2024-01-31T05:04:49.626 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48228.log 2024-01-31T05:04:49.627 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52022.log.gz 2024-01-31T05:04:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45866.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45866.log.gz 2024-01-31T05:04:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-01-31T05:04:49.629 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52968.log 2024-01-31T05:04:49.629 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48228.log.gz 2024-01-31T05:04:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-01-31T05:04:49.630 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48659.log 2024-01-31T05:04:49.631 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52968.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-31T05:04:49.631 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52968.log.gz 2024-01-31T05:04:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48659.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48659.log.gz 2024-01-31T05:04:49.632 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57892.log 2024-01-31T05:04:49.633 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36582.log 2024-01-31T05:04:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57892.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57892.log.gz 2024-01-31T05:04:49.635 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61832.log 2024-01-31T05:04:49.636 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44383.log 2024-01-31T05:04:49.636 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36582.log.gz 2024-01-31T05:04:49.638 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61832.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61832.log.gz 2024-01-31T05:04:49.638 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47538.log 2024-01-31T05:04:49.639 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-01-31T05:04:49.639 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44383.log.gz 2024-01-31T05:04:49.640 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47538.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47538.log.gz 2024-01-31T05:04:49.640 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2024-01-31T05:04:49.641 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57052.log 2024-01-31T05:04:49.641 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-01-31T05:04:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51226.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2024-01-31T05:04:49.642 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56541.log 2024-01-31T05:04:49.643 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48759.log 2024-01-31T05:04:49.644 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57052.log.gz 2024-01-31T05:04:49.645 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49383.log 2024-01-31T05:04:49.645 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56541.log.gz 2024-01-31T05:04:49.646 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48759.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48759.log.gz 2024-01-31T05:04:49.646 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2024-01-31T05:04:49.647 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42983.log 2024-01-31T05:04:49.648 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49383.log.gz 2024-01-31T05:04:49.649 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62604.log 2024-01-31T05:04:49.649 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-01-31T05:04:49.650 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53204.log 2024-01-31T05:04:49.652 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48969.log 2024-01-31T05:04:49.652 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62604.log.gz 2024-01-31T05:04:49.652 INFO:teuthology.orchestra.run.smithi097.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.42983.log.gz 2024-01-31T05:04:49.653 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53204.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53204.log.gz 2024-01-31T05:04:49.653 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-01-31T05:04:49.654 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55576.log 2024-01-31T05:04:49.654 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48969.log.gz 2024-01-31T05:04:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45614.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-01-31T05:04:49.655 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45320.log 2024-01-31T05:04:49.656 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55576.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-31T05:04:49.657 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55576.log.gz 2024-01-31T05:04:49.658 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45320.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.45320.log.gz -5 --verbose 2024-01-31T05:04:49.658 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/ceph-client.admin.61916.log 2024-01-31T05:04:49.659 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45092.log 2024-01-31T05:04:49.659 INFO:teuthology.orchestra.run.smithi097.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-31T05:04:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41828.log 2024-01-31T05:04:49.661 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61916.log.gz 2024-01-31T05:04:49.661 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45092.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45092.log.gz 2024-01-31T05:04:49.662 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60347.log 2024-01-31T05:04:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41828.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41828.log.gz 2024-01-31T05:04:49.663 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61958.log 2024-01-31T05:04:49.664 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.60347.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-31T05:04:49.665 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61958.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-01-31T05:04:49.666 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61958.log.gz 2024-01-31T05:04:49.666 INFO:teuthology.orchestra.run.smithi097.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.0.60347.log.gz 2024-01-31T05:04:49.667 INFO:teuthology.orchestra.run.smithi097.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-31T05:04:49.667 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58617.log 2024-01-31T05:04:49.668 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.48018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.47030.log 2024-01-31T05:04:49.668 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-01-31T05:04:49.670 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-01-31T05:04:49.670 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58617.log.gz 2024-01-31T05:04:49.671 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.47030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41685.log 2024-01-31T05:04:49.672 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42147.log: 92.4% -- replaced with /var/log/ceph/ceph-client.2.47030.log.gz 2024-01-31T05:04:49.673 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-31T05:04:49.673 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-01-31T05:04:49.674 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43820.log 2024-01-31T05:04:49.674 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41685.log.gz 2024-01-31T05:04:49.675 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52064.log 2024-01-31T05:04:49.677 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55993.log 2024-01-31T05:04:49.677 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43820.log.gz 2024-01-31T05:04:49.678 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.52064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49751.log 2024-01-31T05:04:49.678 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52064.log.gz 2024-01-31T05:04:49.680 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43532.log 2024-01-31T05:04:49.680 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55993.log.gz 2024-01-31T05:04:49.681 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54268.log 2024-01-31T05:04:49.681 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49751.log.gz 2024-01-31T05:04:49.682 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43532.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43532.log.gz 2024-01-31T05:04:49.683 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43574.log 2024-01-31T05:04:49.683 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54268.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54268.log.gz 2024-01-31T05:04:49.684 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-01-31T05:04:49.685 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56583.log 2024-01-31T05:04:49.686 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43574.log.gz 2024-01-31T05:04:49.687 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47286.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-01-31T05:04:49.688 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56583.log: /var/log/ceph/ceph-client.admin.50433.log 2024-01-31T05:04:49.688 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56583.log.gz 2024-01-31T05:04:49.689 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61510.log 2024-01-31T05:04:49.691 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55660.log 2024-01-31T05:04:49.691 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50433.log.gz 2024-01-31T05:04:49.692 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59580.log 2024-01-31T05:04:49.692 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61510.log.gz 2024-01-31T05:04:49.694 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.55660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57175.log 2024-01-31T05:04:49.694 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55660.log.gz 2024-01-31T05:04:49.695 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62168.log 2024-01-31T05:04:49.695 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59580.log.gz 2024-01-31T05:04:49.696 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57934.log 2024-01-31T05:04:49.696 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57175.log.gz 2024-01-31T05:04:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62168.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62168.log.gz 2024-01-31T05:04:49.698 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54578.log 2024-01-31T05:04:49.699 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57934.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57934.log.gz 2024-01-31T05:04:49.700 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58659.log 2024-01-31T05:04:49.701 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.54578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53146.log 2024-01-31T05:04:49.701 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54578.log.gz 2024-01-31T05:04:49.703 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.58659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49977.log 2024-01-31T05:04:49.703 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58659.log.gz 2024-01-31T05:04:49.704 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.53146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59366.log 2024-01-31T05:04:49.704 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53146.log.gz 2024-01-31T05:04:49.705 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49977.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49977.log.gz 2024-01-31T05:04:49.706 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51640.log 2024-01-31T05:04:49.706 INFO:teuthology.orchestra.run.smithi097.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-31T05:04:49.707 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.59366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59664.log 2024-01-31T05:04:49.708 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.51640.log: 90.0% -- replaced with /var/log/ceph/ceph-client.0.59366.log.gz 2024-01-31T05:04:49.708 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62544.log 2024-01-31T05:04:49.709 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51640.log.gz 2024-01-31T05:04:49.710 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56415.log 2024-01-31T05:04:49.710 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59664.log.gz 2024-01-31T05:04:49.711 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47580.log 2024-01-31T05:04:49.711 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62544.log.gz 2024-01-31T05:04:49.712 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50829.log 2024-01-31T05:04:49.712 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56415.log.gz 2024-01-31T05:04:49.714 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59011.log 2024-01-31T05:04:49.714 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47580.log.gz 2024-01-31T05:04:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.50829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62210.log 2024-01-31T05:04:49.715 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50829.log.gz 2024-01-31T05:04:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.59011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45992.log 2024-01-31T05:04:49.717 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59011.log.gz 2024-01-31T05:04:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.62210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57133.log 2024-01-31T05:04:49.718 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62210.log.gz 2024-01-31T05:04:49.719 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41506.log 2024-01-31T05:04:49.720 INFO:teuthology.orchestra.run.smithi097.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45992.log.gz 2024-01-31T05:04:49.720 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.57133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49467.log 2024-01-31T05:04:49.721 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57133.log.gz 2024-01-31T05:04:49.722 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.41506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43207.log 2024-01-31T05:04:49.722 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41506.log.gz 2024-01-31T05:04:49.723 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.49467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56204.log 2024-01-31T05:04:49.724 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49467.log.gz 2024-01-31T05:04:49.724 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.43207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56667.log 2024-01-31T05:04:49.725 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43207.log.gz 2024-01-31T05:04:49.726 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56204.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-31T05:04:49.726 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56204.log.gz 2024-01-31T05:04:49.727 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44030.log 2024-01-31T05:04:49.727 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56667.log.gz 2024-01-31T05:04:49.728 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61113.log 2024-01-31T05:04:49.730 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.44030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56625.log 2024-01-31T05:04:49.730 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44030.log.gz 2024-01-31T05:04:49.731 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.61113.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61113.log.gz 2024-01-31T05:04:49.732 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45035.log 2024-01-31T05:04:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.56625.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56625.log.gz 2024-01-31T05:04:49.733 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42053.log 2024-01-31T05:04:49.735 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.45035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47976.log 2024-01-31T05:04:49.735 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45035.log.gz 2024-01-31T05:04:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.42053.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42053.log.gz 2024-01-31T05:04:49.736 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36769.log 2024-01-31T05:04:49.737 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.47976.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47976.log.gz 2024-01-31T05:04:49.739 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.36769.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36769.log.gz 2024-01-31T05:04:49.768 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-31T05:04:50.057 INFO:teuthology.orchestra.run.smithi188.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-31T05:04:50.066 INFO:teuthology.orchestra.run.smithi188.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-31T05:04:50.075 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-31T05:04:50.081 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-31T05:04:50.085 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-01-31T05:04:50.085 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m0.870s 2024-01-31T05:04:50.085 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m3.321s 2024-01-31T05:04:50.085 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.416s 2024-01-31T05:04:50.324 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-31T05:04:50.361 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-31T05:04:50.422 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-31T05:04:50.467 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-31T05:04:50.471 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-01-31T05:04:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m1.259s 2024-01-31T05:04:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m3.664s 2024-01-31T05:04:50.471 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m1.377s 2024-01-31T05:04:50.472 INFO:tasks.ceph:Archiving logs... 2024-01-31T05:04:50.473 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/remote/smithi097/log 2024-01-31T05:04:50.474 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-31T05:04:51.341 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/remote/smithi188/log 2024-01-31T05:04:51.342 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-31T05:04:51.912 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-31T05:04:51.924 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-31T05:04:51.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-31T05:04:51.932 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-31T05:04:52.087 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-31T05:04:52.098 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-31T05:04:52.098 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:04:52.106 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-31T05:04:52.154 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:04:52.160 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-01-31T05:04:52.164 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:04:52.164 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-01-31T05:04:52.164 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 +269us[ +269us] +/- 66ms 2024-01-31T05:04:52.164 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -572us[ -567us] +/- 75ms 2024-01-31T05:04:52.164 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +267us[ +267us] +/- 77ms 2024-01-31T05:04:52.164 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:04:52.169 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-31T05:04:52.170 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-01-31T05:04:52.170 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 +300us[ +300us] +/- 66ms 2024-01-31T05:04:52.170 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -555us[ -546us] +/- 75ms 2024-01-31T05:04:52.170 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +297us[ +306us] +/- 77ms 2024-01-31T05:04:52.170 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-01-31T05:04:52.171 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-31T05:04:52.182 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-31T05:04:52.182 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-31T05:04:52.249 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-31T05:04:52.480 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-31T05:04:52.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:04:52.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:52.791 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-31T05:04:52.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:52.896 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-31T05:04:52.905 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-31T05:04:52.919 INFO:teuthology.task.internal:Duration was 837.592665 seconds 2024-01-31T05:04:52.920 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-31T05:04:52.931 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-31T05:04:52.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-31T05:04:52.939 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-31T05:04:53.098 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:04:53.100 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-31T05:04:53.574 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-31T05:04:53.574 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-01-31T05:04:53.574 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-31T05:04:53.632 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-01-31T05:04:53.633 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-31T05:04:53.707 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-31T05:04:53.708 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-31T05:04:53.716 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-31T05:04:53.803 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-31T05:04:53.803 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-31T05:04:53.808 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-31T05:04:53.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-31T05:04:54.006 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-31T05:04:54.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-31T05:04:54.104 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-31T05:04:54.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-31T05:04:54.209 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:04:54.217 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:04:54.317 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-01-31T05:04:54.323 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-01-31T05:04:54.377 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:04:54.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:54.427 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-31T05:04:54.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-31T05:04:54.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-31T05:04:54.491 INFO:teuthology.task.internal:Transferring archived files... 2024-01-31T05:04:54.492 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/remote/smithi097 2024-01-31T05:04:54.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-31T05:04:54.598 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464/remote/smithi188 2024-01-31T05:04:54.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-31T05:04:54.696 INFO:teuthology.task.internal:Removing archive directory... 2024-01-31T05:04:54.696 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-31T05:04:54.703 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-31T05:04:54.755 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-31T05:04:54.766 INFO:teuthology.task.internal:Not uploading archives. 2024-01-31T05:04:54.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-31T05:04:54.781 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-31T05:04:54.781 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-31T05:04:54.789 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-31T05:04:54.838 INFO:teuthology.orchestra.run.smithi097.stdout: 395176 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 31 05:04 /home/ubuntu/cephtest 2024-01-31T05:04:54.845 INFO:teuthology.orchestra.run.smithi188.stdout: 395475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 31 05:04 /home/ubuntu/cephtest 2024-01-31T05:04:54.848 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-31T05:04:54.954 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-31T05:04:54.976 INFO:teuthology.nuke:Checking targets against current locks 2024-01-31T05:04:55.002 DEBUG:teuthology.nuke:shortname: smithi097 2024-01-31T05:04:55.003 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T05:04:55.024 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:39:57.227063', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXT2G+E8eNejApYk1lrMi8RXrp+DyrOrZfxPpm2gF18EHfuxOlRhwe49k84Q9yDGWvuDtpmI+rBg7+KZtiC31g='} 2024-01-31T05:04:55.028 DEBUG:teuthology.nuke:shortname: smithi188 2024-01-31T05:04:55.028 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-31T05:04:55.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-31_04:15:12-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7539464', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-31 04:39:57.225617', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNd0o8lWnbxEkjgzyJ5bQhMY9uRxoEYyIwGAmnnKRxtfWCkzpMyC8jCWqf6zSkFg8YwYsJ2+rBvJ3uOL+Q143XA='} 2024-01-31T05:04:55.070 INFO:teuthology.orchestra.console.smithi097:Power off 2024-01-31T05:04:55.070 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-31T05:04:55.096 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-01-31T05:04:55.097 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:04:55.111 INFO:teuthology.orchestra.console.smithi188:Power off 2024-01-31T05:04:55.112 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-31T05:04:55.121 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-01-31T05:04:55.135 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-01-31T05:04:55.136 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:04:55.158 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-01-31T05:04:59.122 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:04:59.159 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:04:59.234 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-01-31T05:04:59.269 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-01-31T05:05:03.236 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:05:03.270 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:05:03.346 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-01-31T05:05:03.380 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-01-31T05:05:03.480 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-01-31T05:05:07.349 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-31T05:05:07.458 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-01-31T05:05:07.559 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-01-31T05:05:07.664 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/pool-perm} duration: 837.5926647186279 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda mon.b-kernel-sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=d9a11201f3204b21b798c4942adc682f status: fail success: false 2024-01-31T05:05:07.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-31T05:05:07.746 INFO:teuthology.run:FAIL